Open Source Robotics
When the AWS DeepRacer team approached us for ideas to promote their new Open Source Software initiative, we didn’t disappoint. DeepBlaster is a software and hardware package that extends the AWS DeepRacer platform.
AWS DeepRacer is a 1/18 scale autonomous self driving car. Originally launched at AWS re:Invent 2018, the purpose of DeepRacer is to provide a fun way for software developers to learn Machine Learning (ML) principles, thus making it easier to grow the pool of Data Scientists which are in high demand.
The Problem
The AWS DeepRacer car requires a large track to be constructed to test your “self driving” ML model. The cost and effort of building or buying a track was a barrier for entry for many developers. AWS needed a way for people to use the car without a track.
The Solution
Releasing the DeepRacer software as Open Source would allow people to find new ways to use the car. Now they just needed some ideas to fuel the creative juices of this new pool of would-be Data Scientists. AWS approached a few of us in the DeepRacer community to brainstorm over what kind of sample projects could be developed.
TL;DR
New Ideas
Cloud Brigade’s CEO Chris Miller has been involved in the DeepRacer Community since he won the first AWS Summit race in 2019. Since then Cloud Brigade has been working with Artificial Intelligence (AI) product teams at AWS to leverage and showcase the growing collection of ML Devices and Microservices.
Chris already established a reputation for out-of-the-norm ideas, including ThePoopinator. When asked for a solution to this problem, Chris suggested mounting a toy “blaster” to the car, and to use AI to identify, target, and “neutralize” inanimate objects. What Chris didn’t realize was that AWS was going to ask Chris to build this solution in record time, just a few weeks before the launch of the DeepRacer OpenSource project.
The Blaster
Just days earlier, Chris had read an article on The Verge about Luke Goodman (aka Out of Darts) who built a small business around his love for “Nerf Modding”. Out of Darts 3D prints and sells parts to modify or build your own “Blaster” which shoots Nerf Rival ammo, little foam balls. Blasters are used in competitive games and events, or just for fun.
We reached out to Luke to learn more about how the Blaster works and to source parts. We decided to use the Out of Darts “Jupiter Blaster” for our prototype.
The Robot Operating System
Next we had to dive into the DeepRacer Open Source project which is built on ROS2, an operating system designed to build robots. ROS2 comprises a series of independent software “nodes” which work together to provide a larger system. Each node is capable of communicating with each other using a system of “topics” which the nodes can “publish” or “subscribe” to, similar to the way IoT devices use Mosquitto (MQTT).
AWS provides a few sample projects, and we decided to use the Follow The Leader (FTL) project as inspiration, allowing us to leverage some existing code and focus on the new functionality we needed to create. FTL allows the DeepRacer car to identify a person using Object Detection, and to follow that person as they walk around a room.
FTL provided functionality examples that were closely related to our needs, including :
- Object detection and tracking
- Action mapping
- Servo/motor control
- Service and Message definitions
Designing A Robot
Now that we were familiar with ROS2 and FTL, we set out to design our own system which would require software, a microcontroller, electronics, and custom 3D printed parts. We started with how to break up the functionality of DeepBlaster into the necessary software nodes :
- Object Detection node
- Targeting node
- Blaster Control node
- Interfaces node
Using the existing Object Detection node to provide X,Y coordinates of a “target”, we focused on the Targeting node which is responsible for telling the DeepBlaster where to aim the turret, and when to fire the blaster. While this is similar to the Navigation node in the FTL example, the action space is more dynamic. That is to say the target location changes as the car is in motion, and the turret will need to make constant corrections before firing.
We created a very simple servo controlled camera mount to build our initial Targeting node, our Jupiter Blaster was on backorder and we needed to make progress. Initially we used the DeepRacer’s “steering control” port for our servo. Unfortunately the DeepRacer hardware was not designed to support additional servos, and we quickly decided to build our own Blaster Control Module using an Arduino microcontroller.
With a targeting node defining an “action space”, a series of actions to take when a target was detected, we started working on the DeepBlaster Control node which would then communicate with the Arduino. This node would subscribe to the Blaster Control topic that is published by the Targeting node.
We could have just built the Blaster Control logic into the Targeting node, but we felt that the hardware limitation we were solving warranted a separate node which could be used by others to extend DeepRacer in other ways, including adding sensors, servos, and other device control.
In order to use an Arduino with the DeepRacer, we were really limited to using USB as a connection. This meant we needed to create our own serial communication protocol, a stable and fast method of exchanging messages between devices. We figured there had to be an existing method that made sense, and we found the Robust Serial library filled that need.
One issue with using Computer Vision on a moving device is motion induced blur. The native method of controlling servos is to move the servo to a position as quickly as possible, and that was going to cause blur. After experimenting with ideas to have more precise control over the servo motion, we found the Servo Trajectory library, which allows us to control acceleration, deceleration, and time to position. In theory, this will help us combat blur.
Building a Turret
Now that we had a rough software prototype, we needed a custom turret that would allow us to mount the Jupiter Blaster to the DeepRacer car. We used the DeepRacer EVO’s Lidar bracket for inspiration as it conveniently fit right on top of the battery, and would provide a foundation for the turret.
We also found a 3D model on Thing-a-verse which provided a micro-servo controlled flange with a 1:23 gear ratio, allowing a 180 degree hobby servo to rotate 360 degrees. We worked with Idea Fab Labs, a maker space next door to our office, to help us customize the 3d model for a prototype turret for DeepBlaster.
Blaster Electronics
Mounting the Jupiter Blaster to the turret platform was easy (Hot Glue to the rescue!), but controlling the blaster was another complex hurdle. The blaster uses a pair of high speed “flywheels” to propel the foam balls, just like a baseball pitching machine. The flywheels must be “at speed” before you feed the ammo, and if you jam them, the motors pull 28A (amps) of current!
Initially we experimented with USB controlled relays, but those top out at 10A, and the DeepRacer has limited USB ports. We decided to use 30A Mosfets, a high current transistor frequently used in amplifiers. This was overkill for the 1A ammo “feeder” motor, but convenient.
We modified the blaster wiring diagram to use the Mosfets instead of the included micro-switches, and designed a cabling system to allow the blaster to rotate independently of the DeepRacer with limited drag from the wiring.
Assembling DeepBlaster
Reassembling more accurately describes the process of building a prototype like DeepBlaster, over and over. We broke a Servo, blew out a Mosfet, and went through several sticks of Hot Glue. We also lost a lot of ammo somewhere in the maker space. Did we mention DeepBlaster can fire 15 rounds per second?
With DeepBlaster assembled, we were able to test the system, make software refinements, write documentation, make a promo video, and our favorite part – Open Source our DeepBlaster software.
OK, But Why???
Why would we put so much effort into building a DeepBlaster? Simply put – we love a good challenge. Cloud Brigade is a boutique technical consultancy which transform our client’s vision into a well designed reality. Oh, and our CEO has an inner twelve year old that couldn’t resist the opportunity.
Let’s address the elephant in the room – we just weaponized Artificial Intelligence, and that raises significant ethical questions about how we use AI. While we take this topic very seriously, this project was done as a tongue-in-cheek fun way to play with toys, and to inspire others to use this technology in ways that are actually helpful to humanity.
AWS DeepRacer and the DeepBlaster Open Source projects provide a hardware and software platform in which aspiring software developers can build a working robotic system to accomplish other tasks. We were reminded of how robots are used in unsafe situations to collect information and take action while protecting human life. Such examples include bomb detecting, radiation measurement, and search and rescue.
DeepBlaster can also support the AWS DeepRacer Community by providing new ways to compete, such as races which are scored on activity completion in addition to speed around a course.
Take My Money!
So you want a DeepBlaster? If so, that’s AWESOME! We want to hear from you! The DeepBlaster hardware exists as a prototype, and in the furious process of building DeepBlaster in three weeks, we identified several weak spots and areas for improvement. We’re currently working on a revised turret system with a stronger servo, lighter in weight, and potentially a smaller/lighter blaster. If there’s enough interest, we would be happy to provide a DeepBlaster kit with everything needed to build your own.
In the meantime, we recommend you check out our GitHub projects here, install them on AWS DeepRacer and start playing with the software.
What’s Next
If you like what you read here, the Cloud Brigade team offers expert Machine Learning as well as Big Data services to help your organization with its insights. We look forward to hearing from you.
Please reach out to us using our Contact Form with any questions.
If you would like to follow our work, please signup for our newsletter.