Age is just a number. And nothing proves that adage better than our latest group of high school “Jetson” interns, who spent eight weeks using deep learning and neural networks to build robots that may one day be used on our campus in Santa Clara.
These talented students from local high schools brought their passion for robotics to our embedded group, which focuses on intelligent machines like robots and drones.
We divided them into three teams and tasked them with building Jetson-powered robots that perform various functions for business facilities. But it wasn’t all work — the Jetson interns also stuffed themselves with popcorn and donuts and attended our speaker series with NVIDIA execs.
Their three projects — involving humanoid robots, projector lights and 2D-laser range finders — are the culmination of weeks of learning, teamwork and laughter.
Team Neural Ninja
Maddie Waldie, Nikhil Suresh and Jackson Moffet from Team Neural Ninja programmed a humanoid robot to recognize and respond to gestures such as a wave. The team trained over 300 neural networks to weed out false positives, where the robot would accidentally respond to a background figure’s movement.
The hardest part of the project, according to Nikhil, was finding the right neural network that could both recognize gestures and ignore background noise. After hours of trial and error, they ultimately succeeded — the robot can now recognize “sorry” in sign language, an arm gesture “X” for no and a “Y” for yes. It can also speak and move, which includes doing a dab — surprising many passersby.
“It was hard at times, when there was a constant cycle of training and getting new data and seeing it not work. But when we finally got a good network, it was such a happy moment,” Maddie said. “Once you get something that works, it’s all worth it.”
Team CCCC (ForeSee)
Tasked with the responsibility of programming a robot to avoid complex obstacles, Team CCCC spent eight weeks learning about the intersection of computer vision and deep learning.
Currently, robots are often stumped by obstacles such as mesh railings, which can’t be detected by sensors that use reflection or 2D-laser range finders. Most solutions are costly, but Rahul Amara, Josh Hejna, Mokshith Voodarla and Anish Singhani developed a solution using inexpensive cameras and deep learning.
To program a robot to navigate between any two points and successfully avoid all obstacles, Team CCCC trained a deep neural network on images of objects such as railings and stairs.
The challenge, then, was designing software that would be both efficient and accurate. With the help of NVIDIA’s development platform, the four succeeded in finding a network that optimized both parameters.
“One of the advantages of working here at NVIDIA is being able to communicate with the people who developed the projects and have more years of experience,” Rahul said. “Whenever we ran into issues, we knew we had a network of people we could talk to and learn from.”
Team GreenMachine
Team GreenMachine, composed of Shruthi Jaganathan, Isaac Wilcove and Karly Hou, developed a Jetson TX2-powered waste classification robot to teach NVIDIANs where their various leftovers, utensils, cups and plates should go.
The robot, which consists of a projector head on top of a mobile cart, will be implemented at cafés in our Santa Clara offices. The projector shines different colored lights — purple for reusable, blue for recyclable, green for compostable and orange for trash — informing the user where to put each item.
Camera calibration was the most difficult part of the project, according to Shruthi. The sensors would often be confused by the color from the projector light, which made it more difficult to detect the texture of the object. For example, purple light on a plastic plate might confuse the camera into thinking that the plate was compostable. However, the team trained the network by adding hundreds of images including color, and ultimately, the model was successful.
“I’m really thankful to have been an intern here for two years — I learned everything I know about deep learning at NVIDIA,” Isaac said. “Now, I know I want to do this kind of work in the future.”
All of these student projects can be found on GitHub.