Drone pilots are getting their flying skills put to the test with FlightGoggles, a simulation powered by NVIDIA RTX technology that delivers a virtual race course inside a photorealistic environment.
Competitors from around the world are participating in Lockheed Martin’s AlphaPilot Challenge, an open challenge to develop an NVIDIA Jetson-powered AI framework for high-speed autonomous racing drones. For the upcoming race, over 1,000 contestants registered and organized into more than 300 teams.
AlphaPilot selected the top nine teams by challenging them to compete in a race course built in the Massachusetts Institute of Technology FlightGoggles simulator.
FlightGoggles was developed last year by the research group of Sertac Karaman, associate professor of aeronautics and astronautics at MIT, to train autonomous drones to fly in complex environments. It utilizes photogrammetry and virtual reality to provide a realistic robotics simulation. For the AlphaPilot Challenge, the team at MIT built an entirely new environment to create a race course for the competing teams.
Using NVIDIA Quadro RTX 8000 GPUs, Karaman and his colleagues created the realistic environment through photogrammetry, the process of building 3D assets using photographs.
RTX Brings Photorealistic Details to Light
The FlightGoggles simulation features high-resolution graphics and 3D models. Its drone racing arena is made up of over 80 unique graphics assets, with 1,000 copies of these used throughout the entire environment.
The team at MIT built the massive environment using real-time graphics engine Unity and NVIDIA Quadro RTX 8000. The Turing-based GPU delivers accelerated performance for computing 3D models and offers the extensive memory needed to create photorealistic renderings for the simulation.
The powerful GPU also enhanced the light-mapping procedure for pre-rendering the effects of sunlight to help calculate the brightness of surfaces in a scene. Using dual Quadro RTX 8000 GPUs, the MIT team was able to immensely speed up the light-mapping procedure — instead of taking days, it was completed in only a few hours.
“We needed a solution that would help us iterate rapidly, but also had a lot of RAM so we could create an environment made up of 40 million triangles. With the Quadro RTX 8000 GPUs, we were able to complete the entire environment in a month,” said Karaman. “The GPUs gave us the powerful performance we needed to keep generating models, iterate quickly and create the simulation faster than we thought possible.”
All the photogrammetry assets that MIT used in FlightGoggles is now available for free on the Unity Asset Store and the FlightGoggles website.
The top nine teams will compete later this year in the AlphaPilot drone race, which ESPN plans to broadcast live.
To learn more about MIT FlightGoggles, check out the video below: