Toyota Exec Explains Why Simulation Key to Autonomous Driving

by Brian Caulfield

Imagine strapping an enormous VR headset to a car. Now, place that car inside a facility the size of a football field that simulates a full range of motion.

Toyota Research Institute CEO Gill Pratt provided a peek inside that incredible facility during a keynote speech to a crowd of more than 3,000 at our GPU Technology Conference in Silicon Valley Thursday.

Pratt also announced that Toyota will be opening a new autonomous driving research facility in Ann Arbor, Michigan, that will be staffed by about 50 researchers to complement Toyota’s ongoing efforts in Japan, Silicon Valley and Cambridge, Mass., near the Massachusetts Institute of Technology.

Pratt is a legendary figure in the robotics community. He joined Toyota last year from the U.S. Defense Advanced Research Projects Agency — the emerging technology arm of the U.S. Defense Department. In January, the world’s largest automaker announced it would invest $1 billion into artificial intelligence over the next five years.

With visuals powered by NVIDIA GPUs, Toyota’s simulation work is the key to taking the technologies developed by these research teams and rolling them out to Toyota’s vast fleet of automobiles.

GPU-powered deep learning technology for autonomous systems can help them learn from huge quantities of real-world data.


Call it the trillion mile problem. Pratt explained that, after he was publicly named CEO of Toyota Research, in January, he couldn’t sleep. That’s because when he did the math — there are about 100 million Toyotas on the roads, each driving about 10,000 miles annually — he realized that Toyota cars are driven about 1 trillion miles per year.

It would only take a few defects to cause enough accidents during those trillion miles of driving to cause an “existential crisis.” That makes rolling out new technologies, particularly technologies that take over for the driver, daunting.

Part of the solution to that problem is simulation at facilities like the one Pratt showed off at GTC. Another part of the answer is to think of autonomy in new ways.

That’s because, if you’re going to hand off complete control to a car, making it a chauffeur, you have to be confident it can perform perfectly. Another wrinkle: the so-called “handoff problem.” If a car gets overwhelmed while you’re dozing or checking your smartphone, will the driver take over driving quickly enough?

By contrast, parallel autonomy, an idea robotics researchers have been pursuing for years, enables autonomous systems to improve safety, without needing to be as perfect. Such systems are “always on,” and ready to take over if the driver gets in trouble. Think of the way an adult teaches a child to swing a golf club, Pratt explained.

Pratt calls this the “guardian angel,” model. Technology can take over, to help swerve at the last moment to avoid a collision, and stay in the background the rest of the time. It can save lives, without having to be as perfect as the “chauffeur model.”

Another upside is the “guardian angel” model preserves what researchers call agency, or the sense that you’re in control of your technology. That’s part of what people have always found thrilling about cars, Pratt said.

Simulation, however, is key to understanding how car and driver will work together as more such features are introduced. That’s why Toyota’s NVIDIA-powered simulation facility is just the start of our work on simulation.

The idea is to use GPUs — and deep learning — to train autonomous systems to experience real-world situations before they’re put on real-world roads.

“If you think about this ‘guardian angel’ model, the car is going to push back at you, it may warn you with beepers and lights, but in the end the guardian angel will grab your wheel,” Pratt says. “How are you going to respond? What are you going to do when the car is having a skid?”

The stakes couldn’t be higher. Even small improvements in safety can have a huge result when spread across Toyota’s vast fleet of vehicles, and the auto industry as a whole.

“1.2 million people,” Pratt said, referring to the number of people killed every year in auto accidents, “deserve nothing less.”