GANTheftAuto: Harrison Kinsley on AI-Generated Gaming Environments

Kinsley and collaborator Daniel Kukieła created a neural network that generates its own gaming environment modeled on ‘Grand Theft Auto.’
by Angie Lee

Humans playing games against machines is nothing new — think of Deep Blue or AlphaGo. But now, computers can even develop their own games for people to play.

Programming enthusiast and social media influencer Harrison Kinsley spoke with NVIDIA AI Podcast host Noah Kravitz about creating GANTheftAuto, an AI-based neural network that generates a playable chunk of the classic video game Grand Theft Auto V.

Inspired by NVIDIA GameGAN’s recreation of PAC-MAN, Kinsley and his collaborator Daniel Kukieła decided to test how far they could take the idea of training an AI model to create its own gaming environment.

Key Points From This Episode:

  • GANTheftAuto is an AI model that generates features of the Grand Theft Auto gaming environment — from its overall landscape to details like the sunlight reflecting off a car — in real time, as a person plays the game. The model is based on a chunk of training data, not the actual code from the game.
  • Kinsley and Kukieła had to overcome challenges such as gathering enough data to train the model, training it in a reasonable amount of time and getting it to simulate real-life physics.
  • Aside from his own work, Kinsley helps people all over the world with their AI-based projects by posting programming tutorials and explainer videos to his YouTube channel, which is hosted under the name “sentdex” and has over a million subscribers.


“It’s one of those things that gives you goosebumps when you see it … there’s something really cool about an AI-generated environment.” — Harrison Kinsley [5:46]

“I think AI-based software is the future.” — Harrison Kinsley [26:37]

You Might Also Like:

Matt Ginsberg Built a GPU-Powered Crossword Solver to Take on Top Word Nerds

Dr.Fill, the crossword puzzle-playing AI created by Matt Ginsberg — a serial entrepreneur, pioneering AI researcher and former research professor — scored higher than any humans earlier this year at the American Crossword Puzzle Tournament.

Perfect Pairing: NVIDIA’s David Luebke on the Intersection of AI and Graphics

The NVIDIA Research team employs more than 200 scientists around the world who are driving innovation across a range of industries. David Luebke, who founded the team in 2006 and is now the company’s vice president of graphics research, describes how AI and graphics go together “like peanut butter and jelly.”

Cycle of DOOM Now Complete: Researchers Use AI to Generate New Levels for Seminal Videogame

DOOM, of course, is foundational to 3D gaming. 3D gaming, of course, is foundational to GPUs. And GPUs are foundational to deep learning, which is, now, thanks to a team of Italian researchers, being used to make new levels for … DOOM.

Subscribe to the AI Podcast

Get the AI Podcast through iTunes, Google Podcasts, Google Play, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn. If your favorite isn’t listed here, drop us a note.

Make the AI Podcast Better

Have a few minutes to spare? Fill out this listener survey. Your answers will help us make a better podcast.