GAN Theft Auto: The Neural Network version of GTA 5

New and interesting developments involving AI and GTA 5 seem to be popping in and out of everywhere as of late, with GAN Theft Auto being the latest in such a series.

The GTA series seems like a perfect video game series for testing AI, making some researchers' fascination with GTA 5 unsurprising. It is a massive game that many people are aware of, and it features people and vehicles with a rich AI system. A while ago, some Intel researchers used machine learning to make GTA 5 seem photorealistic.

There is a new development in testing out AI within GTA 5 from a different source altogether. YouTuber sentdex recently had assistance from NVIDIA's GameGAN research team, where a neural network tries to recreate GTA 5 from memory perfectly.

---

A Neural Network version of GTA 5

The GAN in GAN Theft Auto stands for Generative Adversarial Network, a framework that utilizes machine learning and was developed by Ian Goodfellow and his research team in 2014. It's still a new field of study. Hence, the quality of the video above might not seem top class.

Note: sentdex has a Github listed here for GTA 5 fans, curious to check out a demo that utilizes GAN in GTA 5.

A summary of GAN Theft Auto

GAN Theft Auto currently ranking #3 for Grand Theft Auto in YT search in a private session.Poor souls just lookin for some GTA content to veg out with dunno what they're about to stumble upon. 😂 pic.twitter.com/2RVsHE7Pms

— Harrison Kinsley (@Sentdex) June 19, 2021

Players watching the video above will immediately notice how blurry and pixelated it seems, even at 1080p60. The reason is the computer generating everything based on its neural network. In a way, it's almost like a dream.

The entire GTA 5 is intended to be the neural network for this project. Interestingly, the footage showcases everything being controlled by itself, which then reacts to the player's inputs.

All of this takes place on a single bridge, which might not seem impressive at first glance. However, creating this on a regular computer would be very difficult, considering how GPU-intensive developing the system would be. By comparison. Playing the demo should work on any computer that can handle GTA 5.

GAN Theft Auto development

Developing the AI (Image via sentdex)

With the help of a DGX Station A100 80GB, sentdex needed to collect some data from GTA 5 to make this work. Unfortunately, manually playing GTA 5 was out of the question, as sentdex was only loaned the DGX Station A100 80GB, and it would take months for him to get the data he needs.

Instead, he opts to get some assistance from his friend, Daniel, who then creates a rule-based AI system that drove GTA 5 and collects data simultaneously. Of course, running GTA 5 in one instance wouldn't be enough, so they tried to play GTA 5 in as many instances as possible. In the picture above, there are 12 AI driving around on the same bridge.

After some data was collected, GAN Theft Auto was taking the first steps to become something. Unfortunately, the initial photos were even more pixelated than what the thumbnail would indicate, but it's still an impressive feat of technology nonetheless.

More tests

Daniel and sentdex continued to test out various things (Image via sentdex)

Of course, not all tests were successful. Even with the superb technology they had, some crashes still happened. That said, the overall experiment was a success, and they were able to develop GAN Theft Auto as it exists presently. It will be interesting to see what discoveries sentdex and his friends can make in the future regarding the ongoing developments of AI.

The rest of the YouTube video goes into some more technical aspects about GAN Theft Auto and how it works, as well as how GTA 5 players can test out a demo in his Github.