Through the looking glass: Generative AI is mostly known for creating alarmingly believable text and static images, and the tools have begun to assemble fairly realistic video clips. Interactive experiences, including games, might be one of the next steps. Google’s research could enable AI to impact the construction of virtual worlds, using AI to train AI.
Google DeepMind recently unveiled Genie, an AI model that can make small interactive games from static images. Google says the basis behind the technology can facilitate other applications, like training AI-powered robots.
Genie was trained on around 200,000 hours of publicly available gameplay footage (likely from Google subsidiary YouTube). It’s unclear whether anyone using it could run into copyright issues with the creators of those games.
Remarkably, the AI doesn’t require text or action tags to interpret what it sees. It works solely from video frames, deciphering which elements should act as terrain and what should act as a controllable avatar. Genie can also interpret three-dimensional depth in images to emulate parallax scrolling in the resulting game.
The tool can build small video game levels from paintings, sketches, renders, photos, and other kinds of images. Because Genie can work with pictures it has never seen before, it can also build environments from AI-generated content, opening the prospect of using content from one AI to create something in another AI.
However, there is room for improvement. Like other AI models, Genie is capable of hallucinating. Furthermore, this initial iteration can only remember up to 16 frames, severely limiting the size of the levels it can build while remaining consistent. The model’s 1fps operation speed also leaves much room for improvement.
To ensure accuracy, Genie’s initial demonstration focuses on replicating 2D platform games. However, Google suggests it could be trained to create other types of games and virtual content.
For example, Google said that Genie can help train certain robots more effectively. Training using human-made simulations provides a limited variety of scenarios, but Genie can build a far broader range of virtual environments and objects for the robot to learn from.
Through further development, Genie could turn interactive set pieces into the next major field for generative AI. Videos are currently becoming a central focus, especially since OpenAI unveiled Sora last month, which can create realistic 60-second 1080p clips. Unsurprisingly, the AI-generated footage isn’t perfect, but it represents a significant leap forward for the technology.
