An 8-bit plumber with a mustache and a penchant for mushrooms was the height of groundbreaking entertainment in 1985. Now, an AI model can recreate the entire game based on a set of basic commands (and sometimes hilarious glitches in the process).
The MarioVGG demonstrated in a new scientific paper from video game character developer Virtuals Protocol shows how AI could contribute to collaboration in AI-driven video game production in the future—or perhaps why it shouldn’t.
MarioVGG is an experiment in getting AI to produce believable gameplay videos based on a series of prompts about how environments should look and behave, and how characters should act. It’s a long-form version of the kind of text-to-video tools that are quickly becoming popular. It was trained on over 737,000 frames of Super Mario Bros. gameplay footage.
However, the AI Mario doesn’t do everything the character does in the Nintendo game, as the researchers limited the model’s interaction capabilities to just two movements; “run right” and “run right and jump”. The resulting images form a video, although it suggests a player is deliberately making it more challenging by avoiding power-ups, only jumping to one height, or moving to the left.
Below you can see a bit of MarioVGG’s performance. It’s simultaneously astonishingly good at recreating the look of the game, but also riddled with glaring errors and flaws. While the AI generated sequences are largely coherent in matching a user’s input with the way Mario acts, it’s definitely not at the same speed, with each frame taking a little bit of time for the AI to create. Not to mention Mario occasionally disappearing from the screen or transforming into an enemy character for a while.
Super MarAIo Modeling
Still, seeing the AI demonstrate a partial understanding of the cause and effect between user input and resulting gameplay brings potential code-free video game development a lot closer to reality. AI models like MarioVGG aren’t going to replace video game developers or the standard engine any time soon, but the idea of simply explaining to an AI how you want a game’s physics and environments to function instead of coding everything by hand is far too tempting to be deterred by technical hurdles.
There is still much to solve, as video games involve a constantly evolving set of interactions between the player and the game environment. This is much more complex than static images or simple actions in a video. Accurately recreating this interactivity in real time is a challenge that MarioVGG has not yet solved – not that MarioVGG is the only one doing so.
Google’s GameNGen recently showed off a playable version of Doom produced by an AI model. Despite being an older game, Super Mario Bros. requires more nuanced control over character movement and environmental interaction than Doom, so the result isn’t as fast or as precise. AI is currently much better at text-based games with occasional illustrations than it is at something like a Mario game. But as in the early days of video games, an evolution toward the fast-paced, interactive video games of Nintendo’s golden age could come faster than any Goomba.
