The old SIMA was good at following instructions, but the second version now has access to Gemini models and can explore 3D worlds on its own, with zero advance training.
That’s great for video games, where it can think and perform complex reasoning around its goals.
Learning from concepts
It can also learn across games, taking cues from «mining» in one game and transferring it to «harvesting» in another, meaning it can iterate and get better over time.
While it plays or navigates, it can talk and listen to the user, explaining in natural language what it’s doing and taking instructions on its targets.
Good for training robots
This is great for «embodied AI» (aka robots), Google says, as they now have an engine that can «perceive, understand and take action in complex, interactive 3D environments.»
Learning from games and transferring knowledge between them also makes for a self-improving model that can perform increasingly complex tasks, Google says — which is a «major milestone» in training agents in generated worlds.
The model lives inside a research lab at Google for now, as a way to pave the path to general intelligence, but one day it could not only solve your trickiest games for you — but navigate the real world learned from simulations in, say, Genie 3.
Read more: Google’s announcement. Tweets from DeepMind, Demis Hassabis. Discussion on r/Singularity.