Google DeepMind’s New AI Agent Learns, Adapts and Performs Video games Like a Human – Decrypt

Google DeepMind launched SIMA 2 on Thursday—a brand new AI agent that the corporate claims behaves like a “companion” inside digital worlds. With the launch of SIMA 2, DeepMind goals to advance past easy on-screen actions and transfer towards AI that may plan, clarify itself, and study by expertise.

“It is a vital step within the path of Synthetic Normal Intelligence (AGI), with essential implications for the way forward for robotics and AI-embodiment on the whole,” the corporate stated on its web site.

The primary model of SIMA (Scalable Instructable Multiworld Agent), launched in March 2024, realized a whole lot of fundamental abilities by watching the display screen and utilizing digital keyboard and mouse controls. The brand new model of SIMA, Google stated, takes issues a step additional by letting the AI suppose for itself.

“SIMA 2 is our most succesful AI agent for digital 3D worlds,” Google DeepMind wrote on X. “Powered by Gemini, it goes past following fundamental directions to suppose, perceive, and take actions in interactive environments–which means you possibly can speak to it by textual content, voice, and even photographs.”

Through the use of the Gemini AI mannequin, Google stated SIMA can interpret high-level targets, speak by the steps it intends to take, and collaborate inside video games with a stage of reasoning the unique system couldn’t attain.

DeepMind reported stronger generalization throughout digital environments, and that SIMA 2 accomplished longer, extra advanced duties, which included logic prompts, sketches drawn on the display screen, and emojis.

“On account of this potential, SIMA 2’s efficiency is considerably nearer to that of a human participant on a variety of duties,” Google wrote, noting that SIMA 2 had a 65% process completion charge, in comparison with 31% by SIMA 1.

The system additionally interpreted directions and acted inside totally new 3D worlds generated by Genie 3, one other DeepMind mission launched final 12 months that creates interactive environments from a single picture or textual content immediate. SIMA 2 oriented itself, understood targets, and took significant actions in worlds it had by no means encountered till moments earlier than testing.

“SIMA 2 is now much better at finishing up detailed directions, even in worlds it is by no means seen earlier than,” Google wrote. “It may switch realized ideas like ‘mining’ in a single sport and apply it to ‘harvesting’ in one other—connecting the dots between comparable duties.”

After studying from human demonstrations, researchers stated the agent switched into self-directed play, utilizing trial and error and Gemini-generated suggestions to create new expertise information, together with a coaching loop the place SIMA 2 generated duties, tried them, after which fed its personal trajectory information again into the following model of the mannequin.

Whereas Google hailed SIMA 2 as a step ahead for synthetic intelligence, the analysis additionally recognized gaps that also have to be addressed, together with combating very lengthy, multi-step duties, working inside a restricted reminiscence window, and going through visual-interpretation challenges frequent to 3D AI methods.

Even so, DeepMind stated the platform served as a testbed for abilities that might ultimately migrate into robotics and navigation.

“Our SIMA 2 analysis gives a robust path in direction of functions in robotics and one other step in direction of AGI in the actual world,” it stated.