"Google DeepMind's AI Learns to Collaborate in Video Games Without Prior Experience"
Google Deepmind has developed a multiworld agent, SIMA, trained to play multiple 3D games and understand verbal instructions from human players, without access to the game's internal code or rules. The model was trained on hours of human gameplay videos and annotations, allowing it to associate visual representations with actions and interactions. The AI can generalize to games it hasn't seen, but may struggle with unique mechanics. The researchers aim to create a more natural game-playing companion and have used imitation learning from human behavior, given goals in text, rather than reinforcement learning with a strict reward structure.