Android Bench Expands with Eight New LLMs as Harbor Framework Rolls Out
Google’s Android Bench adds eight new LLMs (including Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus, Qwen 3.7 Max) and switches to the Harbor testing framework, placing Gemini 3.1 Pro in fifth behind GPT-5.4 and Claude models; Fable 5 posts the top accuracy (~84.5%) but at a high cost per run, while Gemini 3.1 Pro is cheaper (~$87) and Gemini 3.5 Flash runs the most expensive and longest; Google invites developers to run benchmarks and submit results via GitHub to shape future Android Bench tasks.