Memory Over Speed: The AI Inference Shift

Ben Thompson argues that the AI compute boom is moving from GPU-dominated training to memory-centric, agentic-inference architectures; Cerebras’ wafer-scale chips offer extraordinary on-chip memory and bandwidth for fast answer inference but face cost and scalability limits, while the long-term potential lies in memory hierarchies that support autonomous agentic work, potentially reducing Nvidia’s dominance and reconfiguring compute across training, inference, and even space data centers.