FreeToken Lets 753B GLM-5.2 Run on a Single Desktop GPU
Researchers unveil FreeToken, an edge-native Mixture-of-Experts serving engine that maps model state and computation to a user’s hardware, enabling massive frontier models like 753B GLM-5.2 to run on a single workstation GPU. Demonstrations show 35B on an 8 GB laptop GPU, 284B on a gaming desktop, and the full 753B on a single workstation GPU, with OpenAI/Anthropic-compatible endpoints and a Windows/Linux desktop app. FreeToken achieves 1.5–2.3× decode throughput over llama.cpp, Ollama, and KTransformers, thanks to bandwidth-aware scheduling, semantic-aware caching, and elastic memory management that keep results exact and avoid heavy CPU offloads. It's Apache-2.0, pip-installable (freetoken), and designed for solo developers, SMBs, and regulated workloads where data stays on-device; targeted use cases include private code analysis, offline contract review, synthetic data, and local agent workloads.
