OpenMythos revives Claude Mythos as a lean looped transformer withMoE depth
OpenMythos is an open-source PyTorch reconstruction of Claude Mythos proposing a Recurrent-Depth Transformer (RDT) that uses a fixed set of weights looped up to 16 times, with a Mixture-of-Experts FFN and Multi-Latent Attention to enable deep reasoning with far fewer parameters. The design re-injects input at each loop, employs stability techniques like Linear Time-Invariant constraints and Adaptive Computation Time, and adds depth-wise LoRA adapters to differentiate loop steps. The project argues that 770M parameters can match a 1.3B transformer when trained on identical data, reframing depth as inference-time computation rather than parameter count. It releases four artifacts: a configurable PyTorch implementation, LTI stability primitives, depth-wise LoRA adapters, and a reproducible loop-dynamics baseline.
