把Loopies绕起来！
Loop the Loopies!
摘要
我们推出了Loopie——迄今为止最强大的循环式Transformer模型。Loopie系列包含两种混合专家模型：一个拥有20B个参数的模型，其中2B个参数为活跃参数；另一个则拥有6B个参数，其中0.6B个为活跃参数。循环式Transformer一直面临一个挑战：当预训练计算资源增加N倍时，直接增加参数数量通常比将模型循环N次效果更好。Loopie解决了这一问题。通过广泛的消融实验，包括与普通的30B-A3B模型进行对比后可以发现，Loopie在相同的计算资源下表现显著优于普通Transformer模型。我们的新型训练后处理流程使得Loopie具备强大的推理能力。在2025年的IMO和IPhO竞赛中，Loopie无需任何辅助工具就能获得金牌成绩。
English Abstract
We present Loopie, the most powerful looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6Bparameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N-fold increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times. Loopie addresses this challenge. Extensive ablation studies, including comparisons with a vanilla 30B-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget. Our novel post-training pipeline equips Loopie with strong reasoning abilities. At the 2025 IMO and IPhO, Loopie achieves gold-medal performance without tools.