将潜在思维形式化：大语言模型中的四种思维表示公理
Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMs
摘要
我们提出了一个关于LLM中潜在思维表示的公理式评估框架，该框架包含一些与下游基准分数无关的评估指标，这些指标能够揭示那些被基准测试精度所掩盖的表示缺陷。现有的评估方法将表示质量与模型能力混为一谈。因此，这些失败现象并非源于表示本身，而是源于处理这些表示的模型。我们定义了四个功能公理：因果关系、最小性、可分离性以及稳定性，并为每个公理制定了相应的定量衡量标准，这些指标是直接在表示上计算的，与下游的准确率无关。我们对23种推理任务中的开放权重LLM进行了评估，例如空间推理、事实问答等。我们发现，没有任何一个模型同时满足这四个公理；这些表示能够可靠地区分不同任务类型，但无法区分同一任务中的两个问题；此外，这些表示所编码的信息远远少于输入嵌入中已有的信息。这种失败现象在各种密集化、经过推理优化的以及通过强化学习训练的模型中都存在，这表明这种差异属于结构性的，而非模型规模或训练方法的特性。
English Abstract
We introduce an axiomatic evaluation framework for latent thought representations in LLMs, comprising metrics that are independent of downstream benchmark scores and reveal representational failures that benchmark accuracy masks. Existing evaluations conflate representation quality with model capacity. Therefore, failures cannot be attributed to the representation rather than to the model that processes it. We formalize four functional axioms (Causality, Minimality, Separability, and Stability) and define a quantitative measure for each, computed directly on the representation independently of downstream accuracy. We audit open-weight LLMs across 23 reasoning tasks (e.g., Spatial Reasoning, Factual QA). We find that no candidate satisfies all four axioms simultaneously, that the representations distinguish task type reliably but cannot distinguish between two questions within the same task, and that the representations encode little information beyond what is already present in the input embedding. The failure is consistent across dense, reasoning-distilled, and RL-trained model families, indicating that the gap is structural rather than a property of model size or training procedure.