不稳定的特性、可复现的子空间：理解稀疏自编码器中种子依赖性的问题
Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders
摘要
稀疏自编码器具有特征稳定性特点：稳定的特征能够携带大量预测性信息，而不稳定的特征则反映出可复现的低维结构——尽管这些不稳定特征在个体层面上无法被复现。由Qwen/Qwen2.5-Coder-32B-Instruct生成的稀疏自编码器被广泛用于解释神经网络的特征表示，但其实用性取决于所学习到的特征是否能够在不同的训练过程中被复现。我们通过特征稳定性来研究这个问题：对于每一个自编码器特征，我们估计在独立训练的模型中重新出现类似特征的概率。这样就能得到一个可扩展的特征信号，从而区分出稳定与不稳定的特征。在大规模研究中，我们发现模型、层、词典大小以及自编码器变体等因素都会对特征稳定性产生影响。结果显示，稳定特征能够携带大部分与重建和预测相关的信号，而不稳定特征则影响力较弱，且在激活统计和自动解释中更多地受到低频表面结构的驱动。从几何角度来看，不稳定的特征虽然个体层面无法被复现，但它们集中在可复现的低秩子空间中，这表明模型的依赖性往往反映了激活空间中的基础模糊性，而非纯粹的噪声。通过可控的合成模型，我们可以明确这一机制：低秩的真实特征可以在子空间层面被恢复，同时仍然无法作为独立的自编码器潜变量被识别出来。最后，通过整合不同种子样本中的独特特征，我们可以构建出更加稳定的自编码器，同时保留所解释的方差。总的来说，这些结果表明，不稳定的特征并非仅仅是失效或噪声的潜在特征：它们虽然个体层面的功能影响较弱，但却反映了可复现的低维结构，而标准自编码器在不同种子样本上的处理方式也有所不同。
English Abstract
Sparse autoencoders exhibit feature stability patterns where stable features carry most predictive signal while unstable features reflect reproducible low-dimensional structure despite individual non-reproducibility. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Sparse autoencoders (SAEs) are widely used to interpret neural network representations, but their utility depends on whether the learned features are reproducible across training runs. We study this question through feature stability : for each SAE feature, we estimate the probability that a similar feature reappears in an independently trained SAE. This yields a scalable per-feature signal that separates stable from unstable features. In a large-scale study across seeds, models, layers, dictionary sizes, and SAE variants, we find a pronounced functional asymmetry: stable features carry most of the reconstruction- and prediction-relevant signal , while unstable features have weak marginal impact and are dominated by low-frequency surface-form triggers in both activation statistics and automatic explanations . Geometrically, unstable features are individually non-reproducible but concentrate in reproducible lower-rank subspaces, suggesting that seed dependence often reflects basis ambiguity within a shared region of activation space rather than pure noise. A controlled synthetic model makes this mechanism explicit, showing that low-rank ground-truth features can be recovered at the subspace level while remaining non-identifiable as individual SAE latents across seeds. Finally, by pooling unique cross-seed features , we construct more stable SAEs while preserving explained variance in this setting. Together, these results show that unstable features are not merely failed or noisy latents: they have weak individual functional impact, but reflect reproducible low-dimensional structure that standard SAEs resolve differently across seeds.