现在，谁应该负责解码任务呢？我们需要找到可靠的路径来整合那些被隐藏起来的扩散语言模型。
Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models
摘要
带面具的扩散语言模型（Masked Diffusion Language Models，MDLMs）作为一种独特的序列生成方法而脱颖而出。随着MDLMs在能力及知识覆盖方面不断拓展，一个重要的问题就是如何整合它们的知识资源。为此，我们首先研究了MDLMs独特的解码机制。我们发现，成功的生成过程会在与答案相关的位置上表现出稳定的置信度变化；而那些不可靠的生成过程则可以通过引入其他模型的可靠信息来加以修正。基于这一发现，我们提出了TIE（基于轨迹的迭代融合）框架——一种知识融合方法。在该方法中，MDLMs会迭代地识别可靠的解码路径，并将这些信息传递给其他模型。TIE能够跟踪与答案相关位置的置信度变化，从而确定哪个模型当前遵循更可靠的路径，并选择性地将部分去噪后的序列传递给不同模型。由于处于更有前景路径上的模型在去噪过程中可能会发生变化，因此TIE允许不同模型在生成的不同阶段发挥互补作用。从各种推理任务中的出色表现来看，我们的分析表明，TIE确实为MDLM的融合问题提供了一种实用的解决方案。
English Abstract
Masked Diffusion Language Models (MDLMs) have emerged as a distinct paradigm for sequence generation. As MDLMs become diverse in capabilities and knowledge coverage, an important question is how to combine their knowledge. Toward this, we first investigate the unique decoding dynamics of MDLMs. We find that successful generations exhibit stable confidence dynamics over answer-relevant positions, while unreliable trajectories can often be corrected by injecting promising intermediate states from other models. Guided by this observation, we propose TIE (Trajectory-based Iterative Ensembling), a knowledge fusion framework in which MDLMs iteratively identify reliable decoding trajectories and relay them across models. TIE tracks confidence dynamics over answer-relevant positions to determine which model currently follows a more reliable trajectory and selectively transfers partially denoised sequences across models. As the model on the more promising trajectory often changes across denoising steps, TIE allows different models to contribute complementary strengths at different stages of generation. Strong performance across diverse reasoning tasks, along with our analyses, suggests that TIE offers a practical approach to the underexplored problem of MDLM ensembling.