PerceptionDLM：基于多模态扩散语言模型的并行区域感知技术
PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models
摘要
多模态大语言模型在视觉理解任务上取得了显著进展。不过，目前大多数多模态大语言模型都采用自回归生成方式，这限制了它们在需要同时处理多个区域的感知任务中的效率。在本研究中，我们提出了PerceptionDLM这一多模态扩散语言模型，该模型专为高效的并行区域感知而设计。基于PerceptionDLM-Base这一性能出色的基准模型，我们的架构充分利用了大语言模型的并行解码特性。具体来说，我们采用了高效的提示技术和结构化注意力掩码机制，使得模型能够同时处理多个被遮罩的区域，从而能够在序列和令牌层面同时生成各区域的描述信息。与那些按顺序处理各个区域的现有方法相比，这种设计显著提高了推理效率。为了系统评估大语言模型在视觉感知方面的并行能力，我们构建了新的Parallel Detailed Localized Captioning Benchmark（ParaDLC-Bench）模型，该模型将DLC-Bench的框架扩展为支持每张图像包含多个区域遮罩的情况，从而可以同时评估描述质量与推理效率。实验表明，PerceptionDLM在区域描述任务中仍能保持良好的性能，同时在多区域感知任务中则能显著提升处理速度。我们的研究结果凸显了多模态扩散语言模型在高效并行视觉感知方面的巨大潜力。据我们所知，我们是第一个利用扩散语言模型的优势来实现并行区域描述与感知的模型。代码、模型和数据集均已发布。
English Abstract
Multimodal large language models (MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we propose PerceptionDLM, a multimodal diffusion language model optimized for efficient parallel region perception. Built upon PerceptionDLM-Base, a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs, our architecture fully leverages the parallel decoding nature of DLMs. Specifically, we introduce efficient prompting and structured attention masking to enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improves inference efficiency compared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property of visual perception capability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench) by scaling the DLC-Bench to include multiple region masks per image, enabling joint evaluation of both caption quality and inference efficiency. Experiments demonstrate that PerceptionDLM maintains competitive performance in region captioning while achieving substantial speed improvements for multi-region perception tasks. Our results highlight the potential of multimodal diffusion language models for efficient, parallel visual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages of diffusion language models. Code, models, and datasets are released.