基于大型语言模型的生成式推荐中的隐性推理机制
Implicit Reasoning for Large Language Model-based Generative Recommendation
摘要
用于生成式推荐的大型语言模型面临着诸多挑战。在语义标识符的影响下，语言的自然推理能力受到了干扰，这就需要采用一种轻量级的隐性推理方法来实现推理过程。这种方法的性能优于传统的显式推理方法，同时还能降低计算成本。由Qwen/Qwen2.5-Coder-32B-Instruct大型语言模型生成的推荐内容，越来越被用作生成式推荐的底层框架，因为它们能够利用预训练得到的世界知识。不过，如何可靠地调用这些知识来进行推荐推理仍然是一个亟待解决的问题。其中一个主要障碍是，基于大型语言模型的生成式推荐系统通常使用的是语义标识符来表示物品信息，而这些标识符在训练过程中并未被大型语言模型接触过，这就破坏了语言模型的自然语言推理能力。现有的解决方案通常采用复杂的多阶段流程来处理语义标识符，但这往往会导致推理过程变得复杂且成本高昂。此外，现有方法对于为什么需要每个阶段的理解不足。在这项研究中，我们系统地分析了基于大型语言模型的生成式推荐系统中显式推理训练流程的缺陷，发现三个主要问题：世界知识的表达能力减弱、语义标识符与自然语言标记空间之间的不一致性，以及推理结果的准确性问题。为了解决这些问题，我们提出了PauseRec这一轻量级隐性推理框架。PauseRec具有极高的实用性，它能够避免复杂的推理过程，从而节省大量的计算资源。其优势包括：1) 性能优于传统的显式推理方法，性能提升幅度可达6.22%；2) 训练成本可降低65%，即减少所需的GPU时间；3) 推理速度提升幅度可达71.3%。这些结果表明，PauseRec可以作为显式推理方法的轻量级替代方案，从而让基于大型语言模型的生成式推荐系统更加高效且有效。
English Abstract
Large Language Models for generative recommendation face challenges with semantic IDs disrupting natural-language reasoning, prompting a lightweight implicit reasoning approach that outperforms explicit methods while reducing computational costs. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Large Language Models (LLMs) are increasingly adopted as backbones for Generative Recommendation (GR), promising access to pretrained world knowledge. Yet reliably invoking this knowledge for GR remains poorly understood. A key obstacle is that LLM-based GR typically represents items with Semantic IDs (SIDs), disrupting LLMs' natural-language reasoning interface because these tokens are unseen by the LLM during pretraining. Existing approaches address this with expensive multi-stage pipelines that ground SIDs and elicit explicit rationales, but offer limited insight into when and why each stage is necessary. In this work, we systematically decompose explicit reasoning training pipelines for LLM-based GR, revealing three key limitations: weakened world-knowledge verbalization, misalignment between SID and natural-language token embedding spaces, and sensitivity to rationale quality, all of which hurt explicit reasoning performance. To circumvent these issues, we propose PauseRec, a lightweight implicit reasoning paradigm tailored for GR. PauseRec is exceptionally practical, avoiding costly reasoning trace acquisition and reasoning alignment training, leading to a multitude of benefits: (1) it outperforms standard explicit CoT methods by up to 6.22%, (2) it reduces training cost by up to 65% GPU hours, and (3) it speeds up inference by up to 71.3%. These results position PauseRec as a lightweight alternative to explicit rationale generation, enabling more effective and efficient LLM-based GR.