MotionVLA：一种用于仿人机器人运动的视觉-语言-动作模型
MotionVLA: Vision-Language-Action Model for Humanoid Motion
摘要
从场景图像和文本中生成逼真的人形运动效果，涉及到低频的姿势信息以及高频的物理动态信息。然而，许多现有方法都使用单一的编码方式来处理运动数据，这导致不同性质的运动信号被压缩到同一个量化空间中。我们对人类运动数据的频域分析表明，采用单一编码方式会导致量化结果与运动统计信息出现明显不匹配：五个DCT系数能够捕捉到93%的关节位置信息，但只能捕捉到37%的关节速度信息。这种处理方式会使得量化结果偏向于姿势信息，从而无法充分反映高频速度信息。另一个挑战在于如何调整标准自回归模型，以有效处理运动序列中的高频物理信号。因此，我们提出了DSFT这一双流频率分割技术，它将运动分为基础流和物理流两部分，分别进行DCT压缩和BPE编码。此外，我们还提出了MotionVLA模型，该模型基于Qwen3.5框架，将基础流和物理流的信息整合在一起进行处理。在HumanML3D和MBench上的实验表明，尽管采用了轻量级的2B主干网络，MotionVLA仍然能够将真实数据中的多样性差异降低50%以上，同时提升了运动与条件的一致性，提高了3.8%。这种基于频率感知的双流分离方式，为自回归运动的生成提供了一种有效的解决方案。代码地址：https://github.com/AIGeeksGroup/MotionVLA。官网：https://aigeeksgroup.github.io/MotionVLA。
English Abstract
Generating realistic humanoid motion from scene images and text involves both low-frequency pose semantics and high-frequency physical dynamics. However, many existing methods tokenize motion with a single shared codebook, forcing heterogeneous motion signals into the same quantization space. Our frequency-domain analysis of human motion data reveals a clear mismatch between single-codebook quantization and motion statistics: five DCT coefficients capture 93% of joint-position energy but only 37% of joint-velocity energy, which can bias quantization toward pose statistics and under-represent high-frequency velocity components. A second challenge lies in adapting a standard autoregressive model to effectively model high-frequency physical signals in motion sequences. Therefore, we propose DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams and compresses them independently with DCT truncation and BPE. Furthermore, we present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence, where Phys tokens are predicted after Base tokens. Experiments on HumanML3D and MBench show that, despite using a lightweight 2B backbone, MotionVLA reduces the Diversity gap to real data by over 50% on HumanML3D and improves Motion-Condition Consistency by 3.8% on MBench, supporting frequency-aware dual-stream decoupling as an effective formulation for autoregressive motion generation. Code: https://github.com/AIGeeksGroup/MotionVLA. Website: https://aigeeksgroup.github.io/MotionVLA.