PhysiFormer：学习在世界空间中模拟力学原理
PhysiFormer: Learning to Simulate Mechanics in World Space
摘要
我们提出了PhysiFormer——一种用于描述物理上合理的3D物体运动的扩散Transformer模型。与那些在视点相关的像素空间中进行操作的视频世界模型不同，PhysiFormer将物体表示为以世界坐标表示的3D网格。根据初始顶点位置和速度以及物体的材料类型（刚性或弹性），该模型可以预测未来顶点的运动轨迹。虽然一些相关的神经物理方法依赖于特定的潜在空间或明确地强调刚性和因果关系，但PhysiFormer表明，无需这些先验假设，只要将顶点轨迹预测视为一个直接在世界坐标下的去噪扩散过程，就能获得优秀的性能。其概率性建模方式能够捕捉到学习到的动态中的不确定性，从而从初始条件出发产生多种可能的未来状态，因此这种框架对于处理不可观测的不确定性问题非常有用。该模型采用了时间、空间和物体层面的注意力机制，从而实现了效率提升，同时无需对物体进行显式编码即可进行多物体推理。经过10万多个模拟轨迹的训练后，PhysiFormer能够处理刚性与弹性力学场景，还能适用于混合材料环境、现实中不存在的几何形状以及大量物体的情况。在轨迹准确性、刚性的保持以及基于动量的物理一致性方面，PhysiFormer明显优于自回归基线模型。我们的研究结果表明，坐标空间扩散方法为实现机器人技术、图形处理和物理设计领域中的视点无关且能感知几何特征的全球建模方式提供了有效的途径。相关可视化内容、代码和模型可访问于https://yimingc9.github.io/physiformer。
English Abstract
We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation-invariant multi-object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed-material settings, unseen real-world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency. Our results position coordinate-space diffusion as a promising step toward view-invariant, geometry-aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.