TuneJury：一种用于改善音乐生成偏好对齐的开放式评估方法
TuneJury: An Open Metric for Improving Music Generation Preference Alignment
摘要
我们介绍了TuneJury模型——一种开放的、基于实例级别的文本转音乐生成模型。该模型能够根据给定的文本提示和音频片段来预测音乐偏好得分。目前发布的模型训练数据来源于公开可用的用户偏好标签，这些标签涵盖了各种形式的偏好对比（如A与B之间的比较）、指标对齐相关的偏好配对、众包式的成对比较，以及专家级的审美评价。在两个音频片段之间的偏好得分差异在我们的保留测试数据集上表现非常稳定，而且可以通过简单的得分阈值来实现数据的筛选。TuneJury模型不仅适用于保留的测试数据，也适用于其他非标准的数据集。在后者上，该模型的性能仍然能够与之前的基线模型相媲美。对于在训练之后生成的生成结果，我们引入了锚点校准机制——这是一种事后性的、针对每个系统的Bradley-Terry校准方法。这种方法能够在显著更高的数据效率下实现与从头开始重新训练时相当的结果。同样的固定奖励机制使得TuneJury在三个下游应用中都能保持一致的性能：推理时的N次最佳选择、DITTO风格的潜在优化，以及专家级的后训练优化。TuneJury模型的代码可以在https://github.com/yonghyunk1m/TuneJury下载。
English Abstract
We introduce TuneJury, an open, instance-level pairwise reward model for text-to-music that predicts a music preference score from a text prompt and an audio clip. The released checkpoint is trained on publicly available human-preference labels covering arena-style (A vs. B) votes, metric-alignment preference pairs, crowdsourced pairwise comparisons, and expert aesthetic ratings. The predicted score margin between two clips is well calibrated on our held-out test split, supporting data filtering via a simple score threshold. TuneJury generalizes to both held-out test pairs and out-of-distribution benchmarks, remaining competitive with prior baselines on the latter. For generators released after training, we introduce anchor calibration, a post-hoc, per-system Bradley-Terry calibration that recovers agreement at substantially better data efficiency than from-scratch retraining. The same frozen reward drives consistent reward-axis gains across three downstream applications: inference-time best-of-N selection, DITTO-style latent optimization, and expert-iteration post-training. TuneJury is available at https://github.com/yonghyunk1m/TuneJury.