风格偏见：一些人类视觉线索导致了MLLMs中的大多数社会偏见
StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs
摘要
多模态大语言模型在具有个人和社会重要性的场景中得到广泛应用，但那些影响这些模型对人的判断的视觉因素却仍然不太清楚。以往的研究通常比较不同的个体或群体，因此很难区分外观特征与身份差异之间的影响。我们提出了StylisticBias这一评估工具，用于衡量多模态大语言模型中各属性层面的社会偏见。我们生成了500张逼真的面部图像，每张图像对应约50种单一属性变化，最终得到约25,000张图像。这种设计方式保持身份不变，每次只改变一个视觉属性，从而可以准确测量各种视觉特征对模型判断的影响。我们对6种多模态大语言模型在25种二元社会判断场景中的表现进行了评估。我们发现，年龄和体型对身份层面的影响最为显著，而时尚风格和其他视觉特征则对属性层面的影响最大。此外，大约15种属性就导致了总变异的近80%，说明偏见主要集中在某些少数视觉特征上。在那些与外观相关的语义判断中，比如社会经济状况和风格相关的判断中，这种敏感性最为明显。我们将StylisticBias作为多模态模型中细致评估偏见的工具发布出来。代码和数据集可访问于：https://github.com/timo-cavelius/StylisticBias以及https://hf.co/datasets/shaghayegh/stylistic-bias-dataset。
English Abstract
Multimodal large language models (MLLMs) are increasingly deployed in personally and societally consequential settings, yet the visual cues that shape how these models judge people remain poorly understood. Prior work often compares different (groups of) individuals, making it difficult to separate appearance effects from identity differences. We introduce StylisticBias, a controlled benchmark for evaluating attribute-level social bias in MLLMs. We generate 500 photorealistic base faces and create about 50 single-attribute variations per face, producing about 25K images. This design keeps identity fixed and changes one visual attribute at a time. It lets us measure how specific cues shift model judgments. We evaluate six MLLMs across 25 binary social judgment scenarios. We find that age and body type dominate identity-level effects, while fashion style and other visual cues drive the largest attribute-level shifts. We further find that about 15 attributes account for nearly 80\% of the total variation, showing that bias is concentrated in a small set of visual cues. Sensitivity is strongest in judgments that are semantically aligned with appearance, especially socioeconomic and style-related judgments. We release StylisticBias as a benchmark for fine-grained bias evaluation in multimodal models. Code and dataset: https://github.com/timo-cavelius/StylisticBias and https://hf.co/datasets/shaghayegh/stylistic-bias-dataset.