UniverSat：适用于地球观测的、不依赖分辨率和模态的Transformer模型
UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation
摘要
视觉Transformer（ViT）在计算机视觉领域占据主导地位。然而，它们对固定样片编码器的依赖，使得其在地球观测领域的应用受到限制——因为地球观测中的输入方式、规模和分辨率差异很大。我们提出了UniverSat这一基于ViT风格的模型框架，该框架使用通用样片编码器来将不同空间、光谱和时间维度的样片，以及来自光学与非光学传感器的数据，映射到同一个嵌入空间中，从而使用统一的权重体系来训练模型。这样就能在多种不同的多模态数据上训练出单一模型，得到鲁棒且不受传感器影响的空间特征。我们在GeoBench、PANGEABench和SpectralEarth等标准地球观测基准测试中取得了良好的分类和分割效果，证明了这种方法的有效性。我们的代码和模型可访问于https://github.com/gastruc/UniverSat。
English Abstract
Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where input modalities, scales, and resolutions vary widely. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a shared set of weights. This enables training a single model on heterogeneous multimodal corpora via self-supervision, yielding robust, sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. Our code and models are available at https://github.com/gastruc/UniverSat.