Abstract
Self-supervised learning through masked autoencoders has attracted great attention for remote sensing (RS) foundation model (FM) development, enabling improved representation learning across diverse sensors and downstream tasks. However, existing RS FMs often either suffer from substantial computational complexity during both training and inference or exhibit limited representational capacity. These issues restrict their practical applicability in RS. To address this limitation, we propose an adaptation for enhancing the efficiency of RS FMs by integrating the Soft mixture-of-experts (MoE) mechanism into the FM. The integration of Soft MoEs into the FM allows modality-specific expert specialization alongside shared cross-sensor representation learning. To demonstrate the effectiveness of our adaptation, we apply it on the Cross-Sensor Masked Autoencoder (CSMAE) model, resulting in the Cross-Sensor Mixture-of-Experts (CSMoE) model. In addition, we introduce a thematic-climatic descriptor-driven sampling strategy for the construction of a representative and diverse training set to train our CSMoE model. Extensive experiments on scene classification, semantic segmentation, and content-based image retrieval demonstrate that our adaptation yields a reduction in computational requirements while maintaining or improving representational performance. Compared to state-of-the-art RS FMs, CSMoE achieves a superior trade-off between representational capacity, accuracy, and computational efficiency. On average, CSMoE achieves more than twice the computational efficiency of existing RS FMs, while maintaining competitive performance across all experiments. These results show the effectiveness of the proposed adaptation for creating computationally efficient RS FMs. The code for the model, the training set creation, and the model weights will be available at this https URL.
Abstract (translated)
通过掩码自动编码器进行自我监督学习在远程 sensing（RS）基础模型（FM）开发中引起了极大的关注，使各种传感器和下游任务中的表示学习得到改善。然而，现有的 RS FMs 经常要么在训练和推理过程中面临较大的计算复杂度问题，要么表现出有限的表征能力。这些问题限制了它们在 RS 中的实际应用性。 为了解决这一局限性，我们提出了一种通过将 Soft 混合专家（MoE）机制整合到基础模型中来提升 RS FMs 效率的方法。Soft MoEs 的集成使模态特异性专家专门化与共享的跨传感器表示学习相结合成为可能。为了展示我们改进的有效性，我们将这种方法应用于 Cross-Sensor Masked Autoencoder (CSMAE) 模型上，从而产生了 Cross-Sensor Mixture-of-Experts (CSMoE) 模型。 此外，我们还提出了一种主题气候描述符驱动的采样策略来构建一个具有代表性和多样性的训练集，以用于我们的 CSMoE 模型的训练。在场景分类、语义分割和基于内容的图像检索方面的广泛实验表明，我们的改进降低了计算需求，同时保持或提高了表示性能。 与最先进的 RS FMs 相比，CSMoE 达到了表征能力、准确性和计算效率之间更佳的权衡点。平均而言，CSMoE 达到现有 RS FMs 计算效率的大约两倍，并在所有实验中均表现出竞争性的性能。这些结果表明了所提议改进的有效性，用于创建具有高计算效率的 RS 基础模型。 该模型、训练集生成代码及模型权重将在以下网址提供：[插入链接]
URL
https://arxiv.org/abs/2509.14104
