Abstract
We address robust separation assurance for small Unmanned Aircraft Systems (sUAS) under GPS degradation and spoofing via Multi-Agent Reinforcement Learning (MARL). In cooperative surveillance, each aircraft (or agent) broadcasts its GPS-derived position; when such position broadcasts are corrupted, the entire observed air traffic state becomes unreliable. We cast this state observation corruption as a zero-sum game between the agents and an adversary: with probability R, the adversary perturbs the observed state to maximally degrade each agent's safety performance. We derive a closed-form expression for this adversarial perturbation, bypassing adversarial training entirely and enabling linear-time evaluation in the state dimension. We show that this expression approximates the true worst-case adversarial perturbation with second-order accuracy. We further bound the safety performance gap between clean and corrupted observations, showing that it degrades at most linearly with the corruption probability under Kullback-Leibler regularization. Finally, we integrate the closed-form adversarial policy into a MARL policy gradient algorithm to obtain a robust counter-policy for the agents. In a high-density sUAS simulation, we observe near-zero collision rates under corruption levels up to 35%, outperforming a baseline policy trained without adversarial perturbations.
Abstract (translated)
本文通过多智能体强化学习（MARL）解决了小型无人机系统（sUAS）在GPS降级与欺骗条件下的鲁棒性防撞保障问题。在协同监视场景中，每架无人机（或智能体）广播其GPS定位信息；当此类位置广播受到污染时，整个观测到的空中交通状态将变得不可靠。我们将这种状态观测污染建模为智能体与攻击者之间的零和博弈：以概率R，攻击者会扰动观测状态以最大程度降低各智能体的安全性能。我们推导出该对抗性扰动的闭式表达，完全规避对抗训练，并实现状态维度上的线性时间复杂度评估。证明该表达式能以二阶精度逼近真实最坏情况对抗扰动。进一步，我们量化了清洁观测与污染观测之间的安全性能差距，证明在库尔贝克-莱布勒正则化下，该差距最多随污染概率线性退化。最终，我们将闭式对抗策略集成至MARL策略梯度算法，为智能体生成鲁棒对抗策略。在高密度小型无人机系统仿真中，即使在35%污染水平下，该方法仍实现近零碰撞率，优于未进行对抗训练的基准策略。
URL
https://arxiv.org/abs/2603.28900
