Abstract
Facial analysis is a key component in a wide range of applications such as security, autonomous driving, entertainment, and healthcare. Despite the availability of various facial RGB datasets, the thermal modality, which plays a crucial role in life sciences, medicine, and biometrics, has been largely overlooked. To address this gap, we introduce the T-FAKE dataset, a new large-scale synthetic thermal dataset with sparse and dense landmarks. To facilitate the creation of the dataset, we propose a novel RGB2Thermal loss function, which enables the transfer of thermal style to RGB faces. By utilizing the Wasserstein distance between thermal and RGB patches and the statistical analysis of clinical temperature distributions on faces, we ensure that the generated thermal images closely resemble real samples. Using RGB2Thermal style transfer based on our RGB2Thermal loss function, we create the T-FAKE dataset, a large-scale synthetic thermal dataset of faces. Leveraging our novel T-FAKE dataset, probabilistic landmark prediction, and label adaptation networks, we demonstrate significant improvements in landmark detection methods on thermal images across different landmark conventions. Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations. Our code and models are available at this https URL.
Abstract (translated)
面部分析是许多应用的关键组件，如安全、自动驾驶、娱乐和医疗。尽管各种面部RGB数据集已经存在，但生物医学和生物测量学中扮演关键角色的热模量（thermal modality）却受到了很大的忽视。为了填补这一空白，我们引入了T-FAKE数据集，一个大型合成热数据集，具有稀疏和密集的标志。为了方便数据集的创建，我们提出了一个新颖的RGB2Thermal损失函数，该函数可以将热风格传递到RGB面部。通过利用热和RGB补丁之间的Wasserstein距离和面部临床温度分布的统计分析，我们确保生成的热图像与真实样本非常相似。基于我们的RGB2Thermal损失函数，我们创建了T-FAKE数据集，一个大型合成热数据集。通过利用我们新颖的T-FAKE数据集、概率地标预测和标签适应网络，我们在不同地标范式下显著提高了热图像中的地标检测方法。我们的模型在稀疏70个地标和密集478个地标注释上表现优异。我们的代码和模型可在此处访问：https://www.thisurl.com/
URL
https://arxiv.org/abs/2408.15127
