Abstract
Large Language Models (LLMs) have been extensively applied in time series analysis. Yet, their utility in the few-shot classification (i.e., a crucial training scenario due to the limited training data available in industrial applications) concerning multivariate time series data remains underexplored. We aim to leverage the extensive pre-trained knowledge in LLMs to overcome the data scarcity problem within multivariate time series. Specifically, we propose LLMFew, an LLM-enhanced framework to investigate the feasibility and capacity of LLMs for few-shot multivariate time series classification. This model introduces a Patch-wise Temporal Convolution Encoder (PTCEnc) to align time series data with the textual embedding input of LLMs. We further fine-tune the pre-trained LLM decoder with Low-rank Adaptations (LoRA) to enhance its feature representation learning ability in time series data. Experimental results show that our model outperformed state-of-the-art baselines by a large margin, achieving 125.2% and 50.2% improvement in classification accuracy on Handwriting and EthanolConcentration datasets, respectively. Moreover, our experimental results demonstrate that LLM-based methods perform well across a variety of datasets in few-shot MTSC, delivering reliable results compared to traditional models. This success paves the way for their deployment in industrial environments where data are limited.
Abstract (translated)
大型语言模型（LLMs）已被广泛应用于时间序列分析中。然而，它们在少量样本分类中的效用（即由于工业应用中可用的训练数据有限，这是一个至关重要的训练场景），尤其是在处理多变量时间序列数据时，仍然有待探索。我们旨在利用大规模预训练的语言模型的知识来解决多变量时间序列数据中存在的数据稀缺问题。为此，我们提出了LLMFew框架，这是基于大型语言模型增强的一种方法，用于研究大型语言模型在少量样本的多变量时间序列分类中的可行性和能力。 该模型引入了“Patch-wise Temporal Convolution Encoder (PTCEnc)”来对齐时间序列数据与大型语言模型输入文本嵌入。此外，我们还通过低秩适应（LoRA）微调预训练的语言模型解码器，以增强其在处理时间序列数据时的特征表示学习能力。 实验结果表明，我们的模型相比于最先进的基准方法，在分类准确度上分别提高了125.2%和50.2%，具体体现在Handwriting和EthanolConcentration数据集上的表现。此外，我们的实验证明了基于大型语言模型的方法在少量样本多变量时间序列分类的各种数据集中都表现出色，并且与传统模型相比提供了可靠的结果。 这一成功为这些方法在工业环境中部署铺平了道路，在这些环境中由于数据有限而难以采用传统的机器学习技术。
URL
https://arxiv.org/abs/2502.00059
