大型語言模型 vs. 嵌入方法應用於醫療多變量時間序列：基於缺失率的系統性分析

Abstract

多變量時間序列（MTS）數據在精準醫療中發揮著關鍵作用，促進疾病風險預測和患者監測等核心任務。然而，傳統的嵌入方法在處理 MTS 數據時面臨許多挑戰，主要受限於高缺失率和複雜的變量間關係。大型語言模型（LLMs）在捕捉複雜模式方面展現出潛力，進而引發了對其在 MTS 分析中的廣泛探索。然而，MTS 數據的高缺失率挑戰了 LLMs 的時序列特性，使其相對於傳統方法的穩定優勢成為值得探討的問題。 我們提出 HSCANE-LLM，這是一種結合「分層可擴展數值嵌入（HSCANE）」和「重編程 LLMs」的混合框架。HSCANE 採用自注意力機制建模時間和空間依賴關係，並透過平衡輸入解析度和詞元效率來確保可擴展性。同時，藉由重編程技術整合預訓練 LLMs，使其能夠在無需微調的情況下識別複雜模式。可學習的輸入變換層將時間序列對齊至自然語言表示，降低計算資源需求和訓練時間，同時保留 LLMs 的預訓練知識。 在三個醫療 MTS 數據集上的實驗結果表明，HSCANE-LLM 在高缺失率情境中表現優異，超越了最新的嵌入式方法和簡單的重編程 LLMs，憑藉其混合結構有效應對數據稀疏性。然而，在較低缺失率的情況下，嵌入方法仍具有競爭力，突出了模型選擇需基於數據特性進行權衡。本研究針對缺失率進行了系統性分析，強調適應性模型選擇對於精準醫療中 MTS 分析的重要性。HSCANE-LLM 的成功不僅提升 LLMs 在 MTS 任務中的實際應用價值，更降低 LLMs 在醫療及更廣泛應用場景中的技術門檻。

Multivariate time series (MTS) data play a critical role in precision medicine, supporting key tasks such as disease risk prediction and patient monitoring. However, traditional embedding methods struggle with MTS data due to high missing rates and complex inter-variable relationships. Large language models (LLMs) have shown promise in capturing intricate patterns, sparking interest in their potential for MTS analysis. However, the high missing rates in MTS data challenge the sequential nature of LLMs, raising questions about their consistent superiority over traditional methods. We propose HSCANE-LLM, a hybrid framework combining Hierarchical Scalable Numerical Embedding (HSCANE) with reprogrammed LLMs. HSCANE employs self-attention to model temporal and spatial dependencies while ensuring scalability by balancing input resolution and token efficiency. Meanwhile, pre-trained LLMs are integrated via reprogramming, allowing them to detect complex patterns without fine-tuning. Learnable input transformation layers align time series with natural language representations, reducing the need for computational resources and training time, while preserving the LLM's pre-trained knowledge. Experiments on three medical MTS datasets reveal that HSCANE-LLM excels in high-missing-rate scenarios, outperforming both state-of-the-art embedding-based methods and naive reprogrammed LLMs, leveraging its hybrid structure to handle data sparsity. However, embedding methods remain competitive in cases with lower missing rates, highlighting trade-offs in model selection based on data characteristics. This study conducts a systematic analysis based on missing rates, emphasizing the importance of adaptive model selection for effective MTS analysis in precision medicine. The success of HSCANE-LLM not only enhances the practical applicability of LLMs for MTS tasks but also reduces the technical barriers to integrating LLMs into broader real-world applications in healthcare and beyond.