利用生理數據和概述領域自適應BERT模型向量化之臨床病歷早期預測急性腎衰竭

Abstract

背景: 急性腎損傷（AKI）是一種會迅速惡化腎功能的嚴重病症，尤其在加護病房（ICU）環境中會導致顯著發病率與死亡率。傳統 AKI 診斷主要依賴血清肌酸酐和尿量等指標，這些方法可能存在診斷延遲問題。早期識別 AKI 對於及時介入治療和改善患者預後至關重要。本研究探索利用先進自然語言處理（NLP）技術挖掘臨床病歷文本，通過分析 MIMIC-III 資料集中的臨床敘事記錄來提升 AKI早期檢測能力。 目標: 本研究提出 AKIBERT 模型，該模型採用 AKI 特定領域語料庫進行預訓練並整合時間序列數據分析。旨在開發一個用於 ICU 環境 AKI 預測的綜合系統，並探討最先進語言模型在此領域的應用潛力。 方法: 本研究採用結構化臨床特徵與非結構化臨床文本兩種數據形式。經預處理後，包含 AKI 狀態與生命徵象的結構化數據集篩選出 33,825 位患者。針對非結構化數據，使用基於 BERT 的嵌入方法（包括 BERT、BioBERT、ClinicalBERT、BCBERT），結合 LSTM 模型處理入院後 24 小時內的臨床數據。評估指標包含AUROC、特異性與敏感性，並採用校準曲線評估模型可靠性。本研究在結構化預測框架內比較各 BERT 變體的效能差異，並創新性地開發 AKI 領域專用模型AKIBERT 以提升文本預測能力。 結果: 採用最大規模 AKI 專用語料庫預訓練的 AKIBERT T7 模型，在 24-36小時與 24-168 小時預測窗口分別達到 0.8728 與 0.8488 的 AUROC 值。校準分析顯示 AKIBERT 具有更佳的概率校準特性。並在兩個預測時間窗中，AKIBERT 均顯著優於標準 BERT 模型，這表明即使是有限度的 AKI 領域適應性訓練，也能通過增強臨床文本解析能力顯著提升 AKI 預測效能。

Background: Acute Kidney Injury (AKI) is a severe condition that rapidly impairs kidney function, leading to significant morbidity and mortality, especially in ICU settings. Traditionally, AKI is diagnosed using indicators such as serum creatinine and urine output, which may delay detection. Early recognition of AKI is crucial for timely intervention and improving patient outcomes. This research explores advanced Natural Language Process ing (NLP) to mine clinical notes, aiming to improve early detection of AKI by utilizing the insights in clinical narratives from the MIMIC-III dataset. Objectives: This study introduces AKIBERT, a new AKI-domain model pretrained with an AKI-specific corpus, incorporating time-series data analysis. The aim is to de velop a comprehensive system for predicting AKI in ICU settings and to explore the ca pabilities of state-of-the-art language models in this context. Methods: I employed two data types: structured clinical features and unstructured clinical notes. The structured dataset, encompassing AKI status and vital signs, was filtered down to 33,825 patients after preprocessing. For unstructured data, I utilized BERT based embedding methods, including BERT, BioBERT, ClinicalBERT, BCBERT, fol lowed by LSTM models to process clinical data from the initial 24 hours post-admission. I evaluated performance metrics such as AUROC, specificity, and sensitivity, and em ployed calibration curves to assess the reliability of the models. The efficacy of various BERT-based embedding methods was compared within a structured prediction framework. Moreover, this research introduced AKIBERT, a domain-specific BERT model, developed to enhance prediction capabilities for AKI-related texts. Results: AKIBERT T7, the model pretrained with the most extensive AKI-specific corpus, achieved AUROC values of 0.8728 and 0.8488 for the 24-36 and 24-168 hour pre diction windows, respectively. Calibration analysis underscores AKIBERT’s reliability, showing better probability alignment compared to other BERT variants. Notably, in both prediction windows, AKIBERT consistently outperforms the standard BERT model, in dicating that even limited AKI-specific training can significantly improve AKI prediction by enhancing the interpretation of clinical narratives.