請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/101006完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 林澤 | zh_TW |
| dc.contributor.advisor | Che Lin | en |
| dc.contributor.author | 簡大容 | zh_TW |
| dc.contributor.author | Ta-Jung Chien | en |
| dc.date.accessioned | 2025-11-26T16:26:29Z | - |
| dc.date.available | 2025-11-27 | - |
| dc.date.copyright | 2025-11-26 | - |
| dc.date.issued | 2025 | - |
| dc.date.submitted | 2025-09-25 | - |
| dc.identifier.citation | [1] S. Bai, J. Z. Kolter, and V. Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
[2] L. Breiman. Random forests. Machine Learning, 45(1):5–32, 2001. [3] W. Cao, D. Wang, J. Li, H. Zhou, L. Li, and Y. Li. Brits: Bidirectional recurrent imputation for time series. In Advances in Neural Information Processing Systems (NeurIPS), 2018. [4] Z. Che, S. Purushotham, K. Cho, D. Sontag, and Y. Liu. Recurrent neural networks for multivariate time series with missing values (gru-d). In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018. [5] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems (NeurIPS), 2018. [6] T. Chen and C. Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, 2016. [7] Y. Chen et al. Platelet-to-white blood cell ratio as a prognostic biomarker in critical illness: a systematic review. Frontiers in Medicine, 10:1167053, 2023. [8] A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. Physiobank, physiotoolkit, and physionet: Components of a new research resource for complex physiologic signals. Circulation, 101(23):e215–e220, 2000. [9] M. Gregersen et al. Low diastolic blood pressure, increased heart rate and mortality in intensive care patients. Acta Anaesthesiologica Scandinavica, 59(4):480–489, 2015. [10] M. Horn, M. Moor, C. Bock, B. Rieck, and K. Borgwardt. Set functions for time series, 2020. [11] M. Horn, M. Moor, C. Bock, B. Rieck, and K. Borgwardt. Set functions for time series (strats). In International Conference on Learning Representations (ICLR), 2020. [12] C.-K. Huang, Y.-H. Hsieh, T.-J. Chien, L.-C. Chien, S.-H. Sun, T.-H. Su, J.-H. Kao, and C. Lin. Scalable numerical embeddings for multivariate time series: Enhancing healthcare data representation learning. arXiv preprint arXiv:2405.16557, 2024. [13] Y. Huang et al. Representation of numerical values in language models: An inherent limitation of tokenization. Transactions of the Association for Computational Linguistics, 12:279–284, 2024. [14] A. E. Johnson, T. J. Pollard, L. Shen, L.-W. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. A. Celi, and R. G. Mark. Mimic-iii, a freely accessible critical care database. Scientific Data, 3:160035, 2016. [15] P. J. Johnson et al. Assessment of liver function in patients with hepatocellular carcinoma: a new evidence-based approach—the albi grade. Journal of Clinical Oncology, 33(6):550–558, 2015. [16] J. Kaplan, T. Henighan, et al. Scaling laws for neural language models. Journal of Machine Learning Research, 21:540–548, 2023. [17] M. Li et al. Serum creatinine and blood urea nitrogen as predictors of mortality in intensive care patients. Critical Care, 22(1):317, 2018. [18] B. Lim, S. . Arik, N. Loeff, and T. Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2019. [19] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection, 2018. [20] X. Liu et al. Prognostic value of the ast/alt ratio in patients with liver cirrhosis and hepatocellular carcinoma. Hepatology Research, 53(1):25–35, 2023. [21] Y. Luo, X. Cai, Y. Zhang, and J. Xu. Multivariate time series imputation with generative adversarial networks (deep sti). In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), 2019. [22] P. Peykani et al. Large language models in healthcare: Opportunities and challenges. Nature Medicine, 30(7):1085–1093, 2024. [23] T. Saito and M. Rehmsmeier. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PloS One, 10(3):e0118432, 2015. [24] S. N. Shukla and B. M. Marlin. Interpolation-prediction networks for irregularly sampledtimeseries(mtan). InInternationalConferenceonLearningRepresentations (ICLR), 2019. [25] S. N. Shukla and B. M. Marlin. Multi-time attention networks for irregularly sampled time series. In Proceedings of the International Conference on Learning Representations (ICLR), 2021. [26] H. Song et al. Trajgpt: Generative pretraining for longitudinal clinical trajectories. Nature Machine Intelligence, 6(5):544–560, 2024. [27] H. Song, D. Rajan, J. J. Thiagarajan, and A. Spanias. Attend and diagnose: Clinical time series analysis using attention models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, pages 4091–4098, 2018. [28] H. Song, D. Rajan, J. J. Thiagarajan, and A. Spanias. Attend and diagnose: Clinical time series analysis using attention models (sand). In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018. [29] J. Su, Y. Lu, S. Pan, B. Wei, and Y. Liu. Roformer: Enhanced transformer with rotary position embedding. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2021. [30] B. Suetrong and K. R. Walley. Lactic acidosis: pathophysiology, classification, and clinical management. Chest, 151(2):240–252, 2017. [31] C. Sun et al. Comet: Multimodal ehr foundation model with cross-modal alignment. Nature Communications, 14(1):61–65, 2023. [32] S. Tipirneni and C. K. Reddy. Self-supervised pretraining improves learning of ehr time series. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), pages 3490–3498, 2020. [33] S. Tipirneni and C. K. Reddy. Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series. ACM Transactions on Knowledge Discovery from Data, 16(6):Article 105, 2022. [34] T. Van der Poll et al. Thrombocytopenia in sepsis: incidence, mechanisms, and prognostic significance. Critical Care Medicine, 41(3):816–827, 2013. [35] H. Wu, J. Xu, J. Wang, and M. Long. Spacetimeformer: Pretraining temporal-spatial transformers for time series forecasting. In International Conference on Learning Representations (ICLR), 2023. [36] Z. Zeng, J. Yu, M. Wu, et al. Comet: Cross-modal pretraining for clinical and omics data. Nature Machine Intelligence, 5:601–613, 2023. [37] G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff. A transformerbased framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), 2021. [38] J. Zhang et al. The blood urea nitrogen to creatinine ratio is associated with mortality in critically ill patients. Journal of Intensive Care Medicine, 38(3):216–224, 2023. [39] X. Zhang, F. Wang, X. Ji, and J. Tang. Hitanet: Hierarchical time-aware attention networks for ehr modeling. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI), 2020. [40] Z. Zhang and J. Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2022. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/101006 | - |
| dc.description.abstract | 不規則取樣的多變量電子病歷（Electronic Health Records, EHRs）時間序列，因其取樣間隔不均、測量時間不同步以及普遍存在缺漏值，對於臨床預後模型建構帶來了重大挑戰。本研究提出一個新穎的嵌入框架，結合廣播式（broadcast-based）的架構與哈達瑪乘積數值嵌入模組（Hadamard Product Numerical Embedding, HPNE）。在此設計中，HPNE 將每一個特徵嵌入向量劃分為多個子空間，並透過與數值相關的門控機制進行調製，從而在無需填補缺漏值的情況下，對稀疏數值特徵進行細緻表徵。該方法既能保持特徵的辨識性，又能增強跨特徵交互關係的捕捉能力。另一方面，廣播機制有效整合特徵與數值的資訊維度，產生對缺漏具韌性、且可擴展至異質性臨床資料集的表徵。
在實驗評估上，我們於重症加護病房（ICU）數據集（PhysioNet 2012 與 MIMIC-III）進行住院死亡率預測，以及於慢性病數據集（台大肝細胞癌 HCC）進行長期預測。實驗結果顯示，本研究方法在不同領域下皆能穩定超越強基線模型，達到最新的預測準確度，且無需依賴額外的缺漏值填補。進一步的遷移學習實驗亦顯示，當模型先於大型 ICU 資料集進行預訓練，再應用於規模較小的資料集時，能帶來顯著的性能提升，證明 HPNE 所學得的特徵嵌入確實具有語意上的意涵，並展現跨資料集的良好泛化能力。 總體而言，本研究提出的框架在準確性與穩健性上均優於現有方法，即便在急性醫療與慢性疾病族群之間存在顯著分佈差異的情境下，仍能保持穩定效能。這些發現凸顯了高效且無需填補的數值嵌入方法，作為未來多樣化 EHR 預後建模的可靠基礎，具有相當的潛力與應用價值。 | zh_TW |
| dc.description.abstract | Irregularly sampled multivariate time series in electronic health records (EHRs) present major challenges for prognostic modeling due to uneven sampling, asynchronous measurements, and pervasive missing data. We propose a novel embedding framework that combines a broadcast-based architecture with a Hadamard Product Numerical Embedding (HPNE) module. HPNE partitions each feature embedding into subspaces and modulates them with value-dependent gates, enabling fine-grained representation of sparse numerical features without imputation. This preserves feature identity while enhancing the capture of cross-feature interactions. A broadcast mechanism further integrates feature and value dimensions efficiently, yielding representations that are robust to missingness and scalable to heterogeneous clinical datasets.
We evaluate the model on intensive care unit (ICU) cohorts (PhysioNet 2012 and MIMIC-III) for in-hospital mortality prediction and on a chronic disease cohort (HCC, hepatocellular carcinoma) for long-term prediction. Across both domains, our method consistently outperforms strong baselines, achieving state-of-the-art predictive accuracy while requiring no explicit imputation. Transfer learning experiments further show that embeddings pretrained on a large ICU dataset improve performance on smaller cohorts, demonstrating that HPNE captures semantically meaningful feature representations with strong cross-dataset generalizability. Overall, our framework surpasses the prior approaches in both accuracy and robustness, even under distribution shifts between acute and chronic care populations. These findings highlight the potential of efficient, imputation-free embeddings as a foundation for reliable prognostic modeling across diverse EHR applications. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2025-11-26T16:26:29Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2025-11-26T16:26:29Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | Acknowledgements i
摘要 iii Abstract v Contents vii List of Figures xi List of Tables xiv Chapter 1 Introduction 1 Chapter 2 Related Work 4 2.1 Handling Missing Values in Multivariate Time Series . . . . . . . . . 4 2.2 Imputation-Free and Continuous-Time Modeling . . . . . . . . . . . 5 2.3 Transformer-Based Models for Multivariate Time Series . . . . . . . 5 2.4 Transformer Approaches for EHR Data . . . . . . . . . . . . . . . . 6 Chapter 3 Methodology 8 3.1 Input Representation and Preprocessing . . . . . . . . . . . . . . . . 8 3.1.1 Summarization Strategy . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1.2 Value-Centric Representation with EVAT . . . . . . . . . . . . . . 9 3.1.3 Mask Construction . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.2 Token Embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2.1 Hadamard Product Numerical Embedding (HPNE) . . . . . . . . . 11 3.2.2 Broadcast Modulation . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.2.3 Categorical Embedding . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2.4 Temporal Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3 Transformer Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.3.1 Sequence construction (flatten) . . . . . . . . . . . . . . . . . . . . 15 3.3.2 Self-Attention Mechanism . . . . . . . . . . . . . . . . . . . . . . 16 3.3.3 Self-Attention Mechanism with HPNE . . . . . . . . . . . . . . . . 19 3.4 Aggregation and Prediction Head . . . . . . . . . . . . . . . . . . . 20 3.5 Focal Loss and the Optimization Target . . . . . . . . . . . . . . . . 21 3.6 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.6.1 Accuracy and Confusion Matrix . . . . . . . . . . . . . . . . . . . 22 3.6.2 Confusion Matrix related rates. . . . . . . . . . . . . . . . . . . . . 22 3.6.3 Area under the Precision–Recall curve (AUPRC) . . . . . . . . . . 23 3.6.4 Area under the ROC curve (AUROC) . . . . . . . . . . . . . . . . 24 3.6.5 Concordance Index (c-index) . . . . . . . . . . . . . . . . . . . . . 24 Chapter 4 Experimental Settings 26 4.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.1.1 PhysioNet 2012 Challenge Dataset (P12) . . . . . . . . . . . . . . . 26 4.1.2 MIMIC-III Dataset (MI3) . . . . . . . . . . . . . . . . . . . . . . . 27 4.1.3 NTUH HCC Cohort Dataset (HCC) . . . . . . . . . . . . . . . . . 28 4.2 Baseline Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.3 Training and Evaluation Setting . . . . . . . . . . . . . . . . . . . . 31 4.3.1 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.3.2 Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.3.3 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Chapter 5 Results and Discussion 34 5.1 Overall Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.2 Transfer learning across ICU datasets . . . . . . . . . . . . . . . . . 36 5.3 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.3.1 Feature–Value Binding Strategies . . . . . . . . . . . . . . . . . . . 38 5.4 Effect of the value projection dimension k . . . . . . . . . . . . . . . 40 5.5 Interpreting Feature Embedding Structures . . . . . . . . . . . . . . 42 5.6 Clinical Interpretability of Feature Attention Networks . . . . . . . . 44 5.6.1 ICU mortality (MIMIC-III). . . . . . . . . . . . . . . . . . . . . . . 44 5.6.2 Chronic liver disease (HCC). . . . . . . . . . . . . . . . . . . . . . 45 5.7 Temporal encoding methods comparison . . . . . . . . . . . . . . . 45 Chapter 6 Limitations and Future Work 50 6.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.1.1 Practical constraints in deployment . . . . . . . . . . . . . . . . . . 50 6.1.2 Evaluation scope . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 6.2.1 Multimodal extensions . . . . . . . . . . . . . . . . . . . . . . . . 51 6.2.2 Improving interpretability . . . . . . . . . . . . . . . . . . . . . . . 52 6.2.3 Scaling and pretraining . . . . . . . . . . . . . . . . . . . . . . . . 52 6.2.4 Dynamic feature encoding . . . . . . . . . . . . . . . . . . . . . . 52 Chapter 7 Conclusion 54 References 56 | - |
| dc.language.iso | en | - |
| dc.subject | 深度學習 | - |
| dc.subject | 電子病歷 | - |
| dc.subject | 缺失值處理 | - |
| dc.subject | 多變量時間序列資料 | - |
| dc.subject | deep learning | - |
| dc.subject | electronic health record | - |
| dc.subject | missing value | - |
| dc.subject | multivariate time series data | - |
| dc.title | 不規則多變量時間序列中數值特徵的高效表徵方法：應用於加護醫療與慢性疾病預測 | zh_TW |
| dc.title | Efficient Representation of Numerical Embeddings in Irregular Multivariate Time Series: From Intensive Care to Chronic Disease Prediction | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 114-1 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 孫紹華;劉子毓 | zh_TW |
| dc.contributor.oralexamcommittee | Shao-Hua Sun;Tzu-Yu Liu | en |
| dc.subject.keyword | 深度學習,電子病歷缺失值處理多變量時間序列資料 | zh_TW |
| dc.subject.keyword | deep learning,electronic health recordmissing valuemultivariate time series data | en |
| dc.relation.page | 61 | - |
| dc.identifier.doi | 10.6342/NTU202504465 | - |
| dc.rights.note | 未授權 | - |
| dc.date.accepted | 2025-09-25 | - |
| dc.contributor.author-college | 電機資訊學院 | - |
| dc.contributor.author-dept | 電信工程學研究所 | - |
| dc.date.embargo-lift | N/A | - |
| 顯示於系所單位： | 電信工程學研究所 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-114-1.pdf 未授權公開取用 | 6.32 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。