請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/88407完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 林澤 | zh_TW |
| dc.contributor.advisor | Che Lin | en |
| dc.contributor.author | 張舒翔 | zh_TW |
| dc.contributor.author | Shu-Hsiang Chang | en |
| dc.date.accessioned | 2023-08-15T16:09:23Z | - |
| dc.date.available | 2023-11-09 | - |
| dc.date.copyright | 2023-08-15 | - |
| dc.date.issued | 2023 | - |
| dc.date.submitted | 2023-07-20 | - |
| dc.identifier.citation | B. Shickel, P. J. Tighe, A. Bihorac, and P. Rashidi, “Deep ehr: a survey of recent advances in deep learning techniques for electronic health record (ehr) analysis,” IEEE journal of biomedical and health informatics, vol. 22, no. 5, pp. 1589–1604, 2017.
S. Singh and J. Khim, “Statistical theory for imbalanced binary classification,” arXiv preprint arXiv:2107.01777, 2021. S. Lahiri and S. Lahiri, Resampling methods for dependent data. Springer Science & Business Media, 2003. A. Estabrooks, T. Jo, and N. Japkowicz, “A multiple resampling method for learning from imbalanced data sets,” Computational intelligence, vol. 20, no. 1, pp. 18–36, 2004. C. Huang, Y. Li, C. C. Loy, and X. Tang, “Learning deep representation for imbalanced classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 5375–5384. S. H. Khan, M. Hayat, M. Bennamoun, F. A. Sohel, and R. Togneri, “Cost-sensitive learning of deep feature representations from imbalanced data,” IEEE transactions on neural networks and learning systems, vol. 29, no. 8, pp. 3573–3587, 2017. Y. Cui, M. Jia, T.-Y. Lin, Y. Song, and S. Belongie, “Class-balanced loss based on effective number of samples,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9268–9277. Y. Yang and Z. Xu, “Rethinking the value of labels for improving class-imbalanced learning,” Advances in neural information processing systems, vol. 33, pp. 19 290–19 301, 2020. K. Cao, C. Wei, A. Gaidon, N. Arechiga, and T. Ma, “Learning imbalanced datasets with label-distribution-aware margin loss,” Advances in neural information processing systems, vol. 32, 2019. G. Haixiang, L. Yijing, J. Shang, G. Mingyun, H. Yuanyue, and G. Bing, “Learning from class-imbalanced data: Review of methods and applications,” Expert systems with applications, vol. 73, pp. 220–239, 2017. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988. Q. Dong, S. Gong, and X. Zhu, “Class rectification hard mining for imbalanced deep learning,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 1851–1860. F. H. K. d. S. Tanaka and C. Aranha, “Data augmentation using gans,” arXiv preprint arXiv:1904.09135, 2019. N. Charoenphakdee, J. Vongkulbhisal, N. Chairatanakul, and M. Sugiyama, “On focal loss for class-posterior probability estimation: A theoretical perspective,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 5202–5211. J. Ren, C. Yu, X. Ma, H. Zhao, S. Yi et al., “Balanced meta-softmax for long-tailed visual recognition,” Advances in neural information processing systems, vol. 33, pp. 4175–4186, 2020. R. J. Little and D. B. Rubin, Statistical analysis with missing data. John Wiley & Sons, 2019, vol. 793. J. L. Schafer, “Multiple imputation: a primer,” Statistical methods in medical research, vol. 8, no. 1, pp. 3–15, 1999. Z. Che, S. Purushotham, K. Cho, D. Sontag, and Y. Liu, “Recurrent neural networks for multivariate time series with missing values,” Scientific reports, vol. 8, no. 1, p. 6085, 2018. B. K. Beaulieu-Jones, D. R. Lavage, J. W. Snyder, J. H. Moore, S. A. Pendergrass, and C. R. Bauer, “Characterizing and managing missing structured data in electronic health records: data analysis,” JMIR medical informatics, vol. 6, no. 1, p. e8960, 2018. A. Ghosh, H. Kumar, and P. S. Sastry, “Robust loss functions under label noise for deep neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 31, no. 1, 2017. Z. Zhang and M. Sabuncu, “Generalized cross entropy loss for training deep neural networks with noisy labels,” Advances in neural information processing systems, vol. 31, 2018. M. Ren, W. Zeng, B. Yang, and R. Urtasun, “Learning to reweight examples for robust deep learning,” in International conference on machine learning. PMLR, 2018, pp. 4334–4343. J. Shu, Q. Xie, L. Yi, Q. Zhao, S. Zhou, Z. Xu, and D. Meng, “Meta-weight-net: Learning an explicit mapping for sample weighting,” Advances in neural information processing systems, vol. 32, 2019. Z. Liu, P. Wei, J. Jiang, W. Cao, J. Bian, and Y. Chang, “Mesa: boost ensemble imbalanced learning with meta-sampler,” Advances in Neural Information Processing Systems, vol. 33, pp. 14 463–14 474, 2020. E. Jang, S. Gu, and B. Poole, “Categorical reparameterization with gumbel-softmax,” arXiv preprint arXiv:1611.01144, 2016. M. W. Gardner and S. Dorling, “Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences,” Atmospheric environment, vol. 32, no. 14-15, pp. 2627–2636, 1998. L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers. Springer, 2010, pp. 177–186. P. J. Werbos, “Backpropagation through time: what it does and how to do it,” Proceedings of the IEEE, vol. 78, no. 10, pp. 1550–1560, 1990. V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann ma- chines,” in Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 807–814. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. M.-T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” arXiv preprint arXiv:1508.04025, 2015. Y. Wang, M. Huang, X. Zhu, and L. Zhao, “Attention-based lstm for aspect-level sentiment classification,” in Proceedings of the 2016 conference on empirical methods in natural language processing, 2016, pp. 606–615. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” arXiv preprint arXiv:1910.13461, 2019. E. Choi, H. He, M. Iyyer, M. Yatskar, W.-t. Yih, Y. Choi, P. Liang, and L. Zettlemoyer, “Quac: Question answering in context,” arXiv preprint arXiv:1808.07036, 2018. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020. Y. Li, S. Rao, J. R. A. Solares, A. Hassaine, R. Ramakrishnan, D. Canoy, Y. Zhu, K. Rahimi, and G. Salimi-Khorshidi, “Behrt: transformer for electronic health records,” Scientific reports, vol. 10, no. 1, pp. 1–12, 2020. A. Sherstinsky, “Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network,” Physica D: Nonlinear Phenomena, vol. 404, p. 132306, 2020. K. O’Shea and R. Nash, “An introduction to convolutional neural networks,” arXiv preprint arXiv:1511.08458, 2015. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778. J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018. A. Radford, K. Narasimhan, T. Salimans, I. Sutskever et al., “Improving language understanding by generative pre-training,” 2018. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” The Journal of Machine Learning Research, vol. 21, no. 1, pp. 5485–5551, 2020. Liu, “Using lightweight self-attention based models on high missing rate time series electronic health records,” 2022, unpublished. F. E. Harrell, R. M. Califf, D. B. Pryor, K. L. Lee, and R. A. Rosati, “Evaluating the yield of medical tests,” Jama, vol. 247, no. 18, pp. 2543–2546, 1982. T. Saito and M. Rehmsmeier, “The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets,” PloS one, vol. 10, no. 3, p. e0118432, 2015. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning. PMLR, 2017, pp. 1126–1135. M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. De Freitas, “Learning to learn by gradient descent by gradient descent,” Advances in neural information processing systems, vol. 29, 2016. S. Ravi and H. Larochelle, “Optimization as a model for few-shot learning,” in International conference on learning representations, 2017. G. Koch, R. Zemel, R. Salakhutdinov et al., “Siamese neural networks for one-shot image recognition,” in ICML deep learning workshop, vol. 2, no. 1. Lille, 2015. J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” Advances in neural information processing systems, vol. 30, 2017. T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 2623–2631. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415, 2016. C. Chen, A. Liaw, L. Breiman et al., “Using random forest to learn imbalanced data,” University of California, Berkeley, vol. 110, no. 1-12, p. 24, 2004. T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785–794. E. L. Kaplan and P. Meier, “Nonparametric estimation from incomplete observations,” Journal of the American statistical association, vol. 53, no. 282, pp. 457–481, 1958. J. Byrd and Z. Lipton, “What is the effect of importance weighting in deep learning?” in International Conference on Machine Learning. PMLR, 2019, pp. 872–881. T. Fang, N. Lu, G. Niu, and M. Sugiyama, “Rethinking importance weighting for deep learning under distribution shift,” Advances in Neural Information Processing Systems, vol. 33, pp. 11 996–12 007, 2020. A. Rajkomar, E. Oren, K. Chen, A. M. Dai, N. Hajaj, M. Hardt, P. J. Liu, X. Liu, J. Marcus, M. Sun et al., “Scalable and accurate deep learning with electronic health records,” NPJ digital medicine, vol. 1, no. 1, pp. 1–10, 2018. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/88407 | - |
| dc.description.abstract | 數據不平衡是深度學習中一個實際且關鍵的問題。此外，在現實世界中，電子醫療病例數據集經常遇到高缺失率的情況。 這兩個問題都可以視為數據中的雜訊，並可能對一般的深度學習算法的泛化性能造成不利影響。在醫療領域中，準確的預測和分類是至關重要的，因為這直接關係到患者的診斷和治療結果。為了解決這些問題，我們在本研究中引入了一種新的元學習(Meta-learning)方法，專門處理電子醫療病例數據集中的二元分類任務中的數據雜訊。這種元學習方法利用從平衡且低缺失率數據中選擇的子集信息，為每個樣本自動分配適當的權重。這樣的權重分配將增強有用信息的樣本並在訓練過程中抑制噪聲樣本的干擾。這樣的處理方式使得模型能夠更好地處理不平衡和缺失的數據，從而提高了預測的準確性和泛化能力。值得注意的是，這種元學習方法與深度學習模型的架構無關，可以同時應對高不平衡率和高缺失率問題。通過實驗驗證，我們證明了這種元學習方法在極端情況下具有更好的效果。在不平衡率為172且缺失率為74.6%的最極端情況下，我們的方法優於沒有元學習的原始模型多達10.3%的接收者操作特徵曲線(AUROC)下的面積，以及3.2%的精確召回曲線（AUPRC）下的面積。我們的結果為訓練有極度雜訊的EHR資料集的模型邁出了重要的第一步。這意味著我們能夠處理高度不平衡和高缺失率的數據，並且在這些困難情況下提升了模型的預測性能。我們相信，隨著更多的研究和創新，元學習將為深度學習領域帶來更多的突破和進展，並在解決實際問題時發揮更大的作用。 | zh_TW |
| dc.description.abstract | Data imbalance is a practical and crucial issue in deep learning. Moreover, real-world datasets, such as electronic health records (EHRs), often suffer from high missing rates. Both issues can be understood as noises in data that may lead to poor generalization results for standard deep learning algorithms. This thesis introduces a novel meta-learning approach to deal with these noise issues in an EHR dataset for a binary classification task. This meta-learning approach leverages the information from a selected subset of balanced, low-missing rate data to automatically assign proper weight to each sample. Such weights would enhance the informative samples and suppress the opposites during training. Furthermore, the meta-learning approach is model-agnostic for deep learning-based architectures that simultaneously handle the high imbalance ratio and high missing rate problems. Through experiments, we demonstrate that this meta-learning approach is better in extreme cases. In the most extreme one, with an imbalance ratio of 172 and a 74.6% missing rate, our method outperforms the original model without meta-learning by as much as 10.3% of the area under the receiver-operating characteristic curve (AUROC) and 3.2% of the area under the precision-recall curve (AUPRC). Our results mark the first step towards training a robust model for extremely noisy EHR datasets. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2023-08-15T16:09:23Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2023-08-15T16:09:23Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | Contents Page
口試委員審定書 i 誌謝 ii 摘要 iii Abstract iv Contents v List of Figures viii List of Tables ix Chapter 1 Introduction 1 Chapter 2 Electronic health record (EHR) data 7 2.1 Data Description 7 2.2 Features 8 2.3 Preprocessing 12 2.3.1 Data Cleaning 14 2.3.2 Cohort Study Data Selection 15 2.3.3 Summarization 16 2.3.4 Imputation 18 2.4 Subgroups of the Cohort Data 18 Chapter 3 Methodology 22 3.1 Feedforward Neural Network 22 3.2 Self-Attention 24 3.2.1 Transformer 24 3.2.2 Transformer for EHR datasets 27 3.3 Evaluation Metrics 29 3.3.1 AUPRC & AUROC 29 3.3.2 Concordance Index 31 3.4 Focal Loss 32 3.5 Meta-Learning 33 3.5.1 Learning to reweight (L2RW) 34 3.5.2 Meta-Reweighting 35 3.5.3 Training process of meta re-weighting 36 3.5.4 Selection of an unbiased and clean validation set 37 3.5.4.1 Low missing rate criterion 37 3.5.4.2 Low classification error criterion 37 3.5.5 Physical interpretation of reweighting 38 Chapter 4 Experiments 41 4.1 Prediction Task 41 4.2 Training Details 41 4.3 Results at different imbalance ratios 42 4.4 Low missing rate criterion and low classification error criterion (Meta-EHR & Meta-EHR-CE) 44 4.5 Kaplan-Meier Survival Analysis for First Diagnosis Patients 46 Chapter 5 Discussion 50 5.1 Meta-learning can generalize to extremely small imbalanced datasets 50 5.2 The performance of meta-learning in different subgroups also has the same trend 52 5.3 Compare the results between Meta-EHR and the other reweighting techniques 58 Chapter 6 Conclusion 62 Bibliography 64 | - |
| dc.language.iso | en | - |
| dc.subject | 電子醫療病例 | zh_TW |
| dc.subject | 缺失值 | zh_TW |
| dc.subject | 不平衡學習 | zh_TW |
| dc.subject | 元學習 | zh_TW |
| dc.subject | 時間序列 | zh_TW |
| dc.subject | imbalanced learning | en |
| dc.subject | missing data | en |
| dc.subject | meta-learning | en |
| dc.subject | Electronic health records | en |
| dc.subject | time-series | en |
| dc.title | Meta-EHR：針對具有高不平衡與缺失率電子醫療病歷之元學習方法 | zh_TW |
| dc.title | Meta-EHR: A meta-learning approach for electronic health records with a high imbalance ratio and missing rate | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 111-2 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 王偉仲;李宏毅 | zh_TW |
| dc.contributor.oralexamcommittee | Wei-Chung Wang;Hung-Yi Lee | en |
| dc.subject.keyword | 電子醫療病例,不平衡學習,缺失值,元學習,時間序列, | zh_TW |
| dc.subject.keyword | Electronic health records,imbalanced learning,missing data,meta-learning,time-series, | en |
| dc.relation.page | 71 | - |
| dc.identifier.doi | 10.6342/NTU202301473 | - |
| dc.rights.note | 同意授權(限校園內公開) | - |
| dc.date.accepted | 2023-07-21 | - |
| dc.contributor.author-college | 電機資訊學院 | - |
| dc.contributor.author-dept | 電信工程學研究所 | - |
| 顯示於系所單位： | 電信工程學研究所 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-111-2.pdf 授權僅限NTU校內IP使用（校園外請利用VPN校外連線服務） | 2.71 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。