基於深度集成模型的飄移偵測與預測區間估計之線上學習架構

Abstract

隨著深度學習技術的普及，其在多個領域中得到廣泛應用。然而在許多實際場景中，模型的精確程度需要被嚴格控制，尤其是在智慧製造、自動駕駛、醫療等對誤差容忍度較低的應用中。一旦模型預測出現錯誤，將可能造成嚴重損失，因此，將每次預測的不確定性量化成為一個重要的挑戰，期望模型在預測目標值的同時，也能告訴使用者模型對預測值有多少信心，藉以提高模型預測的可靠性。 本研究提出了一種基於深度集成模型的飄移偵測與預測區間估計之線上學習架構（DEAPI）。該架構使用深度集成模型對資料不確定性與資模型不確定性進行量化，並行成預測區間，其不僅預測反應變數的分佈，還能持續進行線上學習。透過模型的預測分佈與實際值的差距，本研究提出了一個新的飄移偵測器，用以檢測區間估計與實際結果是否相符。同時，將飄移偵測結果回饋給模型更新機制，使學習率和優化器參數能在線上自動調整，提升模型適應新概念的效率。實驗結果表明，DEAPI能給予品質更好的預測區間估計，只要給予深度學習模型少許離線資料進行預訓練後再啟動線上學習，其在線上學習的表現就可以與當前主流的線上學習模型相當。 總體而言，本研究提出的學習架構能在資料流中不斷學習並進行預測區間的估計，同時也可以檢測不同種類的飄移是否發生來警告使用者，並讓模型學習可以更好的適應到新的概念中，最終可以幫助使用者在不確定性中做出更好的決策。

With the widespread adoption of deep learning technology, it has found extensive applications across various fields. However, in many practical scenarios, the precision of models needs to be strictly controlled, especially in applications such as smart manufacturing, autonomous driving, and healthcare, where the tolerance for errors is low. If the model predictions are inaccurate, it can lead to severe consequences. Therefore, quantifying the uncertainty of each prediction has become a significant challenge. It is hoped that while predicting the target values, the model can also convey the confidence level of these predictions to the users, thereby enhancing the reliability of the model’s predictions. This study proposes an online learning framework for drift detection and prediction interval estimation based on deep ensemble models (DEAPI). This framework uses deep ensemble models to quantify both data uncertainty and model uncertainty, forming prediction intervals. It not only predicts the distribution of the response variables but also continuously learns from data stream. By analyzing the discrepancy between the predicted distribution of the model and the actual values, a novel drift detector is introduced to check if the interval estimates align with the actual outcomes. Additionally, the results of the drift detection are fed back to the model update mechanism, enabling automatic adjustment of learning rates and optimizer parameters online, which improves the model’s efficiency in adapting to new concepts. Experimental results demonstrate that DEAPI provides higher quality prediction interval estimates. Once the deep learning model is given a small amount of offline data for pre-training before starting online learning, its online learning performance can be comparable to the current mainstream online learning models. Overall, the learning framework proposed in this study can continuously learn from data streams and estimate prediction intervals. It can also detect different types of drift to alert users, enabling the model to better adapt to new concepts. Ultimately, this helps users make better decisions amidst uncertainty.