請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/52787
標題: | 互補標籤學習之代理損失架構研究 A New Surrogate Loss Framework for Complementary Learning |
作者: | Yu-Ting Chou 周侑廷 |
指導教授: | 林軒田(Hsuan-Tien Lin) |
關鍵字: | 互補標籤,代理損失, Complementary Label,Surrogate Loss, |
出版年 : | 2020 |
學位: | 碩士 |
摘要: | 在弱監督學習領域中,無偏風險估計量(unbiased risk estimator)在訓練資料與測試資料分佈不一致的情況下常被用於訓練分類器。然而,在複雜的模型中無偏風險估計量經常發生過擬合現象,例如深度學習的情境。本論文針對弱監督學習中的互補標籤學習問題(learning with complementary labels)進行探討,分析無偏風險估計量過擬合的成因。透過一系列實驗分析,我們發現即使無偏風險估計量能產生無偏梯度,但產生的梯度方向與目標梯度方向有顯著差距,且變異數值大。本論文提出一個新的代理損失架構(Surrogate Complementary Loss,簡稱SCL),針對互補標籤學習問題使用最小似然估計設計損失函數,有效降低梯度變異並使梯度方向更接近目標梯度方向。實驗數據顯示SCL方法有效降低過擬合現象,並在多個資料集達成更高的分類準確度。 In weakly supervised learning, unbiased risk estimator (URE) is a powerful tool for training classifiers when training and test data are drawn from different distributions. Nevertheless, UREs lead to overfitting in many problem settings when the models are complex like deep networks. In this thesis, we investigate reasons for such overfitting by studying a weakly supervised problem called learning with complementary labels. We argue the quality of gradient estimation matters more in risk minimization. Theoretically, we show that a URE gives an unbiased gradient estimator (UGE). Practically, however, UGEs may suffer from huge variance, which causes empirical gradients to be usually far away from true gradients during minimization. To this end, we propose a novel surrogate complementary loss (SCL) framework that trades zero bias with reduced variance and makes empirical gradients more aligned with true gradients in the direction. Thanks to this characteristic, SCL successfully mitigates the overfitting issue and improves URE-based methods. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/52787 |
DOI: | 10.6342/NTU202002508 |
全文授權: | 有償授權 |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
U0001-0608202006243200.pdf 目前未授權公開取用 | 2.11 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。