L1正規化的平滑近似方法之比較

Abstract

L1正規化的分類器被廣泛應用于獲取稀疏模型，但是其二階不可導特性對優化過程帶來了很大的挑戰。本文中，消除非約束性優化方法的使用限制,通過平滑近似L1的方法使其二階可微，從而可以使用常見的牛頓法解決。進一步探討該方法的應用空間，我們將之與L1和L2正規化問題的最佳化方法做了詳細的對比實驗，結果證實平滑化近似法繼承了L1和L2的某些特性，但應用前景依然不容樂觀。

L1-regularized classi ers are widely used to obtain sparse models; however, thenon-di erentiability of L1-regularized form causes more challenges in optimization. In this thesis, in order to eliminate the limited use of standard unconstrained methods in L1-regularized problems, smooth convex pproximations are used to replace the absolute form to make the problem twice-di erentiable. Thus we can use popular Newton methods to solve the reformulated problem. Results show that after approximation, the modified problem behaves in the middle of L1- and L2-regularized problems. To further investigate the pratical application of this approximation method, we also conduct experiments to compare with the state-of-art methods for olving L1- and L2-regularization problems.