應用自動微分及截斷牛頓法於條件隨機場

Abstract

近年來,很多領域興起將序列的資料標上標籤。條件隨機場則是一種常用來解此類問題的方法,但其封閉形式的海森矩陣並不易導出。這困難致使一些使用二次微分資訊的最佳化方法不適用,如牛頓法。自動微分則是一種技巧,可以用來計算一個函數的導數值而無梯度函數。並且,藉由自動微分來計算海森矩陣與向量之乘積只需梯度函數而無需海森矩陣。本篇論文先說明自動微分的背景知識。然後結合截斷牛頓法及自動微分,並用之於解決條件隨機場。

In recent years, labeling sequential data arises in many fields. Conditional random fields are a popular model for solving this type of problems. Its Hessian matrix in a closed form is not easy to derive. This difficulty causes that optimization methods using second-order information like the Hessian-vector products may not be suitable. Automatic differentiation is a technique to evaluate derivatives of a function without its gradient function. Moreover, computing Hessian-vector products by automatic differentiation only requires the gradient function but not the Hessian matrix. This thesis first gives a study on the background knowledge of automatic differentiation. Then it merges truncated Newton methods with automatic differentiation for solving conditional random fields.