多類別遞迴最小平方誤差分類器

Abstract

區別分類器(Discriminant Classifier)是一種已知既有類別的機器學習技術。主要有費雪區別(Fisher’s Discriminant)及最小平方誤差區別(Minimum-Squared-Error)兩種。一般而言，最小平方誤差區別是用在兩類的資料上。多類別最小平方誤差區別將其延伸到三個類別以上的資料，利用葛蘭-舒密特過程(Gram-Schmidt process)將一組線性獨立的向量轉為一組互相直交且長度為一的類別標籤向量。類別標籤向量必須要直交，以使區別向量之間也互相直交。但是，給予不同的線性獨立向量，多類別最小平方誤差區別會解出不同的類別標籤向量，因而所求出的區別向量也會不同，此方法所求出的解可能並不是最佳解。 本研究發展一個遞迴的演算法，以求得一組區別向量及類別標籤向量來同時使得目標為最佳。此演算法的目標為得到一組區別向量，使得樣本求出的區別分數與其類別標籤越近越好。此演算法由冪法(Power Method)來證明其收斂。本研究所提出的區別方法稱為「多類別遞迴最小平方誤差區別」。 經由上述方法，我們可得到樣本區別分數，要依照其區別分數分類有許多可用的準則。分類準則主要有兩種，一是以距離為基礎，另一是以機率為基礎。此論文中會介紹四種分類準則，鳶尾花(Iris)的資料集將會被用來作為此遞迴演算法及分類準則的範例。 最後，我們會用兩組真實世界中的多類別資料來測試。比較此遞迴演算法搭配不同分類準則的準確度，以及此分類器與費雪區別分類器、多類別最小平方誤差區別分類器的績效。

Discriminant classifier is a type of supervised machine learning technique. There are two approaches to it. One is the Fisher’s discriminant; the other is the Minimum-Squared-Error (MSE) discriminant. The MSE discriminant is usually used to deal with two-class problems. The multi-class MSE approach extends the MSE discriminant to allow problems with more than two classes by providing a set of orthonormal class-label vectors through the Gram-Schmidt process. The resulting class-label vectors are made orthonormal so that the discriminants can be orthogonal as well. However, by giving different linearly independent vectors to the Gram-Schmidt process, the resulting class-label vectors will be different and so do the corresponding discriminants. That is, the solution of multi-class MSE is not unique and may not be the optimal. This research develops an iterative algorithm to obtain the class-label vectors and the discriminant loadings simultaneously while the objective is achieved. The objective is to make the discriminant scores as close to its corresponding class labels as possible. The iterative process is proven to be converged by the power method. The multi-class discriminants found through this iterative algorithm is called multi-class iterative MSE discriminants (IMSED). Through discriminant approaches, we will obtain the discriminant score for each instance. To allocate the instances to classes, there are mainly two types of classification rules. One is distance based; the other is probability based. Four classification rules will be discussed in this research and a probabilistic classification rule will be developed. Iris dataset will be used to illustrate the iterative algorithm and the classification rules. Finally, two real-world data sets with multiple classes are used to compare the IMSED classifier with the Fisher’s discriminant classifier and the multi-class MSE discriminant classifier.