多類別分類指標之區辨力評估

Abstract

在多類別分類問題中，表現機率是分類精確性及分類指標的區辨力的合理評估準則。利用表現機率集合在接收者操作特徵空間中的幾何性質，最大期望效用準則可給出最佳分類程序的明確表達式。同時，透過參數化最佳接收者操作特徵流形，我們可分析其規範性及微分結構。此流形的經驗估計式可被證明具備收斂至一高斯過程，並藉此建立同步信賴區間。延伸二分類中的曲線下面積，此流形下的超體積可做為區辨力的摘要測度。我們給出此超體積可定義以及具備統計意義的條件，並進一步證明其與正確分類機率的等價關係。因此在適當假設下，我們給出對應的U估計式以及參數模型統計推論。相關的數值實驗及資料分析佐證此理論架構的實用性，並可幫助研究者評估多類別分類指標及分類程序。

To evaluate overall discrimination capacity of a marker for multi-class classification tasks, the performance function is a natural assessment tool and fully provides the essential ingredients in receiver operating characteristic (ROC) analysis. The connection between admissible and utility classifiers facilitates illustrating the optimality of likelihood ratio scores as well as constructing a parameterized optimal ROC manifold. The manifolds supply a geometric characterization of the magnitude of separation among multiple classes. It is shown that the hypervolume under the optimal ROC manifold (HUM) is a well-defined and meaningful accuracy measure only in suitable ROC subspaces. Moreover, we provide a rigorous proof for the equality of HUM and its alternative form, the correctness probability, which is directly related to an explicit U-estimator. Our theoretical framework further allows more sophisticated modeling on performance of markers and helps practitioners examine the optimality of applied classification procedures.