使用機器學習模型預測金融機構業務員舞弊行為-考量型I及型II錯誤機率及舞弊行為造成之損失

Abstract

近年來機器學習技術運用在金融業偵測舞弊的應用越來越廣泛，目前多數場景在評估模型成效時大多使用Precision、F1-score或PR-AUC等指標，但以上指標並未將型1錯誤和型2錯誤所付出的金錢成本納入考量。因此本文使用金融機構提供的保險業務員舞弊資料集，納入成本分析的觀點，提出一個評估模型成效的新方法，並將該方法套用在隨機森林、XGBoost以及VotingClassifier三種機器學習模型上，比較哪一種模型在偵測業務員舞弊的成效最佳。另外，保險業務員舞弊資料集納入多種面向的特徵，考量特徵維度過大可能對模型預測造成負面影響，本研究使用Boruta演算法進行特徵篩選，並比較特徵篩選前後對於模型預測是否有差異。實驗結果顯示，Random Forest和VotingClassifier的表現非常接近，在使用新方法評估的成效方面皆優於XGBoost，且事前使用Boruta演算法對資料集進行特徵篩選，對於模型的預測表現有提升的效果。

In recent years, the application of machine learning techniques in fraud detection within the financial industry has become increasingly widespread. Model performance is commonly evaluated using metrics such as Precision, F1-score, or PR-AUC. However, these metrics do not take the monetary costs associated with Type I and Type II errors into consideration. Therefore, this study introduces a new evaluation method that incorporates cost analysis to assess model performance using fraud dataset of insurance sales agents provided by financial institution. This method is applied to three machine learning models—Random Forest, XGBoost, and VotingClassifier—to compare which model performs best in detecting insurance sales agent fraud. Additionally, the dataset used for detecting sales agent fraud includes a wide range of features. Considering that high feature dimensionality may negatively impact model performance, this study employs the Boruta algorithm for feature selection and compares the prediction results before and after feature selection. Experimental results show that Random Forest and VotingClassifier perform similarly, both outperforming XGBoost when evaluated using the proposed cost-based method. Moreover, applying the Boruta algorithm for feature selection prior to training improves the performance of the models.