使用於智慧化教室的人臉偵測、追蹤及辨識

Abstract

隨著卷積神經網路的推演，人臉相關技術跟以前相比已有大規模的突破，在人臉偵測以及人臉辨識已經達到商用的水平。本論文主要基於Insightface的人臉偵測以及識別的框架來嘗試一些改善的方法，包含以人臉追蹤來改進人臉識別的辨識率，並使用深度殘差等變映射模型DREAM block實現正臉和側臉之間的特徵向量轉換。首先，在人臉偵測方面，我們挑選了兩種人臉偵測演算法，MTCNN和RetinaFace，在我們的資料集上評估此兩種方法的偵測效能，實驗結果顯示RetinaFace效果比較好。然後我們再進行人臉校正以及各項正規化處理後，基於四種特徵提取模型（LResNet34、50、100E-IR，MobileFaceNet）來抽取特徵向量，最後進行人臉辨識效能比較。我們資料集的場景主要為教室，總共有三種資料集，面臨多角度人臉辨識及追蹤問題，以及坐在教室後排較小人臉辨識問題，透過DRAEM block在MobileFaceNet以及LResNet34E-IR模型上人臉辨識平均提升0.06% (1號資料集提升0.03%、2號資料集提升0.1%、3號資料集提升0.03%)。使用人臉追蹤輔助人臉識別使我們的人臉識別準確度平均提高2.12% (1號資料集提升1.18%、2號資料集提升3.85%、3號資料集提升1.33%)。

Due to the success of CNN, face-related technologies have achieved large-scale breakthroughs recently, and have reached a mature level for commercial applications. Based on Insightface's face detection and recognition framework, this study proposes some improved methods for face recognition, including the use of face-detection-based face tracking for robust face recognition, and the use of DREAM block for feature conversion between front and profile faces. More specifically, we evaluated two face detection algorithms, MTCNN and RetinaFace, based on our classroom video datasets. Once face detection, we perform face alignment and several normalization techniques, and then send the face to four feature extractors, including LResNet34、50、100E-IR,MobileFaceNet. The final step of face recognition is based on the distance/similarity of two face vectors. There are three kinds of datasets in total. We are facing multiple face recognition problems and smaller face recognition problems for students far from the camera. By using DRAEM block, the LResNet34E-IR and MobileFaceNet model for face recognition can achieve an average accuracy improvement of 0.06% (from 80.89% to 80.92% for corpus 1, from 43.52% to 43.62% for corpus 2, and from 53.58% to 53.61% for corpus 3). By using face tracking to enhance face recognition, the average accuracy improvement is 2.12% (from 97.57% to 98.75% for corpus 1, from 63.51% to 67.36% for corpus 2, and from 67.14% to 68.47% for corpus 3).