以卷積神經網路實現之頭戴式視線追蹤系統

Abstract

從科學研究到商業應用如人機介面、心理學、人類行為，眼神追蹤均為一個極重要的工具，尤其在虛擬實境與擴增實境等頭戴式裝置內，視線資訊更可以做為一個最自然的使用者輸入。然而市面上多數眼神追蹤技術都需要一到數個紅外光光源來定位與追蹤瞳孔輪廓，使得許多眼動儀在有太陽光的室外環境下，精準度大幅下降甚至無法運作，擴增實境的使用範圍也因此受限。 為解決這個問題，架設於頭戴式裝置內、單純基於可見光的圖像表徵演算法應運而生。然而，儘管藉由多點校正可以讓圖像表徵演算法有一定的精準度，由於此類演算法使用整張的眼睛圖片，若多張圖片之間因相機位移導致對齊程度下降，演算法的精準度亦會同時劇烈下滑。 因此，比起直接使用未經處理的原始圖片資訊，我們需要抽取更穩定、更具代表性的特徵。基於卷積神經網路的蓬勃發展，在這份研究裡，我們藉由卷積神經網路來抽取眼睛圖片的抽象資訊，再將此抽象特徵經由九點線性校正技術，來投射出相對應的二維視線座標點。 實驗結果顯示，在不需額外追蹤擷取眼睛區域、涵蓋 80 種隨機光線的測試下，本研究提出之眼神追蹤系統針對 10 位受試者的精準度可以降到平均 2 度誤差內。

From researches to commercial products such as psychology, human behavior and human-machine interface, eye-tracking is an important tool across many domains; especially in head-mounted devices, for example, Virtual-Reality(VR) and Augmented-Reality(AR), gaze information is the most natural user input. However, most of the commercial eye-tracking techniques use several infrared ray(IR) illuminators as additional light sources to detect and track the pupil contour. When using these eye-trackers in the outdoor environments under the sunlight, most of them can not work, which limits the usage of wearable AR devices. To address this issue, head-mounted, appearance-based algorithms have been proposed recently. However, since these kinds of algorithm directly process the raw eye images, they need multiple calibrating points to achieve the same accuracy as IR-based methods; when eye images not keeping aligned due to camera-eye shift, the error of eye gaze prediction will dramatically increase. In this research, owing to the rapid development of Convolutional Neural Network(CNN), we take advantage of the CNN model to extract robust and representative eye features, then apply a 9-point linear combination tenique on those features to map the eye appearance to the 2D gaze position. From the experimental results, the proposed eye-tracking system can achieve a mean error less than 2 degrees for 10 testing subjects, under 80 lighting conditions without detecting and cropping the eye region.