DeepGesture:利用卷積類神經網絡改善運動神經損傷者觸控手勢的辨識

Abstract

銀髮族與上肢行動不便者具有操作平板上的困難,目前常見手機預設的辨識器未必適合特殊需求的人們,我們提出 DeepGesture , 學習使用者的行為,建立兩階段的模型,在第一階段,卷積網絡模型分辨使用者手勢,模型顯著地改善常見手勢的辨識率如點擊與滑動,在點擊手勢辨識成功後,利用嶄新的點擊優化器找出最有重要性的觸控點,以提升點擊的成功率。結果顯示 DeepGesture 可以比預設系統的辨識器達到更高的成功率。

Elderly people and the motor impaired have difficulty in interacting with touch screen devices. Commonly-used mobile system uses a general model for gesture recognition. However, the general threshold-based model may not meet their special needs. Hence, we present DeepGesture, a 2-stage model providing self-learning function for gesture recognition. In first stage, convolution neutral network is used to classify gesture.It remarkably improves the success rate of recognizing common gestures, such as tap and pan etc. After tapping gesture recognized,a novel tap optimizer is used to choose most important touch point to obtain higher tapping success rate. The results show that DeepGesture achieves a higher success rate than iOS default recognizer.