智慧型手機建構日常生活活動模型與辨識系統之研究

Abstract

我們利用手機上的照相機，錄音機以及三軸加速器，結合 SenseCam 的特性而發明一個手機的新應用，我們稱之為 SenseMobile 。 SenseMobile 結合影像，聲音，以及加速度值來建立日常生活活動辨識系統。然而，其他可攜帶型的裝置卻僅僅是辨識一些基本的身體活動，而不是較進階的日常生活活動 (ADLs) ，但在本論文裡，我們萃取實用的特徵值來辨識日常生活活動。 在影像特徵萃取裡，經過影像前處理再取得影像裡的人臉資訊，以及建立局部影像分類系統；而在聲音特徵擷取裡，我們從時間領域，抓取音量、非靜音率、以及辨識人聲的最大峰值，波峰數量，而從頻率領域裡，萃取語音辨識裡常用的梅爾倒頻係數；最後，我們先分類加速值的分佈情形，計算不同分佈占的比例來得到加速度的特徵值。在方法上，我們利用 sliding window 方式進行特徵值取樣，再利用機器學習的數學模型進行辨識。 本論文裡，我們設計兩種實驗：實驗環境裡以及真實環境裡的日常生活活動辨識，在多種活動辨識上，比較 (Support Vector Machine)SVM 和 (Hidden Markov Model)HMM 的精準度，也比較影像，聲音、加速度的特徵值對於多種活動辨識的精準度之優劣，另外在單一活動辨識上，我們針對個別活動分別執行最佳化來提昇活動辨識的精準度。而從這兩種實驗結果證明我們在日常生活活動辨識的成功以及提出需要改進的部份。

We combine the camera, the sound recorder, and the accelerometer on the smart-phone with the machanism of SenseCam to invent a new application of smartphones called as SenseMobile. SenseMobile employs images, sounds and accelerometer values to build an activities of daily living(ADLs) recognition system. However other portable devices merely recognize physical activities instead of high-level activities. In this thesis, we extract effective features to implementing activity recognition. In image feature extraction, we detect human face and cluster local images after pre-processing. For sound feature extraction, in the time domain, we extract volume, non-silent ratio and two human voice features - maximum peak value and number of peaks. Furthermore, From the frequency domain, we extract Mel-frequency cepstral coefficients (MFCCs), which are popular in speech recognition. After clustering vibration types, we calculate probabilities of types in accelerometer feature extraction. Then we sample instances on sliding time window and implement classification on machine learning models. We design two experiments - ADLs recognition in experimental environment and in real environment. In multiple classifications, we compare accuracy from Support Vector Machine(SVM) and Hidden Markov Model(HMM) models, and from distinct data types. In binary classifications, we utilize one-against-all method and optimize individual activity recognition. Eventually, results of two experiments prove success in ADLs recognition and bring forward unsolved defects of SenseMobile.