FastEval Parkinsonism：基於深度學習以手部影像建構之帕金森氏症狀線上即時自我評估系統—以手指打拍為例

Abstract

巴金森氏症候群是國人常見的神經退化性疾病，通常好發於60歲以上的老年人，其臨床上典型症狀包含四肢顫抖、無力、動作緩慢、易駝背、走姿手臂擺幅減少、步行時有小碎步等。MDS-UPDRS巴金森症狀衡量表為醫師臨床上評估症狀嚴重程度之依據，其中第三部分針對動作能力進行評估，包含手指打拍、手掌握合、前臂迴旋等三項手部動作。然而，醫師雖然能以此量表敘述作為評估症狀的依據，但其量表定義未能完全量化，臨床診斷上，常須依靠醫師的經驗及主觀判斷。此外，當患者未到醫院就醫時，難以自我量化評估手部功能是否因藥物控制而獲得改善。因此，為了解決評分不一致以及遠端自我評估的問題，本研究志在開發一巴金森症狀線上評估系統，供使用者上傳自我錄製的手部動作影片，透過深度學習的模型架構來預測手部的MDS-UPDRS巴金森症狀衡量表分數，並同時呈現量化的數值指標，以達到客觀且快速的初步自我評估之目的。本篇研究將以手指打拍為例，完整說明系統流程架構、模型訓練方法與網頁介面建構，並透過分析臨床特徵與預測值之關聯性，以驗證模型精準度及適用範圍。

此研究採用自2020年至2022年於台大醫院總院與癌醫分院收案之資料集，進行210個試驗，有186位受試者參與試驗，包含健康受試者、典型巴金森氏症患者、多重系統退化症患者等，共收集了840部影片。每次試驗，受試者將錄製左、右手各約十秒之短手指打拍影片兩次。透過Google所開發之MediaPipe應用程式介面(API)，可以將手部關鍵點雲特徵從影片中擷取出來，並藉由空洞卷積深度學習模型，以進行模型MDS-UPDRS量表分數之分類任務，同時搭配3D點雲特徵旋轉、隨機切割兩項資料增強技術，以平衡並過取樣資料集，藉此提升模型之精準度。另一方面，手部關鍵點雲特徵也可以用於計算大拇指與食指之距離，可作為量化指標以供分析驗證深度學習模型之效度。

結果顯示以資料增強技術有助於深度模型的訓練。此外，在不同架構的模型中，原始的PDHandNet設計有較好的適用性與計算效率。在多類別MDS-UPDRS的左手及右手分類任務的測試集中，分別具有88.0% 與81.5% 的可容許準確度以及0.4328與0.3177的Cohen Kappa係數。儘管其準確度仍有進步空間，深度學習預測模型的預測結果，經手部定量參數（如：頻率、振幅）以及外部資料集PDMotorDB驗證，仍能區分臨床上的巴金森症狀之嚴重程度。

本研究將深度學習模型與手部定量參數算法匯集成一手部分析預測應用程式介面(Hand Predictor API)，並建立FastEval Parkinsonism網頁 (https://fastevalp.cmdm.tw/)，供使用者能夠經由手機或電腦上傳各式裝置所錄製之彩色手指打拍影片，並快速地進行分析。此網頁有三項特點：其一，經由Hand Predictor API所計算之多個評估指標、手部參數之變化、影片將同時呈現於同一頁面，供使用者能相互參照。其二，為了使左右手的資訊能相互參照，前述之評估指標將以80-20分數量表以雷達圖方式表示，能夠清楚表達左右手的優劣勢以及是否有出現不對稱的巴金森典型特徵。最後，若使用者可以定期量測各項評估指標，將可以即時追蹤其自身的狀態，同時，醫師能根據變化適時調整治療策略，實現個人化醫療。

總結而言，本研究之系統有助於一般老年使用者的家中自我評估巴金森氏症狀，並能協助醫生有效的做量化評估病人隨時間之左右手狀態，能減輕醫師的負擔，提升綜合醫療服務品質。

Parkinson’s disease (PD) is a common neurodegenerative disease that usually occurs in elders over 60 year of age. The Motor Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS) is the gold standard for physicians to clinically evaluate the severity of clinical symptoms (or parkinsonism). The finger-tapping task comprises a motor assessment for bradykinesia, a kind of parkinsonism, in the third part of the MDS-UPDRS. However, although clinicians can assess the patient’s symptoms using this scale description, the evaluation scale is not fully quantified. Thus, the clinical and time-consuming evaluation largely depends on the physician’s experience and subjective judgement. Furthermore, it is hard to monitor the patients’ latest movement status of movements at home.

Thus, an online self-evaluation system for parkinsonism was developed to address the issues of the inconsistency of movement evaluation and the difficulty of self-monitoring. This system allows users to upload self-recorded hand movement videos and predict the MDS-UPDRS score for each hand based on the deep learning-based model. The quantitative hand parameters are also provided to make users preliminarily assess the health status of hands objectively and quickly. This study used take finger tapping as an example to fully explain the system’s framework and model-building procedures and the construction of the web interface. In addition, the accuracy and applicability domain of the model was verified by analyzing the correlation between clinical characteristics and predicted values.

This study collected data from 210 trials from 186 participants at the National Taiwan University Hospital from 2020 to 2022. Finger-tapping videos for each hand were recorded for about 10 seconds twice per trial. The MediaPipe application programming interface (API) developed by Google was used to extract the key point cloud features of the hand from the video. Then, a dilated convolution neural network, combined with two data enhancement technologies (3D key point rotation and the random cropping technique) was used to predict the MDS-UPDRS score. Finally, the distance between the thumb and index finger was calculated from the 3D key points of the hand as a quantitative indicator to verify the performance of the deep learning model.

The results showed that data augmentation techniques were helpful for training deep models. In addition, among models with different architectures, the original PDHandNet design had better applicability and computational efficiency. In the test sets of the left-hand and right-hand classification tasks of the multi-category MDS-UPDRS, the acceptable accuracies were 88.0% and 81.5%, and the Cohen Kappa coefficients were 0.4328 and 0.3177, respectively. While there was room for improvement, the prediction results from the DL model can distinguish the severity of clinical Parkinson's symptoms after verifying by hand quantitative parameters and the external data set, PDMotorDB.

In this study, the algorithms were integrated into a hand analysis application programming interface (Hand Predictor API), and FastEval Parkinsonism (https://fastevalp.cmdm.tw/) was established for users to access. Users can upload and instantly analyze the videos FastEval Parkinsonism has three main features. First, the multiple evaluation indicators calculated by the Hand Predictor API, the changes in hand parameters, and the resulting videos are displayed on the same page for users to refer to. Second, to allow cross-referencing of left and right hand information, the above-mentioned evaluation indicators are expressed as a radar chart with an 80-20 scoring scale, which clearly shows the clinical symmetry of the left and right hands. Finally, if users can regularly measure various evaluation indicators, they will be able to track their own status in real-time.

In summary, the system developed here is helpful for the self-assessment of Parkinson's symptoms at home by older individuals and can assist doctors in effectively quantifying the status of the left and right hands of patients over time, which can reduce the burden on doctors and improve comprehensive medical services quality.