基於強化學習和基底核-丘腦動態網路之帕金森氏症閉迴路深腦電刺激演算法

Abstract

帕金森氏症 (Parkinson’s disease, PD) 是一種影響中樞神經系統的慢性神經 退化性疾病，目前影響全球約一千萬人 [1]。深腦電刺激 (deep brain stimulation, DBS) 的技術在運動障礙和神經系統疾病中的應用，包括 PD、震顫、肌張力障礙、癲癇、強迫症等，已被證明是一種有效的治療方式 [2]。然而，廣泛使用的 開迴路系統仍然存在一些尚待修正的缺點，例如它們的個體依賴性、能量消耗 程度、頻繁回診和試錯性調整的特徵 [3]。閉迴路的策略採用具有判別性的訊 號或生物標誌物，從而使系統能夠透過算法自動調整 DBS 參數 [3]。我們設計強化學習 (reinforcement learning, RL) 與 Gym 框架，模擬基底神經節-丘腦 (basal ganglia-thalamic, BGT) 大腦網路作為訓練環境，並為任何輸入狀態找到適當的刺激參數(頻率和振幅)。特徵提取模塊則作為 BGT 大腦網路(動作電位訊號)與來自真實大腦的胞外訊號之間的映射工具，進而允許未來的動物實驗和臨床試 驗的測試。結果顯示，基於RL的DBS控制策略在能秏上較開迴路系統節省了 68.81% 的平均功率，並修正丘腦(thalamus, TH)中的錯誤響應(平均錯誤響應在正常情況為 0.0; 在PD下修正回 0.0258)，同時為未來應用奠定了基礎。

Parkinson’s disease (PD) is a chronic neurodegenerative disease affecting the central nervous system and currently influencing about 10 million people worldwide [1]. The usage of deep brain stimulation (DBS) technology in movement disorders and neurological diseases, including PD, tremor, dystonia, epilepsy, obsessive-compulsive disorder (OCD), etc., has proven to be an effective treatment modality [2]. However, general open-loop systems pose several shortcomings that have yet to be revised, such as their subject dependency, energy-consuming, frequent-clinic visiting, and trial-and-error adjusting features [3]. The closed-loop strategy employs discriminative signals/biomarkers to enable the system to tune parameters automatically through the designed algorithms [3]. We designed reinforcement learning (RL) with the Gym framework that models the basal ganglia-thalamic (BGT) brain network as a training environment and finds appropriate stimulation parameters (frequency and amplitude) for different input states. The feature extraction module was a mapping tool between the BGT brain network (AP signals) and extracellular signals from real brains, permitting future animal experiments and clinical trials. Results showed that the RL-based DBS control strategy significantly outperforms open-loop systems in energy efficiency, i.e., conserving 68.81% of average power dissipation, and revises error responses in the thalamus (i.e., an average EI of 0.0 in normal and 0.0258 in PD states) while establishing a foundation for future application.