利用深度學習於動態超音波影像進行正中神經的影像分割

Abstract

腕部隧道症候群藉由超音波影像進行的研究必須要仰賴專業醫療人員手動去進行正中神經位置的圈選，耗費大量勞力、技術資源以及時間，且依賴人為觀測的參數以及專業醫療人員個人的經驗，缺乏客觀且全面性的評估，亦無法即時回饋給醫療人員。因此我們將藉由電腦視覺的技術來針對超音波影像進行處理。 考慮到資料蒐集的困難及運算的複雜度，我們縮限於單一影像的處理上，藉由影像的空間特徵進行影像分割。自影像分割可以更進一步被劃分為語意分割以及實例分割，語意分割是針對每個像素進行分類，實例分割進一步的將不同物件進行區分。對於正中神經而言，實例分割與語意分割將會得到相同的效果。 本實驗中與台大醫院合作蒐集共48位病患約30秒且每秒顯示畫格數約為30之手腕橫切面超音波影像，並由專業醫師進行標記正中神經的位置作為資料集。我們將資料進行灰階的預處理並將其中39位病患的資料作為訓練集，另外9位病人的資料作為測試集，來訓練以實例分割為目標的Mask R-CNN以及以語意分割為目標的DeepLab V3+。 我們成功的藉由深度學習架構Mask R-CNN以及DeepLab V3+利用Instance segmentation 與 semantic segmentation的方式將正中神經於超音波影像上進行影像分割，並分別於測試訓練集上得到平均IoU為0.8445與0.8375的成績，同時於預測正中神經質心上得到僅5.1831像素的平均誤差。我們確認了提高IoU能確實降低正中神經質心位置的平均誤差。藉由DeepLab V3+的實驗中得知提高training crop size 以及降低output stride在high level得到解析度較高的feature map能有較好的表現，也成功利用多尺度的影像輸入來解決Mask R-CNN中FPN面對物件大小分布不平均時不能充分被訓練而導致預測失準的問題。使用特徵提取能力較好的CNN結構以及隨機調整亮度和對比能確實提高平均IoU的數值。

An increasing line of evidence shows that the cross-sectional area and motion pattern of the median nerve in symptomatic patients of carpal tunnel syndrome with normal or abnormal nerve conduction studies may be more correlated with their clinical symptoms. However, manual median nerve tracking in ultrasound imaging costs tons of human labor with expertise, and highly depends on the experience of the observer. To overcome these problems, the aim of this research is to apply automatic tracking systems of median nerve in dynamic sonography using deep learning. There are two well-studied topics in computer vision fulfilling our purpose. One is tracking with time sequence data, and another is image segmentation. Consider the computational complexity and the difficulty of data recording, we constrain our problem into per-frame image segmentation with only spatial clues. Image segmentation can be regarded as per-pixel classification and thus deep learning method has out-performed the traditional method in the recent years. Image segmentation can further split into two topics, semantic segmentation for per-pixel classification and instance segmentation, which is about classification of pixels in different objects. We construct the state-of-the-art deep learning model Mask R-CNN and DeepLab V3+ of the instance and semantic segmentation, respectively, to execute image segmentation of the median nerve. 48 patients diagnosed as carpal tunnel syndrome were participated in the research, and a 30 seconds video of dynamic sonography with cross-section view of median nerve was recorded for each of the patient. We randomly chose 39 patients as training set, while the others as validation set. Final evaluations of average IoU of 0.8445 and 0.8375 have been achieved by Mask R-CNN and DeepLab V3+, respectively. The Mask R-CNN approach yielded 5.1831 pixels as the mean error for the centroid of the median nerve through the validation dataset, and the reduction of the mean error was correlated with an increase of average IoU. The results using DeepLab V3+ showed that higher performances were achieved by larger input crop size and smaller output stride during training, which improved the resolution of high level feature map. By applying multi-scale input size augmentation, we successfully solved the unbalanced object size problem that may cause failure of the FPN structure.