KUNet: 基於改良式深層U-Net於顯微鏡影像分割之時空網路

Abstract

由於活細胞的時間行為，顯微鏡影像切割至今仍然是非常具有挑戰性的問題，Unet架構已被認為是細胞影像分割的強大方法。 一般而言，人類專家在標註細胞資料時，經常依賴於時空間的連貫性，以便準確地分離相鄰的細胞並檢測部分可見的細胞。然而，利用時間、空間的特性來處理有時序性細胞影像分割的state of the art Unet，依然沒有明確被提出。本研究利用來處理具時序性空間特徵卷積長短期記憶運算(CLSTM)，以及密集連接(dense connection)的方法取代部分原本Unet的2D convolution的計算，達成state of the art的結果。在本研究中，我們比較了八種UNet編碼器網路(encoder network)架構，發現VGG13的模型架構在testing(公開的資料集)上最終獲得最佳的影像分割結果，並且我們將這些模型的性能以多次實驗數據、學習曲線圖、損失函數曲線相互比較，並從這些實驗中發現CLSTM被證明有助於提升測試階段的準確率。第一個實驗中，我們比較了隨機初始化的權重與預訓練的權重作為初始權重，從Resnet18和VGG16、VGG19三個UNet網路中，預訓練的權重都在測試中有最佳的表現；在實驗II中，我們以相同的實驗環境、訓練驗證的資料與相同的初始權重，測試並了八種神經網路模型作為Encoder的準確度：到CLSTM作者的結論所啟發，實驗III接續實驗II，希望將CLSTM放入不同網路組成的encoder中，討論影像分割結果的進步成效：每一個網路我們分別做了五次訓練、驗證與測試，畫成一束學習曲線圖，得知CLSTM對不同網路convolution計算與學習進程上的進步程度，本實驗亦獲得一個重要的結論，即densely connection 較residual connection更適合做為short skip connection。為了防止over-fitting的問題，在每次的訓練時我們均使用dropout和early-stopping。為了更進一步取得具有公信力的數據，實驗IV將上述模型均繳交競賽的Testing資料，我們再一次印證了個模型的準確率與前述實驗結果相符，此外，也證明了我們提的方法可以登上最新的leader group board，表現出細緻的segmentation。實驗V中，我們更進一步用densely-connected的short skip connection方法改進，最後取得競賽前幾名的成績。

Due to the dynamic movement and variable shape of living cells, the microscopy image analysis remains a challenging task. Unet, FCN based network is normally considered as vital reference for models to perform segmentation of biomedical images. However, there is still no clear way to for Unet to deal with the temporal and spatial characteristics of the cuts of time-series cell images. This study uses the weight pre-trained on a large scale of data as initial weights and CLSTM, which replace most of the pure convolution operation in the original Unet, to process time-series spatial features and obtain better performance than many predecessors' networks. In this study, we compared eight Unet encoder network architectures. We found that the among all the networks of the encoder, the architecture of VGG13 obtain the best image segmentation results on testing data, and we evaluated the performance of these models with multiple experimental data, learning curves, loss function curves. Next, we conducted a series of experiments and found that CLSTM has been proven to help improve the segmentation accuracy: In the first experiment, we compared the randomly initialized weights and the weights of pre-trained model as the initial weights of Unet models. From the three UNet networks: Resnet18 encoder Unet, VGG16 encoder Unet and VGG19 encoder Unet, the pre-trained weights all show better performance in testing phase. In the second experiments, we evaluated the eight encoder networks of Unet models with the intersection over union (IoU) with the same experimental environment, training validation testing data. Inspired by the idea of integrating the CLSTM into encoder structure, we continued experiment 2 and discussed the improvement of using CLSTM to address the spatial temporal information in the eight models in the third experiment. For each network, we have done five trainings, validation and testing. We have drawn a bunch of learning curves and discover that the CLSTM operation can significantly improve the accuracy of segmentation and shorten the learning procedure of VGG encoder Unet. To prevent over-fitting, we use dropout and early stopping during each training. In order to further obtain credible result, in the fourth experiment, we submitted the testing segmentation result of the above models to competition system. From the error score sent by the system, we once again confirmed that the accuracy of the model is consistent with the above experimental results. In addition, it also proves our new method: KUNet, can be on the latest top leader board, showing detailed segmentation. In the fifth experiment, we further improved KUNet by referencing the densely connected method, and finally achieved top place in the competition.