基於Tensor Train張量分解之神經網路壓縮及其在影像分類與分割上的應用

Abstract

本論文的其中一個目標是探討Tensor Train模型壓縮法在醫學影像預測模型上的有效性。此外，我們還旨在澄清這種壓縮方法對壓縮模型之泛化能力的影響。為了評估Tensor Train模型壓縮的有效性，我們在兩個不同的數據集上進行了實驗，分別是chest X-ray dataset和low-grad glioma dataset (LGG)。 具體來說，我們對使用chest X-ray dataset訓練的模型中的dense layer進行了壓縮，並對使用LGG dataset訓練的U-Net模型中的convolution layer進行壓縮。值得注意的是，在chest X-ray dataset的實驗上，與模型的壓縮率相比，壓縮後的測試準確率損失幾乎可以忽略不計。據我們所知，目前尚無其他文獻將此模型壓縮技術應用於U-Net模型。至於泛化能力這方面的討論，我們采用Arora的Compression Framework來建立壓縮模型的泛化誤差與原始模型的訓練損失之間的定量關係。為了驗證這個泛化誤差上界的準確性，我們在MNIST數據集上進行了額外的實驗。這個實驗的結果顯示，儘管此泛化誤差上界和真實的測試誤差之間存在一段差距，但在fine-tune這個階段它們的整體行為是保持一致的。

This thesis primarily focuses on investigating the effectiveness of the Tensor Train model compression method on prediction models of medical image data sets. Additionally, we also aim to clarify how this compression technique impacts the generalization ability of the compressed models. To assess the effectiveness of the Tensor Train compression method, experiments were conducted on two distinct datasets: chest X-ray and low-grade glioma (LGG). Specifically, we applied compression to the dense layer in the models using chest X-ray dataset and the convolution layer in the U-Net model trained on LGG dataset. It is worth noting that in the experiment conducted on the chest X-ray dataset, the loss in testing accuracies after compression was minimal compared to the achieved compression ratio. To the best of our knowledge, there is no existing studies have explored the application of this model compression technique specifically on U-Net models. As for the issue of generalization ability, we adopt Arora's compression framework [Arora et al., 2018] to build a quantitative relationship between the generalization error of the compressed model and training loss of the original model. To validate the performance of this bound, we conducted an additional experiment on the MNIST dataset. Our findings reveal that although a discrepancy exists between the generalization bound and the true testing error, their overall behavior coincides during the fine-tuning phase.