Please use this identifier to cite or link to this item:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/20443
Title: | 單鏡頭深度預測與手術煙霧去除於內視鏡手術 Monocular Depth Estimation and Surgical Smoke Removal for Endoscopic surgery |
Authors: | Ching-Yi Tsai 蔡慶溢 |
Advisor: | 施吉昇(JI-SHENG SHI) |
Keyword: | 內視鏡影像,單鏡頭深度預測,煙霧去除, endoscopic images,monocular depth estimation,smoke removal, |
Publication Year : | 2020 |
Degree: | 碩士 |
Abstract: | 深度預測對於手術輔助系統非常重要,它提供了三度空間的資訊來輔助其他用於系統中的演算法。機器學習的方法對於單眼深度預測有良好的表現,但是它們大多數需要深度的標記(ground truth)來監督學習,使用機器學習方法對內視鏡影響進行深度預測有兩大挑戰。一為不易取得正確深度標記,另一為手術過程中的煙霧干擾。在這篇論文中,我們提出一個利用立體視覺結構特性來自我監督的類神經網路模型。目前已知的去煙方式主要分為規則式(prior-based)與學習式(learning-based),這兩種方法都把手術煙霧去除視為一種霧霾去除問題,而沒有考慮到手術煙霧與霧霾的表現十分不同。為了克服這個問題,我們提出一個以CycleGAN為基底的類神經網路並設計一個子模型: Maximum Random Crop (MRC)單元,它的目的是預測手術煙霧的分佈。為了評估我們提出的去煙模型,我們使用三種數值評估方式,深度的平均絕對誤差、邊緣發現指標和特徵點發現指標。實驗結果顯示我們提出的去煙方式不僅能去除圖像中的煙霧還可以保留圖像中的特徵與圖像的顏色,讓去煙後的影像不會導致錯誤的深度預測。使利用我們的循環生成對抗去煙模型處理後的深度預測不會因為去煙後而結果產生錯誤。 Depth information is essential to intelligent surgical auxiliary systems. It offers 3D information to support those algorithms used in systems. Learningbased methods have excellent performance on depth estimation in a single image. However, using learning-based methods on depth estimation has two significant challenges. One is that most of them need the ground truth of depth for supervised training, which is challenging to acquire for endoscopic surgery; another is the smoke produced by surgical tools impacts the depth estimation results. In this work, we achieve depth estimation by designing a self-supervised network that exploits the structure relationships between pairs of stereo images. The smoke produced by surgical tools reduces the visibility and raises errors in the computer vision algorithms used in intelligent surgical auxiliary systems. Existing methods toward this problem mainly adopt either priorbased methods or learning-based methods. Both refer the surgical smoke removal problem to a fog/haze removal problem, which does not consider the significantly different properties between fog/haze and surgical smoke. To overcome this difference, we propose an end-to-end network based on CycleGAN and introduce a submodule called the Maximum Random Crop (MRC) unit, aiming to estimate the distribution of the surgical smoke. To evaluate the proposed method’s performance, we use four evaluation metrics: SNR, MAE of depth, boundary awareness, and feature point awareness. The experimental results show that the proposed method can remove the surgical smoke and preserve the original smoked images’ features and intensity. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/20443 |
DOI: | 10.6342/NTU202004195 |
Fulltext Rights: | 未授權 |
Appears in Collections: | 資訊工程學系 |
Files in This Item:
File | Size | Format | |
---|---|---|---|
U0001-0109202002134700.pdf Restricted Access | 8.11 MB | Adobe PDF |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.