基於格點結構光圖案實現卷積神經網路端到端三維重建

Abstract

三維內視鏡比傳統內視鏡提供更強的深度感知資訊，因此受到廣 泛討論。雖然在深度感知上已經有明顯改善，目前的三維內視鏡還未 能提供精確的深度值，且長時間使用下容易因視覺輻輳調節衝突讓醫 生感到頭暈及噁心等不適。為了改善上述情況，我們結合結構光及深 度學習技術，提出一個設計給內視鏡的三維量測架構。本研究的三維 量測架構包含自行開發的神經網路SLResnet，以及座標校準演算法。 為了減少不可控因素，我們基於投影機-相機系統上開發及測試演算 法，並為內視鏡的可能使用情境設計實驗。我們的方法透過階高塊的 驗證，最大相對深度誤差可在0.396毫米以內。此方法亦能在深度變 化70度以內進行穩定的預測。在反光存在，且假設待測物體的表面是 連續的情境下，我們的方法能還原至多連續6格結構光圖案被反光遮蔽 的區域。我們亦模擬重建咽喉內的上顎，結果顯示99.7百分比的三維 點符合相對深度差1毫米的規範。基於上述實驗證明了將我們的結構光 技術應用於內視鏡的可行性。

The 3D endoscope has been widely discussed due to its ability to pro- vide more concrete depth information than that of traditional 2D endoscope. While 3D endoscope has significantly improved users’ depth perception, it still does not give accurate depth value. And by focusing on the 3D endo- scopic system for a long time, the user may suffer from symptoms such as dizziness and nausea resulting from the vergence-accommodation conflict. To improve the mentioned problems, this work integrated structured light technique and deep learning and proposed a 3D reconstruction framework for the endoscopic system. This work includes a self-developed SLResnet and a coordinate refinement algorithm. To minimize uncontrollable factors, this work developed our algorithms on the projector-camera system and de- signed several experiments to simulate the potential scenarios in endoscopy. With a 2 mm gauge block for verification, the maximum relative depth error of our method achieved 0.396 mm. This method can reconstruct at steepness up to 70 degrees stably. Under reflection and assuming that the target ob- ject’s surface is continuous, our approach can at most recover the area of 6 connected structured light grids. This work also simulated human upper jaw reconstruction, and over 99.7 percent of the 3D points passed the required 1 mm relative depth error. Based on our experiments, it’s feasible to incorpo- rate the advantage of structured light technique into the endoscope system for advanced depth perception.