請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/59797完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 連豊力(Feng-Li Lian) | |
| dc.contributor.author | Chia-Wei Hu | en |
| dc.contributor.author | 胡家維 | zh_TW |
| dc.date.accessioned | 2021-06-16T09:38:29Z | - |
| dc.date.available | 2017-02-17 | |
| dc.date.copyright | 2017-02-17 | |
| dc.date.issued | 2017 | |
| dc.date.submitted | 2017-02-09 | |
| dc.identifier.citation | Papers:
[1: Wang et al. 2016] T. Bergen and T. Wittenberg, “Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods,” IEEE Journal of Biomedical and Health Informatics, Vol. 20, No. 1, pp. 304-321, January 2016 [2: Mountney & Yang 2009] P. Mountney and G. -Z. Yang, “Dynamic View Expansion for Minimally Invasive Surgery using Simultaneous Localization And Mapping,” in Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1184-1187, 2009 [3: Grasa et al. 2009] O. G. Grasa, J. Civera and J. M. M. Montiel, “EKF monocular SLAM with relocalization for laparoscopic sequences,” in Proceedings of IEEE International Conference on Robotics and Automation, pp. 4816-4821, 9-13 May 2011 [4: Wu et al. 2007] C. –H. Wu, Y. -N Sun, and C. –C. Chang, “Three-Dimensional Modeling From Endoscopic Video Using Geometric Constraints Via Feature Positioning,” IEEE Transactions on Biomedical Engineering, Vol. 54, No. 7, July 2007 [5: Fan & Meng 2011] Y. Fan, and M. Q. -H. Meng, “3D Reconstruction of the WCE Images by Affine SIFT method,” in Proceedings of World Congress on Intelligent Control and Automation, pp. 943 - 947, 21-25 June 2011 [6: Yip et al. 2012] M. C. Yip, D. G. Lowe, S. E. Salcudean, R. N. Rohling, and C. Y. Nguan, “Tissue Tracking and Registration for Image-Guided Surgery,” IEEE Transactions on Medical Imaging, Vol. 31, No. 11, pp. 2169-2182, November 2012 [7: Haouchine et al. 2015] N. Haouchine, J. Dequidt, M. Berger, and S. Cotin, “Monocular 3D Reconstruction and Augmentation of Elastic Surfaces with Self-Occlusion Handling,” IEEE Transactions on visualization and Computer graphics, Vol. 21, No. 12, pp. 1363-1376, December 2015 [8: Lee et al. 2015] P. Y. Lee, S. –L. Yan, M. –H. Hu, J. Marescaux, H. –S. Wu, K. –C. Liu, A. Kumar, M. –L. Wang, “A Computed Stereoscopic Method for Laparoscopic Surgery by using Feature Tracking,” in Proceedings of International Conference on Consumer Electronics-Taiwan, pp. 9-10, 6-8 June 2015 [9: Visentini-Scarzanella et al. 2012] M. Visentini-Scarzanella, D. Stoyanov, and G. –Z. Yang, “Metric Depth Recovery from Monocular Images Using Shape-from-Shading and Specularities,” in Proceedings of International Conference on Image Processing, pp. 25-28, 30 September - 3 October 2012 [10: Kumar et al. 2015] A. Kumar, Y. –Yu. Wang, K. –C. Liu, W. –C. Hung, S. –W. Huang, W. –N. Lie, and C. –C. Huang, “Surface reconstruction from endoscopic image sequence,” in Proceedings of International Conference on Consumer Electronics-Taiwan, pp.404-405, 6-8 June 2015 [11: Giannarou & Yang 2011] S. Giannarou and G. -Z Yang, “Tissue Deformation Recovery with Gaussian Mixture Model Based Structure from Motion,” in Proceedings of Workshop on Augmented Environments for Computer-Assisted Interventions, Vol. 7264, pp. 47-57, 2011 [12: Malti & Bartoli 2014] A. Malti, and A. Bartoli, “Combining Conformal Deformation and Cook–Torrance Shading for 3-D Reconstruction in Laparoscopy,” IEEE Transactions on Biomedical Engineering, Vol. 61, No. 6, pp.1684-1692, June 2014 [13: Herrera et al. 2013] S. E. M. Herrera, A. Malti, O. Morel, and A. Bartoli, “Shape-From-Polarization in Laparoscopy,” in Proceedings of IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1412-1415, 7-11 April 2013 [14: Stoyanov et al. 2004] D. Stoyanov, A. Darzi, and G. Z. Yang, “Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery,” in Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 41-48, 2004 [15: Stoyanov et al. 2010] D. Stoyanov, M. V. Scarzanella, P. Pratt, and G. –Z. Yang, “Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery,” in Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 275-282, 2010 [16: Haouchine et al. 2014] N. Haouchine, J. Dequidt, I. Peterlik, E. Kerrien, M. Berger, and S. Cotin, “Towards an Accurate Tracking of Liver Tumors for Augmented Reality in Robotic Assisted Surgery,” in Proceedings of IEEE International Conference on Robotics & Automation, pp. 4121 – 4126, 31 May – 7 June 2014 [17: Marques et al. 2015] B. Marques, F. Roy, N. Haouchine, E. Jeanvoine, and S. Cotin, “Framework for Augmented Reality in Minimally Invasive Laparoscopic Surgery,” in Proceedings of IEEE International Conference on e-Health Networking, Applications and Services, pp. 22-27, 14-17 October 2015 [18: Parchami et al. 2014] M. Parchami, J. A. Cadeddu, and G. –L. Mariottini, “Endoscopic Stereo Reconstruction: a Comparative Study,” in Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 26-30 August 2014 [19: Dhiraj et al. 2014] Dhiraj, P. Soni, J. L. Raheja, “Development of 3D endoscope for Minimum Invasive Surgical System,” in Proceedings of International Conference on Signal Propagation and Computer Technology, pp. 168-172, 12-13 July 2014 [20: Lathrop et al. 2010] R. A. Lathrop, D. M. Hackworth, and R., Jr., W., III, “Minimally Invasive Holographic Surface Scanning for Soft-Tissue Image Registration,” IEEE Transactions on Biomedical Engineering, Vol. 57, No. 6, pp. 1497-1506, June 2010 [21: Wang et al. 2010] X. –W. Wang, Q. Zhang, Q. Han, R. –G. Yang, M. Carswell, B. Seales, and E. Sutton, “Endoscopic Video Texture Mapping on Pre-Built 3-D Anatomical Objects Without Camera Tracking,” IEEE Transactions on Medical Imaging, Vol. 29, No. 6, pp. 1213-1223, June 2010 [22: Zenbutsu et al. 2011] S. Zenbutsu, T. Yamaguchi, and T. Igarashi, “Multi Modality Fusion Imaging Technique for Laparoscopic Surgery Guiding,” in Proceedings of International Ultrasonics Symposium, pp. 2273 - 2276, 2011 [23: Park et al. 2012] D. R. Park, J. –K. Cho, and Y. –H. Kim, “A Visual Guidance System for Minimal Invasive Surgery Using 3D Ultrasonic and Stereo Endoscopic Images,” in Proceedings of IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics, pp.872-877, 24-27 June 2012 [24: Giannarou et al. 2012] S. Giannarou, Z. Zhang and G. –Z. Yang, “Deformable Structure From Motion by Fusing Visual and Inertial Measurement Data,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4816-1861, 7-12 October 2012 [25: Wang & Tewfik 2012] D. Wang, and A. H. Tewfik, “Real Time 3D Visualization of Intraoperative Organ Deformations Using Structured Dictionary,” IEEE Transactions on Medical Imaging, Vol. 31, No. 4, pp. 924-937, April 2012 [26: Lin et al. 2013] B. –X. Lin, Y. Sun, and X. –N. Qian, “Dense Surface Reconstruction With Shadows in MIS,” IEEE Transactions on Biomedical Engineering, Vol. 60, No. 9, pp.2411-2420, September 2013 [27: Wu & Qu 2007] T. T. Wu and J. Y. Qu, “Optical Imaging for Medical Diagnosis Based on Active Stereo Vision and Motion Tracking,” OPT Express, 6 August 2007 [28: Maurice et al. 2012] X. Maurice, C. Albitar, C. Doignon, and M. de Mathelin, “A Structured Light-based Laparoscope with Real-time Organs’ Surface Reconstruction for Minimally Invasive Surgery,” in Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp.5769-5772, 28 August - 1 September 2012 [29: Abdalbari et al. 2013] A. Abdalbari, X. Huang, and J. Ren, “Automatic Surface Reconstruction for Endoscopy-MR Image Fusion in Image Guided Procedures,” IEEE Canadian Conference on Electrical and Computer Engineering, 5-8 May 2013 [30: Reiter et al. 2014] A. Reiter, A. Sigaras, D. Fowler, and P. K. Allen, “Surgical Structured Light for 3D Minimally Invasive Surgical Imaging,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 14-18 September 2014 Websites: [31: Bouguet 2015] J. Bouguet (2015). Camera Calibration Toolbox for Matlab. [Online]. Available: https://www.vision.caltech.edu/bouguetj/calib_doc/ [32: Fusiello 2000] A. Fusiello, E. Trucco, and A. Verri, “A compact algorithm for rectification of stereo pairs,” Machine Vision and Applications, Vol. 12, No. 1, pp. 16-22, July 2000 A. Fusiello (2007). Epipolar rectification. [Online]. Available: http://www.diegm.uniud.it/fusiello/demo/rect/ [33: Rusu & Cousins 2011] R. B. Rusu and S. Cousins, “3D is here: Point Cloud Library (PCL),” in Proceedings of IEEE International Conference on Robotics and Automation, pp. 1-4, 9-13 May 2011 R. B. Rusu and S. Cousins (2011). Point Cloud Library (PCL). [Online]. Available: http://pointclouds.org/ [34: Gray & Demirdjian 2011] G. Gray, and D. Demirdjian (2011). StereoPlus. [Online]. Available: http://people.csail.mit.edu/demirdji/download/index.html [35: Felzenszwalb & Huttenlocher 2006] P. Felzenszwalb, and D. Huttenlocher, “Efficient Belief Propagation for Early Vision, ” International Journal of Computer Vision, Vol. 70, No. 1, pp. 41–54, October 2006 P. Felzenszwalb (2006). Belief Propagation for Early Vision. [Online]. Available: https://cs.brown.edu/~pff/bp/index.html Books: [36: Koks 2006] Don Koks, “Exploration in Mathematical Physics[electronic resource]: The Concepts Behind an Elegant Language, ” New York, 2006, pp.147-148 [37: Hartley & Zisserman 2003] Richard Hartley, and Andrew Zisserman, “Multiple View Geometry in Computer Vision, ” 2nd ed., New York, 2003, pp.153-157 [38: Gonzalez & Woods 2008] Rafael C. Gonzalez, and Richard E. Woods, “Digital Image Processing, An Adapted Version” 3rd ed., Editor: Shaou-Gang Miaou, Taipei, 2008, pp.156-157 | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/59797 | - |
| dc.description.abstract | 透過內視鏡影像三維重建，醫師可在進行微創手術時獲得手術區域的深度資訊以及重建後的三維影像平面。在本篇論文中主要針對腹腔微創手術的內視鏡影像進行研究及討論。
腹腔內視鏡影像和其他種類影像¬─例如室外或是室內場景等─之間最大的差異在於人體內部為平滑且色調單純的表面，在重建時不容易提取出足夠數量或強健性高的特徵。另外，腹腔為一動態環境，人體自然運動包括呼吸造成的體腔反覆移動或是內臟本身的運動如腸道蠕動等都會影響到重建的準確性。而腹腔內部自然分泌的體液以及手術過程中切割產生的血液會讓手術區域的反射性提高，在影像中產生飽和的區域。而手術過程中手術器械除了造成遮蔽，夾或切割臟器皆會造成目標區域的變形，而且電燒止血產生的煙霧則會造成影像清晰程度下降。以上這些問題都會影響重建腹腔內視鏡影像模型的準確性以及完整性。 在腹腔鏡影像重建方法部分可分為單純視覺影像重建或是使用額外的器材輔助兩種。單純只使用視覺影像重建又可分為單眼視覺以及雙眼視覺，而使用額外器材輔助有三大類，包括利用雷射、投影等方式增加特徵點、融合影像和其他深度資訊、以及利用手術器械在目標區域造成的陰影。而在本篇論文中我們提出的方法為利用雙眼視覺加上投影色光條紋輔助內視鏡影像的重建，首先校正雙眼相機以獲得相機參數和校正後的影像，然後透過灰階化以及背景相減獲得影像中粗略的條紋區域，再利用閥值以及中值濾波器濾除雜訊以獲得邊緣平滑且較完整的條紋區域。接著擷取每一組雙眼視覺的影像中投影條紋的邊界後，可直接計算其視差，而其餘區域的視差值可通過內插相鄰邊界的視差值獲得。最後，有效視差值的像素其三維座標則可以用三角測量計算。 本論文中實驗的部分，我們採用一人體模型的腹部作為影像的目標區域，並以兩個蛇管內視鏡加上微投影機建置實驗所需的感測器。而在實驗中首先取得數組在不同距離和位置所拍攝的腹部影像，接著分別使用論文中所提出的方法重建其三維模型，並針對參數變化對重建結果產生的影響進行比較與討論，最後比較使用本論文所提出方法或是其他重建演算法所獲得的重建結果。 | zh_TW |
| dc.description.abstract | Applying 3D surface reconstruction to acquire the depth information and the reconstructed model of the operating region is essential for the surgeons to obtain clearer and expanded view during minimally invasive surgery (MIS). In the thesis, the focus in on the 3D surface reconstruction of laparoscopy images, the one about abdomen surgery.
The main challenges about reconstructing laparoscopy images are lacking features, saturation caused by the reflection of body fluid or blood, deformation of the operating surfaces, image blurring caused by the camera motion or naturally movement of human body or organs, smoke coming from burning, and surface occlusion. In the thesis, the focus is on reconstructing the smooth and textureless surface. In order to deal with insufficient features, the proposed method is using the stereo camera with projected light stripes to create additional features on the target surface. With the calibrated and rectified stereo camera, background subtraction and filtering are applied to the image pairs to extract the regions of the light stripes. After obtaining the edges of the light stripes, curve matching and calculating the disparity at the curves is straightforward. As for the disparity in the regions between adjacent curves, interpolation is applied. Finally, since the disparity map is reconstructed and the camera parameters are known, the 3D point cloud can be obtained by applying triangulation. For the experiment in the thesis, one stereo camera is constructed by two endoscopes and fixed with one pico projector. Then several sets of image pairs about the bowel region of the human body model are captured at different distances and positions. The reconstruction results of these image sets are shown and discussed. Besides, the result is compared with those results from other algorithms. | en |
| dc.description.provenance | Made available in DSpace on 2021-06-16T09:38:29Z (GMT). No. of bitstreams: 1 ntu-106-R01921016-1.pdf: 4660013 bytes, checksum: 4eafc1f4f9cb563dcf5d8d1482cdf24c (MD5) Previous issue date: 2017 | en |
| dc.description.tableofcontents | Contents
摘要 …..……………………………………….……………………………….............. i Abstract ………………………………………..……………………………………… iii Contents ………..…………………………….……………….………...…………….....v List of Figures ……………………………………………….…………...…………....vii List of Tables ………………………………..……….………………………………. viii Algorithm …………………………………………………….……………………… viii Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Problem Formulation 3 1.3 Contribution of the Thesis 4 1.4 Organization of the Thesis 5 Chapter 2 Literature Survey 6 2.1 Camera Vision Only 7 2.2 Camera Vision with Other Equipment 11 Chapter 3 Mathematical Preliminary 14 3.1 Camera Models and Coordinates 14 3.2 Median Filter 18 3.3 Triangulation 19 3.4 Rodrigues Rotation Formula 22 Chapter 4 3D Surface Reconstruction 27 4.1 Light Stripe Extraction 30 4.2 Curve Extraction 32 4.3 Pixel Matching at Curves 33 4.4 Pixel Matching in Regions between Curves 34 4.5 Disparity Calculation 37 4.6 3D Coordinate Calculation 38 Chapter 5 Experimental Environment and Results 39 5.1 Experimental Equipment and Environment 40 5.2 Camera Calibration 42 5.3 Captured Image Pairs 46 5.4 Image Rectification 49 5.5 Light Stripe Extraction 52 5.6 Curve Extraction 67 5.7 Disparity Map and 3D Point Cloud Reconstruction 75 5.8 Compare Reconstruction Result 102 Chapter 6 Conclusion and Future Works 105 6.1 Conclusion 105 6.2 Future Works 106 References 107 Appendix……………………………….……………….……………………………..112 List of Figures Fig. 1.1 Endoscopy Images……………………………………………………………2 Fig. 2.1 Hierarchical Structure of MIS 3D Reconstruction Methods………………… 6 Fig. 3.1 Pinhole Model……………………………………………….……………….14 Fig. 3.2 Coordinates…………………………………………………………………..16 Fig. 3.3 Apply Median Filter to Remove the Noise in the Image…….………………18 Fig. 3.4 Triangulation: 3D Coordinate Calculation Using Stereo Camera ………......19 Fig. 3.5 Vector Rotates around the Axis of Rotation ..….……………………………22 Fig. 4.1 System Overview: the process of surface reconstruction……………………28 Fig. 4.2 Pixel Matching at Curves…………...……………………………………….33 Fig. 4.3 Pixel Matching in Region between Curves………………………………….34 Fig. 5.1 Stereo Camera…………………………………………………………..…....40 Fig. 5.2 Stereo Camera with Pico Projector………………..…………………………40 Fig. 5.3 Target in the Experiment: Model of Human Body…………………………..41 Fig. 5.4 Left Chess Board Images……………………………………...……………..42 Fig. 5.5 Right Chess Board Images…………………………..………………………43 Fig. 5.6 Relative Relations of the Eight Positions……………………………………46 Fig. 5.7 Distances and Captured Background Image Pairs at Eight Positions……….48 Fig. 5.8 Original Background Image Pairs and the Rectified Ones….…………….…50 Fig. 5.9 Left Mask Images at Position One, with Window Size Equaling 15….…….53 Fig. 5.10 Right Mask Images at Position One, with Window Size Equaling 15………55 Fig. 5.11 Left Mask Images at Position Five, with Window Size Equaling 15……......57 Fig. 5.12 Right Mask Images at Position Five, with Window Size Equaling 15………59 Fig. 5.13 Left Mask Images at Position Eight, with Threshold Equaling 0.4……….…61 Fig. 5.14 Right Mask Images at Position Eight, with Threshold Equaling 0.4………...63 Fig. 5.15 Left Average Background Noises at the Eight Positions…………………….65 Fig. 5.16 Right Average Background Noises at the Eight Positions………………..….66 Fig. 5.17 Left Continuity of Curves at the Eight Positions……………………….……69 Fig. 5.18 Right Continuity of Curves at the Eight Positions………………..…………70 Fig. 5.19 Left Completeness of Curves at the Eight Positions…………………...……72 Fig. 5.20 Right Completeness of Curves at the Eight Positions……………………….73 Fig. 5.21 Disparity Maps and 3D Point Cloud at Position One………………….…….79 Fig. 5.22 Disparity Maps and 3D Point Cloud at Position Two………………..………82 Fig. 5.23 Disparity Maps and 3D Point Cloud at Position Three..………………….…85 Fig. 5.24 Disparity Maps and 3D Point Cloud at Position Four……………….………88 Fig. 5.25 Disparity Maps and 3D Point Cloud at Position Five……………………….91 Fig. 5.26 Disparity Maps and 3D Point Cloud at Position Six…………………..…….94 Fig. 5.27 Disparity Maps and 3D Point Cloud at Position Seven……………...………97 Fig. 5.28 Disparity Maps and 3D Point Cloud at Position Eight……………………100 Fig. 5.29 Background Image and Disparity Maps at Position One………….……….104 Fig. 5.30 The Completeness Value of Three Bounded Disparity Maps……………....104 List of Tables Table 5.1 Intrinsic Parameters of the Cameras……………..……………..……….…44 Table 5.2 Extrinsic Parameters of the Stereo Camera………………………………...44 Algorithm Algorithm 1 Curve Extraction......................................................................................32 | |
| dc.language.iso | en | |
| dc.subject | 光條紋投影 | zh_TW |
| dc.subject | 內視鏡影像 | zh_TW |
| dc.subject | 三維重建 | zh_TW |
| dc.subject | 微創手術 | zh_TW |
| dc.subject | 雙眼相機 | zh_TW |
| dc.subject | minimally invasive surgery | en |
| dc.subject | light stripe projection | en |
| dc.subject | stereo camera | en |
| dc.subject | endoscopy | en |
| dc.subject | 3D surface reconstruction | en |
| dc.title | 以光條紋投影輔助雙眼內視鏡影像的三維重建 | zh_TW |
| dc.title | Light-Stripe-Assisted 3D Reconstruction of Stereo Endoscopy Images | en |
| dc.type | Thesis | |
| dc.date.schoolyear | 105-1 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 簡忠漢,李後燦,黃正民 | |
| dc.subject.keyword | 內視鏡影像,三維重建,微創手術,雙眼相機,光條紋投影, | zh_TW |
| dc.subject.keyword | endoscopy,3D surface reconstruction,minimally invasive surgery,stereo camera,light stripe projection, | en |
| dc.relation.page | 112 | |
| dc.identifier.doi | 10.6342/NTU201700444 | |
| dc.rights.note | 有償授權 | |
| dc.date.accepted | 2017-02-10 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 電機工程學研究所 | zh_TW |
| 顯示於系所單位： | 電機工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-106-1.pdf 未授權公開取用 | 4.55 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。