請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/86521完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 徐宏民(Winston Hsu) | |
| dc.contributor.author | Ru-Fen Jheng | en |
| dc.contributor.author | 鄭如芬 | zh_TW |
| dc.date.accessioned | 2023-03-20T00:00:46Z | - |
| dc.date.copyright | 2022-08-24 | |
| dc.date.issued | 2022 | |
| dc.date.submitted | 2022-08-16 | |
| dc.identifier.citation | [1] Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph., 2009. [2] Dominique Brunet, Edward R Vrscay, and Zhou Wang. On the mathematical properties of the structural similarity index. IEEE Transactions on Image Processing, 2011. [3] John Canny. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence, 1986. [4] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017. [5] Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In European conference on computer vision. Springer, 2016. [6] Angela Dai, Christian Diller, and Matthias Nießner. Sg-nn: Sparse generative neural networks for self-supervised scene completion of rgb-d scans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. [7] Angela Dai, Matthias Nießner, Michael Zollhöfer, Shahram Izadi, and Christian Theobalt. Bundlefusion: Real-time globally consistent 3d reconstruction using on-the-fly surface reintegration. ACM Transactions on Graphics (ToG), 2017. [8] Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jürgen Sturm, and Matthias Nießner. Scancomplete: Large-scale scene completion and semantic segmentation for 3d scans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018. [9] Angela Dai, Charles Ruizhongtai Qi, and Matthias Nießner. Shape completion using 3d-encoder-predictor cnns and shape synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017. [10] Angela Dai, Yawar Siddiqui, Justus Thies, Julien Valentin, and Matthias Nießner. Spsg: Self-supervised photometric scene generation from rgb-d scans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. [11] Soheil Darabi, Eli Shechtman, Connelly Barnes, Dan B Goldman, and Pradeep Sen. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on graphics (TOG), 2012. [12] Emily L Denton, Soumith Chintala, Rob Fergus, et al. Deep generative image models using a laplacian pyramid of adversarial networks. Advances in neural information processing systems, 2015. [13] Alexei A Efros and William T Freeman. Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, 2001. [14] Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. [15] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 2014. [16] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. A papier-mâché approach to learning 3d surface generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018. [17] Xiefan Guo, Hongyu Yang, and Di Huang. Image inpainting via conditional texture and structure dual generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. [18] Xiaoguang Han, Zhen Li, Haibin Huang, Evangelos Kalogerakis, and Yizhou Yu. High-resolution shape completion using deep neural networks for global structure and local geometry inference. In Proceedings of the IEEE international conferenceon computer vision, 2017. [19] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 2017. [20] Jingwei Huang, Angela Dai, Leonidas J Guibas, and Matthias Nießner. 3dlite: towards commodity 3d scanning for content creation. ACM Trans. Graph., 2017. [21] Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kundu, Chiyu Jiang, Leonidas J Guibas, Matthias Nießner, Thomas Funkhouser, et al. Adversarial texture optimization from rgb-d scans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. [22] Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, and Johannes Kopf. Image completion using planar structure guidance. ACM Transactions on graphics (TOG), 2014. [23] Zitian Huang, Yikuan Yu, Jiawen Xu, Feng Ni, and Xinyi Le. Pf-net: Point fractal network for 3d point cloud completion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020. [24] Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, et al. Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In Proceedings of the 24th annual ACM symposium on User interface software and technology, 2011. [25] Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. Poisson surface reconstruction. In Proceedings of the fourth Eurographics symposium on Geometry processing, 2006. [26] Michael Kazhdan and Hugues Hoppe. Screened poisson surface reconstruction. ACM Transactions on Graphics (ToG), 2013. [27] Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European conference on computer vision (ECCV), pages 85–100, 2018. [28] Hongyu Liu, Bin Jiang, Yibing Song, Wei Huang, and Chao Yang. Rethinking image inpainting via a mutual encoder-decoder with feature equalizations. In European Conference on Computer Vision. Springer, 2020. [29] Robert Maier, Kihwan Kim, Daniel Cremers, Jan Kautz, and Matthias Nießner. Intrinsic3d: High-quality 3d reconstruction by joint appearance and geometry optimization with spatially-varying lighting. In Proceedings of the IEEE international conference on computer vision, pages 3114–3122, 2017. [30] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019. [31] Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Z Qureshi, and Mehran Ebrahimi. Edgeconnect: Generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212, 2019. [32] Richard A Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J Davison, Pushmeet Kohi, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In 2011 10th IEEE international symposium on mixed and augmented reality. Ieee, 2011. [33] Michael Oechsle, Lars Mescheder, Michael Niemeyer, Thilo Strauss, and Andreas Geiger. Texture fields: Learning texture representations in function space. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. [34] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019. [35] Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. [36] Muhammad Sarmad, Hyunjoo Jenny Lee, and Young Min Kim. Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. [37] Meng-Li Shih, Shih-Yang Su, Johannes Kopf, and Jia-Bin Huang. 3d photography using context-aware layered depth inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. [38] Shuran Song, Fisher Yu, Andy Zeng, Angel X Chang, Manolis Savva, and Thomas Funkhouser. Semantic scene completion from a single depth image. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017. [39] Yongbin Sun, Ziwei Liu, Yue Wang, and Sanjay E Sarma. Im2avatar: Colorful 3d reconstruction from a single image. arXiv preprint arXiv:1804.06375, 2018. [40] Jiajun Wu, Chengkai Zhang, Xiuming Zhang, Zhoutong Zhang, William T Freeman, and Joshua B Tenenbaum. Learning shape priors for single-view 3d completion and reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV), 2018. [41] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912–1920, 2015. [42] Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann. Disn: Deep implicit surface network for high-quality single-view 3d reconstruction. Advances in Neural Information Processing Systems, 32, 2019. [43] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Interpretable unsupervised learning on 3d point clouds. arXiv preprint arXiv:1712.07262, 2017. [44] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF international conference on computer vision, 2019. [45] Xumin Yu, Yongming Rao, Ziyi Wang, Zuyan Liu, Jiwen Lu, and Jie Zhou. Pointr: Diverse point cloud completion with geometry-aware transformers. In Proceedings of the IEEE/CVF international conference on computer vision, 2021. [46] Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. Pcn: Point completion network. In 2018 International Conference on 3D Vision (3DV). IEEE, 2018. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/86521 | - |
| dc.description.abstract | 現今，由於擴增實境和虛擬實境技術的發展，使用者在三維立體場景中進行編輯的需求迅速增加。然而，現有的三維立體場景補全任務（以及資料集）並無法滿足使用者編輯的需求，因為其場景中的缺失區域是由傳感器偵測限制或物品遮擋產生的。因此，我們提出了任意形狀三維立體場景修復的新任務。與之前的三維立體場景補全任務的資料集中場景保留了缺失區域周圍大部分的主要結構和細節的形狀提示不同，我們所提出的三維立體場景修復資料集（FF-Matterport）包含了由我們提出的任意形狀三維立體遮罩生成演算法所產生大面積而多樣的缺失區域；此演算法模仿了人類在三維立體空間中繪製遮罩的軌跡。此外，先前的三維立體場景補全方法只需對缺失區域周圍的結構和顏色進行插值即可達到不錯的效果，但這些方法無法很好地完成三維立體場景修復這項具有挑戰性但實用的任務，因此我們針對此任務設計了雙流對抗式生成網路。首先，我們的雙流生成式網路結合了三維立體場景中結構與顏色的資訊，以生成具有明確語義邊界的場景並解決了先前方法中插值的問題。為了進一步加強場景中的細節，我們提出了輕量級的雙流鑑別式網路將生成場景的結構與顏色邊緣規範化，使其更加逼真與清晰。我們用提出的FF-Matterport資料集進行了實驗。定性和定量的結果都驗證了我們提出的方法優於現有三維立體場景補全方法且所有提出的架構皆有其效果。 | zh_TW |
| dc.description.abstract | Nowadays, the need for user editing in a 3D scene has rapidly increased due to the development of AR and VR technology. However, the existing 3D scene completion task (and datasets) cannot suit the need because the missing regions in scenes are generated by the sensor limitation or object occlusion. Thus, we present a novel task named free-form 3D scene inpainting. Unlike scenes in previous 3D completion datasets preserving most of the main structures and hints of detailed shapes around missing regions, the proposed inpainting dataset, FF-Matterport, contains large and diverse missing regions formed by our free-form 3D mask generation algorithm that can mimic human drawing trajectories in 3D space. Moreover, prior 3D completion methods cannot perform well on this challenging yet practical task, simply interpolating nearby geometry and color context. Thus, a tailored dual-stream GAN method is proposed. First, our dual-stream generator, fusing both geometry and color information, produces distinct semantic boundaries and solves the interpolation issue. To further enhance the details, our lightweight dual-stream discriminator regularizes the geometry and color edges of the predicted scenes to be realistic and sharp. We conducted experiments with the proposed FF-Matterport dataset. Qualitative and quantitative results validate the superiority of our approach over existing scene completion methods and the efficacy of all proposed components. | en |
| dc.description.provenance | Made available in DSpace on 2023-03-20T00:00:46Z (GMT). No. of bitstreams: 1 U0001-0408202216215500.pdf: 8251930 bytes, checksum: 7a7fda48da9471411818779eaa116fb6 (MD5) Previous issue date: 2022 | en |
| dc.description.tableofcontents | Acknowledgements i 摘要 ii Abstract iii Contents v List of Figures vii List of Tables ix Chapter 1 Introduction 1 Chapter 2 Related Work 5 2.1 3D Completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 2D Image Inpainting . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Chapter 3 Method 8 3.1 Freeform 3D Dataset Generation . . . . . . . . . . . . . . . . . . . . 8 3.2 Dualstream Generator . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Dualstream Discriminator . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 4 Experiments 14 4.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.3 Ablation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 References 19 Appendix A — Appendix for Freeform 3D Mask Generation Algorithm 26 Appendix B — Appendix for Additional Results 28 B.1 Edge Discriminator Qualitative Result . . . . . . . . . . . . . . . . . 28 B.2 Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 | |
| dc.language.iso | en | |
| dc.subject | 任意形狀修復 | zh_TW |
| dc.subject | 深度學習 | zh_TW |
| dc.subject | 對抗式生成網路 | zh_TW |
| dc.subject | 三維立體場景 | zh_TW |
| dc.subject | 三維立體場景修復 | zh_TW |
| dc.subject | 深度學習 | zh_TW |
| dc.subject | 對抗式生成網路 | zh_TW |
| dc.subject | 任意形狀修復 | zh_TW |
| dc.subject | 三維立體場景 | zh_TW |
| dc.subject | 三維立體場景修復 | zh_TW |
| dc.subject | 3D Scene Inpainting | en |
| dc.subject | Deep Learning | en |
| dc.subject | Generative Adversarial Network | en |
| dc.subject | Free-form 3D Scene Inpainting | en |
| dc.subject | 3D Scene | en |
| dc.subject | Deep Learning | en |
| dc.subject | 3D Scene Inpainting | en |
| dc.subject | 3D Scene | en |
| dc.subject | Free-form 3D Scene Inpainting | en |
| dc.subject | Generative Adversarial Network | en |
| dc.title | 以雙流對抗式生成網路之任意形狀三維立體場景修復 | zh_TW |
| dc.title | Free-form 3D Scene Inpainting with Dual-stream GAN | en |
| dc.type | Thesis | |
| dc.date.schoolyear | 110-2 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 陳文進(Wen-Chin Chen),陳奕廷(Yi-Ting Chen),葉梅珍(Mei-Chen Yeh) | |
| dc.subject.keyword | 深度學習,對抗式生成網路,任意形狀修復,三維立體場景,三維立體場景修復, | zh_TW |
| dc.subject.keyword | Deep Learning,Generative Adversarial Network,Free-form 3D Scene Inpainting,3D Scene,3D Scene Inpainting, | en |
| dc.relation.page | 30 | |
| dc.identifier.doi | 10.6342/NTU202202063 | |
| dc.rights.note | 同意授權(全球公開) | |
| dc.date.accepted | 2022-08-16 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
| dc.date.embargo-lift | 2022-08-24 | - |
| 顯示於系所單位： | 資訊工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| U0001-0408202216215500.pdf | 8.06 MB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。