請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/71660完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 傅立成(Li-Chen Fu) | |
| dc.contributor.author | Xiao-Yue Xu | en |
| dc.contributor.author | 徐瀟越 | zh_TW |
| dc.date.accessioned | 2021-06-17T06:05:50Z | - |
| dc.date.available | 2024-06-18 | |
| dc.date.copyright | 2021-02-26 | |
| dc.date.issued | 2021 | |
| dc.date.submitted | 2021-02-18 | |
| dc.identifier.citation | [1] L. F. D’Haro, R. E. Banchs, and H. Li, 9th International Workshop on Spoken Dialogue System Technology, vol. 579. 2019. [2] K. P. Hawkins, N. Vo, S. Bansal, and A. F. Bobick, “Probabilistic human action prediction and wait-sensitive planning for responsive human-robot collaboration,” in IEEE-RAS International Conference on Humanoid Robots(Humanoids), 2013. [3] R. F. Salas-Moreno, R. A. Newcombe, H. Strasdat, P. H. J. Kelly, and A. J. Davison, “SLAM++: Simultaneous Localisation and Mapping at the Level of Objects,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013. [4] G. Grisetti, C. Stachniss, and W. Burgard, “Improved Techniques for Grid Mapping with Rao-Blackwellized Particle Filters,” IEEE Trans. Robot., vol. 23, no. 1, pp. 34–46, 2007. [5] S. Thrun, M. Montemerlo, D. Koller, B. Wegbreit, J. Nieto, and E. Nebot, “FastSLAM: An Efficient Solution to the Simultaneous Localization And Mapping Problem with Unknown Data Association,” J. Mach. Learn. Res., pp. 1–48, 2004. [6] S. Kohlbrecher, O. Von Stryk, J. Meyer, and U. Klingauf, “A Flexible and Scalable SLAM System with Full 3D Motion Estimation,” IEEE Int. Symp. Safety, Secur. Rescue Robot., no. November, pp. 155–160, 2011. [7] J. Zhang and S. Singh, “Low-drift and real-time lidar odometry and mapping,” Auton. Robots, vol. 41, no. 2, pp. 401–416, 2017. [8] T. Shan and B. Englot, “LeGO-LOAM: Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain,” IEEE Int. Conf. Intell. Robot. Syst., pp. 4758–4765, 2018. [9] Google, “Cartographer,” https://github.com/cartographer-project/cartographer. [Online]. Available: https://github.com/cartographer-project/cartographer. [10] J. McCormac, A. Handa, A. Davison, and S. Leutenegger, “SemanticFusion: Dense 3D semantic mapping with convolutional neural networks,” IEEE Int. Conf. Robot. Autom., pp. 4628–4635, 2017. [11] F. D. Ė, A. Babinec, M. Kajan, P. Be, and M. Florek, “Path planning with modified A star algorithm for a mobile robot,” vol. 96, pp. 59–69, 2014. [12] D. B. JOHNSON, “A Note on Dijkstra ’ s Shortest Path Algorithm,” J. Assoc. Comput. Mach., vol. 20, no. 3, pp. 385–388, 1973. [13] I. M. Engineering, “OBSTACLE AVOIDANCE FOR MOBILE ROBOTS USING ARTIFICIAL Min g y d Park *, Jae hyun Jeon * and Min cheol Lee **,” pp. 1530–1535, 2001. [14] P. Vadakkepat, K. C. Tan, and W. Ming-Liang, “Evolutionary Artificial Potential Fields and Their Application in Real Time Robot Path Planning,” in Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512), 2000, vol. 1, pp. 256–263. [15] P. Trautman, J. Ma, R. M. Murray, and A. Krause, “Robot navigation in dense human crowds: The case for cooperation,” in Proceedings - IEEE International Conference on Robotics and Automation, 2013, pp. 2153–2160. [16] C. P. Lam, C. T. Chou, K. H. Chiang, and L. C. Fu, “Human-Centered Robot Navigation #x2014;Towards a Harmoniously Human #x2013;Robot Coexisting Environment,” IEEE Trans. Robot., vol. 27, no. 1, pp. 99–112, 2011. [17] D. Helbing and P. Molnár, “Social force model for pedestrian dynamics,” Phys. Rev. E, vol. 51, no. 5, pp. 4282–4286, 1995. [18] G. Ferrer, A. Garrell, and A. Sanfeliu, “Robot companion: A social-force based approach with human awareness-navigation in crowded environments,” in IEEE International Conference on Intelligent Robots and Systems, 2013, pp. 1688–1694. [19] M. Shiomi, F. Zanlungo, K. Hayashi, and T. Kanda, “Towards a Socially Acceptable Collision Avoidance for a Mobile Robot Navigating Among Pedestrians Using a Pedestrian Model,” Int. J. Soc. Robot., vol. 6, no. 3, pp. 443–455, 2014. [20] F. Zanlungo, T. Ikeda, and T. Kanda, “Social force model with explicit collision prediction,” Epl, vol. 93, no. 6, 2011. [21] H. Yoshida, H. Fujimoto, D. Kawano, Y. Goto, M. Tsuchimoto, and K. Sato, “Range extension autonomous driving for electric vehicles based on optimal velocity trajectory and driving braking force distribution considering road gradient information,” IECON 2015 - 41st Annu. Conf. IEEE Ind. Electron. Soc., no. Figure 1, pp. 4754–4759, 2015. [22] A. Garulli, A. Giannitrapani, A. Rossi, and A. Vicino, “Mobile robot SLAM for line-based environment representation,” Proc. 44th IEEE Conf. Decis. Control. Eur. Control Conf. CDC-ECC ’05, vol. 2005, pp. 2041–2046, 2005. [23] D. J. Spero and R. A. Jarvis, “A New Solution to the Simultaneous Localization and Map Building Problem,” Robot. …, vol. 17, no. 3, pp. 229–241, 2005. [24] R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos, “ORB-SLAM: A Versatile and Accurate Monocular SLAM System,” IEEE Trans. Robot., vol. 31, no. 5, pp. 1147–1163, 2015. [25] G. Grisetti, C. Stachniss, and W. Burgard, “Improving grid-based SLAM with Rao-Blackwellized particle filters by adaptive proposals and selective resampling,” Proc. - IEEE Int. Conf. Robot. Autom., vol. 2005, pp. 2432–2437, 2005. [26] W. W. Michael Montemerlo, Sebastian Thrun, “Conditional particle filters for simultaneous mobile robot localization and people-tracking,” in IEEE International Conference on Robotics Automation, 2002, no. May, pp. 695–701. [27] A. Leigh, J. Pineau, N. Olmedo, and H. Zhang, “Person tracking and following with 2D laser scanners,” Proc. - IEEE Int. Conf. Robot. Autom., vol. 2015-June, no. June, pp. 726–733, 2015. [28] L. E. Navarro-Serment, C. Mertz, and M. Hebert, “Pedestrian detection and tracking using three-dimensional LADAR data,” Int. J. Rob. Res., vol. 29, no. 12, pp. 1516–1528, 2010. [29] K. O. Arras et al., Range-based people detection and tracking for socially enabled service robots, vol. 76, no. STAR. 2012. [30] M. A. H. S Blackman, R Popoli - Norwood, Design and analysis of modern tracking systems(Book). 1999. [31] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017. [32] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, 2017. [33] Z. Chai, Y. Sun, and Z. Xiong, “A novel method for LiDAR camera calibration by plane fitting,” IEEE/ASME Int. Conf. Adv. Intell. Mechatronics, AIM, vol. 2018-July, pp. 286–291, 2018. [34] “Autoware,” https://www.autoware.ai/. . [35] C. Rösmann, W. Feiten, T. Wösch, F. Hoffmann, and T. Bertram, “Trajectory Modification Considering Dynamic Constraints of Autonomous Robots,” in 7th German Conference on Robotics, 2012, pp. 74–79. [36] T. Y. Lin et al., “Microsoft COCO: Common objects in context,” in European conference on computer vision., 2014, vol. 8693 LNCS, no. PART 5, pp. 740–755. [37] J. Fan, G. Zeng, M. Body, and M. S. Hacid, “Seeded region growing: An extensive and comparative study,” Pattern Recognit. Lett., vol. 26, no. 8, pp. 1139–1156, 2005. [38] A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, and W. Burgard, “OctoMap: An efficient probabilistic 3D mapping framework based on octrees,” Auton. Robots, vol. 34, no. 3, pp. 189–206, 2013. [39] and J. M. Kiyosumi Kidono, Takeo Miyasaka, Akihiro Watanabe, Takashi Naito and Abstract—Pedestrian, “Pedestrian recognition using High-definition LIDAR,” 2011 IEEE Intell. Veh. Symp., no. Iv, pp. 405–410, 2011. [40] R. E. Schapire and Y. Singer, “Improved boosting algorithms using confidence-rated predictions,” Mach. Learn., vol. 37, no. 3, pp. 297–336, 1999. [41] M. Häselich, B. Jöbgen, N. Wojke, J. Hedrich, and D. Paulus, “Confidence-Based Pedestrian Tracking in Unstructured Environments Using 3D Laser Distance Measurements,” in IEEE International Conference on Intelligent Robots and Systems, 2014, no. Iros, pp. 4118–4123. [42] B. Kulis and M. I. Jordan, “Revisiting k-means: New algorithms via Bayesian nonparametrics,” arXiv, vol. 1, 2012. [43] R. Z., “A study of a target tracking method using global nearest neighbor algorithm,” Vojnoteh. Glas., 2006. [44] C. Premebida, O. Ludwig, and U. Nunes, “Exploiting LIDAR-based features on pedestrian detection in urban scenarios,” IEEE Conf. Intell. Transp. Syst. Proceedings, ITSC, pp. 18–23, 2009. [45] S. Quinlan and O. Khatib, “Elastic Bands: Connecting Path Planning and Control,” in Proc. 1993 IEEE Int. Conf. on Robotics and Automation (ICRA), 1993, pp. 802–807. [46] D. Mehta, G. Ferrer, and E. Olson, “Autonomous Navigation in Dynamic Social Environments using Multi-Policy Decision Making,” in IEEE International Conference on Intelligent Robots and Systems, 2016, vol. 2016-Novem, pp. 1190–1197. [47] C. Pantofaru, “people,” https://wiki.ros.org/people. . [48] Caroline Pantofaru, “leg detector,” https://github.com/wg-perception/people. . [49] A. Bera, T. Randhavane, R. Prinja, and D. Manocha, “SocioSense: Robot navigation amongst pedestrians with social and psychological constraints,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), no. c. pp. 7018–7025, 2017. [50] M. Pfeiffer, G. Paolo, H. Sommer, J. Nieto, R. Siegwart, and C. Cadena, “A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments,” IEEE Int. Conf. Robot. Autom., pp. 5921–5928, 2018. [51] B. Kim and J. Pineau, “Socially Adaptive Path Planning in Human Environments Using Inverse Reinforcement Learning,” Int. J. Soc. Robot., vol. 8, no. 1, pp. 51–66, 2016. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/71660 | - |
| dc.description.abstract | 本文針對全向性移動機器人平台，提出了一種在室內場景中，基於光達（LiDAR）和RGB相機融合的語義地圖系統。目的有二，基於語義地圖，機器人可以感知環境中物件位置，可執行基於物件語義的導航任務；其次，基於LiDAR良好的偵測能力，可以結合行人躲避算法和靜態障礙物躲避算法，實現更人性化的導航規劃。為了實現這兩個目標，機器人需要配置多線LiDAR和RGB相機，並對LiDAR和相機進行校準配准。之後，通過RGB相機進行物件偵測，將偵測結果附加到對應點雲上，從而得到具有語義資訊的3D點雲地圖。富含語義資訊的語義地圖建立完成後，將被壓縮成八叉樹格式，大大減小地圖文件存儲空間，提高索引速度。在通過語音發出導航指令時，通過構建導航決策模型，機器人將根據周圍人和靜態障礙物的關係，選擇適宜的行進路線進行躲避。 此方法克服了先前方法的缺點，和常規2D路徑規劃相比，通過構建3D地圖，我們不僅可以躲避桌子等下方為空的物件，並且具有地圖具有語義資訊，可以實現語義導航。此外，與之前的研究相比，藉助LiDAR360度的檢測能力，我們能識別到周圍行人的位置和速度，充分預測行人的未來動作，不僅提升了導航安全性，更可提高在居家環境中躲避行人的社交友好程度。 | zh_TW |
| dc.description.abstract | This thesis proposes an object-aware semantic mapping of indoor scenes with LiDAR and camera fusion for an omnidirectional robot. The aim is not only to perceive the position of the objects for semantic navigation task, but also to achieve a more user-friendly navigation planning based on LiDAR, which can simultaneously avoid pedestrians and static obstacles. In order to achieve the goals, the robot needs to equip and calibrate a multi-layer LiDAR and a RGB camera. The RGB camera is used for object detection, and the detection result will be attached to the corresponding point cloud, thereby obtaining a 3D point cloud map with semantic information. After the creation of a semantic map, it will be further compressed into an octree format, which greatly reduces the storage space of the map file and improves the indexing speed. When the robot is instructed to navigate, through a navigation decision model, the robot will choose a suitable route alter evaluating the geometric the relationship among the free space, the surrounding people and the static obstacles. This method overcomes the shortcomings of the previous methods. For example, the methods of constructing dense 3D semantic maps, mainly focus on the accuracy of 3D reconstruction, so they need high-precision ray, 128-layer LiDAR to obtain a very dense space map, which is unrealistic to use for navigation; On the other hand, unlike conventional 2D path planning, by constructing a 3D map, we can not only avoid the objects like tables, but also have the information to achieve semantic navigation. In addition, with the help of 360-degree lidar detection capabilities, we can identify the locations and speeds of surrounding pedestrians and fully predict these future actions. This not only improves navigation safety, but also improves the social acceptability to pedestrians in the home environment. | en |
| dc.description.provenance | Made available in DSpace on 2021-06-17T06:05:50Z (GMT). No. of bitstreams: 1 U0001-1802202116260700.pdf: 4740855 bytes, checksum: 5d3c27a0c1fe60f3be92b6dc299da1c5 (MD5) Previous issue date: 2021 | en |
| dc.description.tableofcontents | 誌謝 I 摘要 II ABSTRACT III TABLE OF CONTENTS IV LIST OF FIGURES VII LIST OF TABLES IX Chapter 1 Introduction 10 1.1 Motivation 10 1.2 Literature Review 4 1.2.1 LiDAR SLAM 5 1.2.2 Semantic Map 8 1.2.3 Social Navigation Approach 12 1.3 Contribution 19 1.4 Thesis Organization 20 Chapter 2 Preliminaries 21 2.1 Omnidirectional Mobile Robot 21 2.1.1 Hardware Structure 21 2.1.2 Omnidirectional Movement 25 2.2 Robot Operating System 27 2.3 Simultaneously Localization and Mapping (SLAM) 29 2.4 Real-time Human Detection and Tracking 31 2.5 Instance Segmentation Model 32 Chapter 3 Social Navigation System Design 35 3.1 System Overview 35 3.2 3D Semantic Mapping 38 3.2.1 LiDAR-Camera Calibration 38 3.2.2 Instance Segmentation Using Mask R-CNN 41 3.2.3 2D-3D Seeded Growth Algorithm 42 3.2.4 Object Update 45 3.2.5 Map Presentation 46 3.3 The Costmap of Human Tracking 48 3.3.1 Human Detection and Tracking 49 3.3.2 The Costmap Design 52 3.4 Social Path Planning Ruler 54 3.4.1 Local Path Planner 54 3.4.2 Motion Space 55 3.4.3 Robot Motion Decision Process 58 3.5 Voice Control Implementation 58 3.5.1 The Structure of Speech to Text 59 3.5.2 The Keep Away Function 60 Chapter 4 Experiments and Results 61 4.1 Experiments Setup 61 4.2 3D Semantic Mapping 64 4.2.1 LiDAR-Camera Calibration Result 64 4.2.2 Mapping Result 65 4.3 Human Detection and Tracking Results 69 4.3.1 Experiment Setup 69 4.3.2 Result of Different Models 70 4.4 Social Navigation Performance in Real World 71 4.4.1 Experiment Setup 71 4.4.2 Human Feeling-based Evaluation 72 Chapter 5 Conclusion and Future Works 79 REFERENCES 81 | |
| dc.language.iso | en | |
| dc.subject | 社交導航 | zh_TW |
| dc.subject | 全向性機器人 | zh_TW |
| dc.subject | 語義地圖 | zh_TW |
| dc.subject | 激光圖像融合 | zh_TW |
| dc.subject | LiDAR image fusion | en |
| dc.subject | Omnidirectional robot | en |
| dc.subject | semantic map | en |
| dc.subject | social navigation | en |
| dc.title | 基於光達與相機融合之三維語義地圖之靈巧運動機器人 | zh_TW |
| dc.title | Agile Movement Mobile Robot under 3D Semantic Map Built by LiDAR and Camera Fusion | en |
| dc.type | Thesis | |
| dc.date.schoolyear | 109-1 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 施吉昇(Chi-Sheng Shih),陳政維(Cheng-Wei Chen),連豊力(Feng-Li Lian),許永真(Yung-jen Hsu) | |
| dc.subject.keyword | 全向性機器人,語義地圖,激光圖像融合,社交導航, | zh_TW |
| dc.subject.keyword | Omnidirectional robot,semantic map,LiDAR image fusion,social navigation, | en |
| dc.relation.page | 85 | |
| dc.identifier.doi | 10.6342/NTU202100745 | |
| dc.rights.note | 有償授權 | |
| dc.date.accepted | 2021-02-19 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 電機工程學研究所 | zh_TW |
| 顯示於系所單位： | 電機工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| U0001-1802202116260700.pdf 未授權公開取用 | 4.63 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。