建立一個沉浸式擴增實境之銑削模擬與訓練系統

Abstract

屆於傳統工具機技術人才的流失與老化，許多以經驗傳承的加工技術已逐漸失傳。加上傳統工具機一對多的教學與安全限制，通常導致學習效果不佳。如何將這些技術保存下來並訓練下一代加工人才，是一件緊迫且重要的課題。本研究利用擴增實境 (Augmented Reality, AR），將加工技術以資訊科技的方式模擬與保存，並以接近操作真實工具機的互動模式，達成一對一的訓練效果與經驗傳承的目的。 擴增實境為利用電腦繪圖與攝影機方位的計算，結合虛擬影像與真實環境的一種技術。本研究建立了一個基於AR的銑削訓練系統。使用者可利用實際的雙手與肢體動作，和與真實環境中一樣的自然操作模式，操作與實體大小一樣的虛擬銑床。本研究藉此希望能夠改善學習效果，並減少使用銑床的潛在危險與成本。 本研究應用Intel® RealSense™ R200攝像機獲取室內場景的彩色和深度資訊；使用Microsoft Kinect v2來追蹤使用者的身體運動資訊，以增強使用者與虛擬物體的互動；結合Leap Motion來追蹤使用者的手部姿態； 最後，藉由Oculus Rift頭戴式顯示器結合真實場景和虛擬物體，並提供AR影像予使用者。 本研究整合了不同相機間的座標系統，讓使用者能夠以雙手與肢體以自然的操作方式，與虛擬物體互動。本研究並提供了及時動態遮蔽處理的方法。該方法使用了校正後的深度資訊作為深度緩衝 (Z-buffer) 覆寫，使每個像素能夠正確顯示出真實物體與虛擬物體之間的遮蔽關係。 最後，使用者測試結果顯示，本研究所提出的AR訓練系統比傳統教學影片的訓練效果更為顯著，同時也證明本系統的互動是直覺、流暢且具有樂趣的。使用者測試結果可做為未來擴增實境應用於工具機訓練與模擬的參考。本研究的示範影片網址為https://youtu.be/k7TeRTYiD-8。

Since the lack of and aging of the traditional machine tool technicians, many machining techniques have gradually lost. Furthermore, because of the uneven resource and the safety issues in the traditional machine tool training, the training effect is usually not significant. It is an imperative and important issue to preserve those techniques and to train the next-generation machine tool technicians. In this study, Augmented Reality (AR) is used to simulate and preserve machining technologies. Through the realistic and immersive interactions with a full-size virtual machine, the purpose of one to one training and passing on the traditional machining techniques can be achieved. AR is a technology combining virtual objects with real scenes, captured from camera, using computer graphics rendering technology. This study created an AR-based milling machine training system. Users can operate a virtual milling machine by using their natural behaviors in a physical environment. The system is expected to enhance users’ learning effect and reduce the accident and the cost in using a milling machine. Intel® RealSense™ Camera R200 was used to get the depth information of the indoor scenes. Microsoft Kinect v2 was used to get user’ body motion information to enhance the interactions between users and virtual objects. Leap Motion was used to get user’s hand gestures. Oculus Rift head-mounted display was used to merge real scene images and virtual images. A calibration board was used to correct the translation error between the different camera coordinates. A calibration method was developed to solve the dynamic occlusion problem in real time. The approach overwrote the Z-buffer in the drawing library with the calibrated depth information. The depth information of each pixel was compared to show the partial occluded images. Finally, user test results show that the training effect of the developed AR system is better than the traditional education video. The user test also shows that the system is intuitive, smooth, and interesting. In this study, the feasibility of the AR system was validated and could be used as a reference for future AR-based training and simulation for machine tools. The AR demo can be found at https://youtu.be/k7TeRTYiD-8.