基於深度學習之深度相機於雙臂共工及即時避障

Abstract

智慧工廠為因應高度客製化及頻繁變化生產需求，逐漸引入多機械手臂協作系統增加生產靈活性及安全性，因此具備感知、決策及即時反應能力的多機械手臂跨平台協作與自主避障功能，為實現柔性自動化及人機協作的關鍵技術。本研究整合實務上常見的Window、Ubuntu作業系統，基於上銀與達明機械手臂間即時協作與避障功能，實現異平台機械手臂共工協作之概念。 系統架構透過多處理器、共享記憶體即時傳輸資料，並以ZeroMQ完成跨作業系統之通訊，確保資料傳輸即時性及穩定同步傳輸，並且基於機器視覺使用深度相機結合YOLO影像辨識模型以及Mediapipe辨識人體關鍵點，即時辨識夾取物和障礙物，作為夾取及避障策略之依據。 在路徑規劃與避障策略上，本論文整合全局規劃的快速探索隨機樹(Rapidly-exploring Random Tree Star, RRT*)及局部規劃的人工勢能場(Artificial Potential Field)演算法，以人工勢能場結合逆向運動控制各軸轉角，搭配六邊形引導方式或結合RRT*方法，可有效解決人工勢能場容易陷入局部極小值的問題。本論文將藉由靜態及動態障礙物實驗進行驗證，並且同步以數位影像相關法(Digital Image Correlation, DIC)觀測機械手臂取物及避障的軌跡。 研究結果成功整合異平台及異廠牌機械手臂，達成即時通訊、物件辨識及避障協作，使系統經封裝後具備高度彈性及實用性，提供跨系統多機械手臂協作及人機共工之參考架構。

To meet the increasing demands of mass customization and dynamic production in smart factories, the implementation of collaborative multi-robot arm systems has become essential for enhancing manufacturing flexibility and operational safety. Consequently, the development of cross-platform collaboration and autonomous obstacle avoidance capabilities—equipped with perception, decision-making, and real-time responsiveness—has become a key technology for enabling flexible automation and human-robot collaboration. This dissertation proposes a system that integrates two industrial robot arms—HIWIN and Techman—operating on distinct operating systems, Windows and Ubuntu, respectively. The proposed framework enables real-time cross-platform cooperation and obstacle avoidance between heterogeneous robotic platforms. A multi-processor architecture is adopted to manage data exchange through shared memory, and inter-process communication across operating systems is achieved using the ZeroMQ messaging protocol, which ensures low-latency and synchronized data transmission. In terms of perception, the system incorporates depth cameras and leverages state-of-the-art deep learning models. The YOLO (You Only Look Once) object detection algorithm is employed for real-time object recognition, while MediaPipe is used to identify human body keypoints. These visual cues serve as critical input for real-time obstacle identification and motion planning. For motion planning and obstacle avoidance, this study integrates a global planner—Rapidly-exploring Random Tree Star (RRT*)—with a local planner—Artificial Potential Field (APF). The APF method is applied in the joint space to compute repulsive forces for each joint and convert them into angular adjustments using inverse kinematics and the Jacobian matrix. Additionally, a hexagonal guidance strategy or RRT*-based replanning is introduced to mitigate the issue of local minima commonly encountered in potential field methods. The combined approach enables efficient and collision-free path planning in both static and dynamic environments. The proposed system is experimentally validated through a series of collaborative manipulation tasks involving both static and dynamic obstacles. Furthermore, Digital Image Correlation (DIC) is employed to capture and analyze the motion trajectories of the robot arms during object picking and obstacle avoidance. The results demonstrate the successful integration of heterogeneous robotic systems across different platforms and manufacturers, achieving real-time communication, object recognition, and collaborative obstacle avoidance. The final system, once modularized, exhibits high flexibility and practicality, offering a reference architecture for cross-platform multi-robot collaboration and human-robot coexistence in smart manufacturing environments.