以強化式學習達到機械手臂避障及能量速度優化之軌跡規劃

Abstract

近年來工業自動化逐漸成為發展趨勢，機器人相關應用也日益增加，工業機器人能取代人力，在工廠產線上具有更高的效率，而隨著人工智慧等演算法的技術發展，機器人在性能與控制策略上皆能有所提升，能較好的應付未知或是較複雜的工作環境。本論文著重於使用強化學習方式優化軌跡，並以機器學習的方法補償動力學模型。 在軌跡優化部分，由於本論文研究對象為市售的工業機械手臂，並無開放底層控制器直接控制各軸馬達，故軌跡規劃時必須按照現有上位控制器具備的模式給予指令，而目前給予運動軌跡的方式乃是給定各個via point的位置與速度，透過三次式連接各點形成軌跡，在此限制條件下提升機械手臂的運動效率，主要又可分為能量和時間兩部分。本研究中以Bi-RRT預先生成軌跡決定起始的via point，再透過強化學習的方式，調整前述via point的位置或時間，過程中得到最高分的輸出即為最佳化的結果，在強化學習的獎勵設計同時考慮避障的問題或力矩的限制條件。 而經實驗驗證，在模擬時手臂動力學模型和實際的動態仍存有些許誤差，則真實世界中和模擬上得到的最佳化軌跡表現並不會完全一致，故為了確保模擬時的準確性，還需進一步補償模擬和現實之間的差異。本研究中即採用機器學習的方法，將手臂的運動狀態與模擬、真實力矩的誤差作為標籤資料，透過類神經網路逼近其非線性模型的對應關係。

Industrial automation has become an important issue in recent years, more and more manipulators are applied to manufactories. Instead of manpower, industries prefer automated machine which has more advantages. For example, industrial automation can provide a high productivity, allowing the company to run a manufacturing plant for 24 hours every day. Besides, it has high safety when the plant is in an extreme environment. While artificial intelligence becomes a growing trend, some technologies are applied to robotics to improve the control policies or performance in unknown environment. This research focuses on trajectory optimization and dynamic model compensation. Without directly controlling each joint motor, trajectory command is sent according to the given form, which contains the via points of the trajectory. This research deals with energy and time optimization by using reinforcement learning. Designing the actor-critic agent and reward contains energy/time consumption and obstacle/torque constraints. The actions will change the position and speed of via points, the aim is to get the action of the highest reward. In the experimental stage, the dynamic model exists some differences between simulation and reality. To ensure the availability of simulation, this research uses neural networks to compensate the model.