基於 TD3 演算法與模糊獎勵系統的 PD 控制器參數最 佳化之設計與應用

Abstract

改善機器人操控器的效能與運動精確性，一直是控制系統領域中的重要研究主題之一。本文提出了一種應用於六自由度機器人操控器的自適應控制架構設計，目的是提升其在變動環境中的追蹤能力與反應效率。此控制系統整合了強化學習與模糊邏輯技術，以實現即時調整比例-微分（PD）控制器的控制參數。所設計的模糊獎勵機制可用於即時評估整體控制表現，考慮因素包括偏差大小、其變化速率以及施加的控制力。該模糊評估系統結合了 TD3（Twin Delayed Deep Deterministic Policy Gradient)強化學習演算法，這是一種基於actor-critic架構的先進策略學習方法。透過與外部環境的互動，系統能逐步學習出最適的控制策略，並根據模糊獎勵提供的回饋來調整 PD 控制器的增益設定。在 Simulink 環境中結合 MATLAB 工具箱所進行的模擬實驗，驗證了該控制器的有效性。透過對六自由度機器手臂的實驗，展示了其在實際應用中的控制表現。這些模擬比較了結合模糊邏輯與強化學習的自適應控制策略與傳統固定增益PID控制器之間的性能差異。結果顯示，該創新方法在追蹤精度方面有顯著提升，且更能適應系統動態的變化。此一控制框架在機器人手臂及其他複雜機電系統的自主控制領域中，具備潛在的應用價值與貢獻。

Improving the performance and accuracy of robotic manipulators has been a very important research areas in control systems engineering. This article presents the development of an adaptive control framework for a six-DOF robotic manipulator, aiming to improve its motion accuracy and responsiveness in a dynamic environment. The proposed controller focuses on the online tuning of a proportional derivative controller parameters combining reinforcement learning with fuzzy logic. The fuzzy reward system is designed to evaluate system performance in real time, considering error, error rate, and applied control force. Fuzzy rewards are combined with a reinforcement learning method, the Twin Delayed Deep Deterministic Policy Gradient algorithm operates using an actor-critic methodology. TD3 lets the system achieve a better control policy which engages with the environment and adjusts the PD controller’s gains based on feedback from fuzzy rewards. The simulations conducted in Simulink with the use of Matlab toolboxes confirmed the efficacy of the controller. The experiments done with the six-DOF robot arm showed its performance in real-life applications. These simulations assessed the performance of an adaptive control strategy that utilizes fuzzy logic and reinforcement learning against a conventional fixed-gain PID controller. The findings indicated that the novel approach notably enhanced tracking precision and adaptability to changes in system dynamics. This control framework could potentially contribute in the area of autonomous control systems for robotic arms and other intricate mechatronic systems.