動態派工與預保養之強化學習方法初探

Abstract

本研究提出一適用於大型生產系統多智能體系統(Multi-Agent System, MAS)，以分散式求解方式對大型智慧生產系統求解的概念，此多智能體系統中包含了分解代理人(Decomposition Agent), 機器代理人(Machine Agent), 協作代理人(Negotiation Agent) 及合作代理人(Coordination Agent)四種不同的代理人，並利用此拆分、求解及合併的思維得到大型生產系統的較佳初始解，進而透過強化學習方法優化生產系統中的決策。 而在多智能體概念中，本研究主要為提升機器代理人(Machine Agent)在求解過程中的效率及初探強化學習在生產系統求解上的應用。故本研究分為兩個部分，第一部分為介紹如何利用深層神經網路及多項式回歸方法預測兩產品單機台的較佳初始解，接著針對此較佳初始解利用動態規劃繼續求解至收歛，並在研究中展現此一方法更為單純動態規劃求解方法節省求解時間，且與之求解出之解無異。 第二部份則為生產系統之強化學習初探。本研究開發出一用於兩產品單機台生產系統的強化學習演算法，其概念為利用強化學習代理人(Agent)與環境(Environment)互動此一特性研發演算法，並直接於演算法內設計類離散事件模擬動作的過程，以實現強化學習中環境這一環節。最後，將在強化學習過程中的決策與動態規劃求解的決策透過模擬比較其平均總完工時間(Average Cycle Time)，結果顯示本研究提出之強化學習演算法能有效求解兩產品單機台生產系統的動態派工與預防保養決策。

This research proposes a multi-agent system that uses a distributed approach to solve a large-scale manufacturing system. This MAS includes four different agents: decomposition agent, machine agent, negotiation agent, and coordination agent. The decomposed systems are solved and then combined to get a better initial solution. Then, a reinforcement learning algorithm is used as an online learning method to further improve the solution of the large-scale manufacturing system. The main objective of this research is to improve the solving time of the machine agent in the MAS as well as to do an2 exploratory study of using reinforcement learning in a manufacturing system. There are two parts in this research. First, we use deep neural network and polynomial regression to predict a better initial solution of a two products one machine system. Then, we use this as initial solution to get an optimal solution using dynamic programming. We compare in terms of solving time and the policy. Second, this research proposes a reinforcement learning algorithm for the two products one machine system. In this algorithm, we design a similar simulation method to implement the environment of a reinforcement learning system. We use the solution found by the method in part one as an initial solution to this algorithm. The result of the study shows that our reinforcement learning algorithm can be used in solving a manufacturing system.