基於深度強化學習之預防性保養政策於上下游可修復的耗損機台

Abstract

製造業中，預防性保養一直以來都是一項重要的議題。保養牽涉到了工廠中許多不同面向的問題，比如說良率、機台可靠性、產能穩定性等問題。如果不重視保養策略的制定，工廠將可能導致良率低落、機台無預警的故障以及上下游間機台的運作不順。而過去在工廠中常使用的方法已經逐漸無法適應複雜且具有隨機性的工廠環境，比方說機台的老化造成的機台可靠性不佳、保養時間的長短的效果不同、機台工作量容易變動等。因此在許多的研究中，如何得到良好的保養策略變成一項重要的研究項目。在本篇研究中，利用模擬的方法來還原工廠中上下游機台的環境，並且利用馬可夫決策過程來轉化問題。基於馬可夫決策過程，我們使用了深度強化學習模型來得到保養策略。除此之外，本篇研究模擬了多種不同的工廠環境以進行模型訓練，並為模型找出可解釋能力，使得我們除了能夠利用模型得到一個良好的保養策略以外，更能夠從中理解模型進行決策的關鍵並從中修正策略。

Preventive maintenance has always been an important issue in manufacturing. Maintenance involves many different aspects of the factory, such as yield, machine reliability, and product stability. If neglecting the importance of maintenance strategies, factories may lead to low yields, machine failures, and unsmooth operation of upstream and downstream machines. However, the methods traditionally used in factories in the past have gradually been unable to adapt to the stochastic factory environment. For example, the deterioration of the machine causes the less reliability of the machine, the effect of the length of maintenance time is different, and the workload of the machine is varying. In this study, we simulate the environment which include upstream machine and downstream machine, and the Markov decision process is used to formulate the problem. Based on the Markov decision process, we use a deep reinforcement learning model to derive the maintenance policy. In addition, we apply this method in a variety of different factory environments. Besides getting a good policy, we can also get an explanation from the model and we can understand the key to the model's decision-making and modify the policy from it.