利用對抗式目標與資料擴增於深度強化學習間的遷移

Abstract

在最近幾年內，深度強化學習已經被充分證明可以用來解決高維度的複雜問題，深度強化學習的下一個重點將會是該如何讓神經網路學會不同環境間的核心概念、或是藉由已經學過的知識來加速學習新環境的速度。現今的強化學習訓練出來的模型大部分都有無法很好的處理新環境的缺陷，即便這個新環境與曾經學過的環境十分相近依舊無法有好的成績。我們提出的方法可以讓強化學習的模型從單一環境中學習到較為核心的特徵，並且可以透過半監督式學習來加速模型於新環境中的學習速度。最後我們在一個非常受歡迎的強化學習環境 —— Arcade Learning Environment (ALE) 檢驗我們提出的方法，並且可以發現我們的方法可以打敗常見的標準方法像是使用預訓練模型以及微調網路權重等。

In the past few years, deep reinforcement learning has been proven that can solve problems which have complex states like video games or board games. The next step of intelligent agents would be able to generalize between tasks, using prior experience to pick up new skills more quickly. However, most reinforcement learning algorithms for now are often suffering from catastrophic forgetting even when facing a very similar target task. Our approach enables the agents to generalize knowledge from a single source task, and boost the learning progress with a semi-supervised learning method when facing a new task. We evaluate this approach on Atari games, a popular reinforcement learning benchmark, and show that it outperforms common baselines based on pre-training and fine-tuning.