利用核函數逼近在離線強化學習中的未標記數據共享

Abstract

離線強化學習方法為從一個固定的數據集中學習策略，但通常需要大量的有標籤數據。由於有標籤數據往往需要人工進行標注，有標籤的數據集通常非常昂貴。相反，無標籤的數據往往成本較低。這種情況凸顯了在離線強化學習中找到有效使用無標籤數據的重要性。在本文中，我們提出了一種利用無標籤數據的離線強化學習方法，並給出了理論保證。我們提出了在再生核希爾伯特空間（RKHS）中的各種特徵值衰減條件，這些條件確定了該算法的複雜性。總的來說，我們的工作提供了一種利用無標籤數據優勢的離線強化學習方法，同時保持理論保證。

Offline reinforcement learning (RL) learns policies from a fixed dataset, but often requires large amounts of data. The challenge arises when labeled datasets are expensive, especially when rewards have to be provided by human labelers for large datasets. In contrast, unlabelled data tends to be less expensive. This situation highlights the importance of finding effective ways to use unlabelled data in offline RL, especially when labelled data is limited or expensive to obtain. In this paper, we present an algorithm to utilize the unlabeled data in the offline RL method with kernel function approximation and give the theoretical guarantee. We present various eigenvalue decay conditions of kernel which determine the complexity of the algorithm. In summary, our work provides a promising approach for exploiting the advantages offered by unlabeled data in offline RL, whilst maintaining theoretical assurances.