差集生成網路--新穎資料生成

Abstract

新穎資料泛指那些不落在訓練資料的分佈中的資料，而他們在某些應用是很重要的，如半監督學習、增強網路的穩定性和異常偵測等。新穎資料通常難以取得，但是如果能夠有演算法能夠產生這些資料並在訓練時使用，那麼將可以大幅增強模型。因此如何產生這些資料是一個常見的研究議題。不同應用所需要的新穎資料往往不太相同，目前針對各種應用也有不同的方法。在這篇論文中，我們提出一個演算法-差集生成對抗網路，能夠產生各種新穎資料。我們發現新穎資料所在的分佈常常是兩個已知分佈的差集，而這兩個已知分佈的資料是比較容易蒐集到的，甚至都可以從訓練資料變化而來。我們將差集對抗網路應用在半監督學習、加強深度網路的穩定性以及異常偵測，實驗結果證明我們的方法是有效的。除此之外，我們也提供理論的證明保證演算法的收歛性。

Unseen data, which are not samples from the distribution of training data and are difficult to collect, have exhibited the importance in many applications (e.g., novelty detection, semi-supervised learning, adversarial training and so on.). In this paper, we introduce a general framework, called Difference-Seeking Generative Adversarial Network (DSGAN), to create various kinds of unseen data. The novelty is to consider the probability density of unseen data distribution to be the difference between those of two distributions p_bar_d and p_d, whose samples are relatively easy to collect. DSGAN can learn the target distribution p_t (or the unseen data distribution) via only the samples from the two distributions p_d and p_bar_d. Under our scenario, p_d is the distribution of seen data and p_bar_d can be obtained from p_d via simple operations, implying that we only need the samples of p_d during training. Three key applications, semi-supervised learning, increasing the robustness of neural network and novelty detection, are taken as case studies to illustrate that DSGAN enables to produce various unseen data. We also provide theoretical analyses about the convergence of DSGAN.