弱監督式影像語意分割之少樣本學習

Abstract

在深度學習技術的幫助之下，近年來語意分割模型的辨識率已經得到了大幅度的提升。然而，由於語意分割的目標為替每一個像素分類，此類模型的訓練必須依靠大量含有完整像素資訊（即類別遮罩）的影像樣本。當某些特定類別的訓練影像不足時，如何正確地分辨這些類別的像素便相當具有挑戰性。為了解決前述的問題，我們在本篇論文提出一稱為「弱監督式少樣本語意分割」之學習設定，限制模型在訓練及測試時皆僅能使用影像以及其本身含有的類別名稱（而非每個像素的類別遮罩）。藉由分析輸入影像以及其包含類別名稱之語意標籤，我們首先對每個訓練樣本產生成其偽類別遮罩以做為後續學習的監督來源。接著，我們提出一個可透過偽類別遮罩訓練之元學習少樣本語意分割模型以產生最終之辨識結果。從本篇論文的實驗可得知，我們的架構使用標準資料集在提出的弱監督式設定下大幅超越先前頂尖水準的辨識率，且在常用的全監督式設定下也有良好的結果。

Although promising results have been achieved by recent deep learning models for few-shot semantic segmentation, most of them require a large amount of data with pixel-level ground truth labels (i.e., masks) for training. However, if such ground truth information is not sufficient for particular image categories, learning to segment such images becomes an even more challenging task. To address the above problem, we propose a novel learning framework in an extremely weakly supervised setting, where only image-level labels are observed during both training and testing (but not pixel-level masks). By observing the input image and its semantic label, we first generate its pseudo pixel-wise semantic mask, which guides the learning of our meta-trained architecture for segmentation purposes. Through extensive experiments on benchmark datasets, we show that our model achieves satisfactory performances under fully supervised settings, while performing favorably against state-of-the-art methods under weakly supervised settings.