基於學習並使用超像素對之影像分割

Abstract

近期，卷積神經網路（CNN）在圖像分割中已被廣泛採用。但是，現有的基於CNN的影像分割算法多是以單一像素為單位進行預測。由於超像素的不規則形狀和尺寸，很難將超像素直接應用於CNN架構。在本文中，我們提出了多種轉換的機制來使得CNN學習基於超像素的圖像分割。首先提出的算法採用包含兩個超像素的正方形影像作為CNN的輸入，然後CNN的輸出結果是兩個超像素是否應該合併。另外，即使只有很少的訓練圖像，我們的方法也可以從中獲得大量的訓練數據。在第一種算法的啟發下，我們進一步提出了第二種算法來從不同角度出發，該算法利用全卷積網絡（FCN）來解決影像切割的問題。提出的第二種算法將堆疊有彩色圖像的多通道圖像以及諸如超像素邊界圖和邊緣檢測結果的幾個特徵圖作為深度神經網絡的輸入，並輸出超像素邊界圖的預測，該預測圖提供了兩個相鄰超像素的邊界是否應該消失或保留，並進一步地讓我們去執行超像素合併算法。也就是說，通過一次解決第一個算法中的所有子問題，FCN以較大的幅度增進了整個分割過程的速度，同時獲得了較高的精度。總體而言，模擬結果顯示，兩種提出的算法都可以實現非常高精度的影像分割結果，並且在所有評估指標上均優於最新的圖像分割方法。

Recently, the CNN has been widely adopted in image segmentation. However, the existing CNN-based segmentation algorithms are pixel-wise. It is hard to apply superpixels into the CNN architectures directly due to the irregular shape and size of superpixels. In this paper, we proposed different kinds of transformation techniques that leverage the CNN for learning superpixel-based image segmentation. The first proposed algorithm takes a square patch that contains two superpixel as the input of the CNN, and then the output of the CNN is whether the two superpixels should be merged or not. Additionally, one can obtain huge amount of training data even if there are only a few training images. Inspired by the first algorithm, we further proposed the second algorithm that utilizes the fully convolutional networks (FCN) to solve the problem from different perspective. The second proposed algorithm takes a multi-channel image consisted of stacked color image and several feature maps such as superpixel boundary map and edge detection result as the input of a deep neural network, and the network outputs the prediction of superpixel boundary map that indicates whether the boundary of two adjacent superpixel should be keep or not, in a way, merging suprepixels. That is, by solving all the subproblems with just one forward pass, the FCN facilitates the speed of the whole segmentation process by a wide margin meanwhile gaining higher accuracy. Overall, simulations show that both proposed algorithms can achieve highly accurate segmentation results and outperforms state-of-the-art image segmentation methods in all evaluation metrics.