結合特定任務特徵融合之金字塔深度學習網路實例語意分割系統

Abstract

近年來，使用深度學習結合卷積神經網路之物件偵測系統，發展日趨成熟。現今物件偵測系統會將偵測到之物件以邊界框(Bounding Box)來進行標記，然而使用邊界框標記之方式，無法分辨框內前景物件與背景。實例語意分割系統可以被視為物件偵測系統的延伸，將偵測到之前景物件以語意分割的方式進行逐像素的標記，就能避免無法分辨背景與前景物件的問題，進而能產生更精細的標記。 為達到能於車載系統上之即時應用，除了需要有良好的準確度外，運行速度也是一項重要的因素。現行實例語意分割方法，採用先生成物件的邊界框後再產生前景物件的語意分割的方法擁有較高的準確度但是必須付出龐大的運算成本並且犧牲運行速度，另一方面，擁有較高運行速度的方法，會使用較精簡的網路架構，來提高運行效率，但是會犧牲前景物件的語意分割的品質進而影響到準確度的表現。基於觀察對於不同類型的任務各自有合適的特徵以達到較佳的表現，因此本論文提出了一個能夠對特定任務提供不同特徵的實例語意分割架構，透過分離不同任務各自需要的特徵，並且使用一個融合特徵模組產生更豐富的特徵，以達到改善前景物件的語意分割的品質，從而提升整體實例語意分割的準確率。 為驗證本論文提出的方法，我們於目前實例語意分割領域最具挑戰性的MS COCO資料集上進行評估，在評估結果中顯示本論文提出的方法能在接近即時速度運算下具有30以上的AP，並且能夠改善前景物件的語意分割的品質。此外，本研究也於車輛中心公車亭行人資料集上進行實驗分析，以驗證此方法之通用性。

In recent years, the object detection system using deep learning combined with convolutional neural network has become more and more popular and reliable. Instance segmentation research that can provide detailed pixel-level annotations on detected objects can be thought of as an extension of the bounding box annotation. In order to achieve application on the vehicle system, in addition to the need for high accuracy, running speed is also an important factor. Several approaches for instance segmentation generate bounding box first and then segment each bounding box to obtain instance segmentation mask. These approaches have better performance, but need more time consumption. On the other hand, to reduce the computation complexity and increase the runtime speed, some approaches use simpler network. Nevertheless, these approaches have to sacrifice the quality of instance segmentation mask which leads to the decrease of accuracy. In this thesis, we propose a novel instance segmentation network which combines Task-Specific Feature Pyramid and feature fusion module to provide more suitable and richer feature for classification and instance segmentation mask generation. With Task-Specific Feature Pyramid, classification and instance segmentation mask generation don’t need to predict based on the same feature, but still can cooperate with each other, and feature fusion module can provide instance segmentation mask generation richer feature. To validate our approach, we evaluate our method on MS COCO dataset, which is the most challenging instance segmentation dataset nowadays. The experimental results show that our method not only can achieve more than 30 AP on MS COCO dataset at near real-time speed, but also improve the quality of instance segmentation mask.