內省式高效語意分割錯誤偵測方法

Abstract

基於深度學習的語意分割技術日漸成為許多自駕載具感知系統中重要的一環，然而，在面對自駕載具駕駛環境分布多樣且複雜的情況下，語意分割模型在實際推論時不可避免的會有較大效能下降的現象，若無法在運行過程中監控模型分割的狀況，對於自駕載具的安全性來說將是個很大的疑慮。因此，本研究提出一套內省式的高效語意分割錯誤偵測模型，透過觀察語意分割模型失效時所對應到的內部特徵狀態來訓練學習，並在推論階段擁有實時預測分割錯誤的能力，藉由輸出像素級別的信心指數圖，了解模型對於當前環境辨識的狀況。 本研究之內省模型提取語意分割內部的多種特徵來做為輸入，並透過基於通道注意力機制的模組與特徵金字塔的模型架構來幫助模型更有效的融合使用這些特徵。在訓練階段時採用與語意分割模型聯合學習(joint-learning)的方式，使模型從過程中獲得更豐富的潛在特徵，大幅提升了預測的準確性與泛化性。除此之外，在訓練過程中還引入了自注意力蒸餾(Self Attention Distillation)，使其在不增加參數的情況下透過編碼器內部淺層與深層的注意力圖互相學習，進一步提升性能表現。最後本研究將此錯誤偵測框架使用在不同的語意分割模型(FCN-8、ESANet)，並且都在分布內資料集(Cityscapes)與分布外資料集(BDD100K)做驗證，比較了三項性能指標(AUPR、AUROC、FPR95)，均為現今相關研究方法中的最佳。另外，本研究的模型採取輕量化的設計，在單張 NVIDIA RTX 2080Ti 顯示卡上比先前性能表現最好的模型減少了約40%的推論時間，並且在嵌入式電腦(JETSON AGX XAVIER)上達到約29FPS的推論速度。

Semantic segmentation has gradually become a critical component in perception system of self-driving vehicles. However, due to diversity and complexity of driving environment, the semantic segmentation model will inevitably have a large performance drop during deployment. It will be a great concern for the safety and reliability. As a result, we propose an introspective and efficient fault detection model to monitor semantic segmentation model during run-time. The introspective model extracts multiple internal features of segmentation model as input to predict pixel-level confidence map. We utilize a channel attention mechanism module and feature pyramid architecture to help the model integrate input features more effectively. In the training phase, we use joint-learning method to train the two model at the same time, which can enhance the generalization ability. Moreover, without the need of increasing computing resource, a self attention distillation method was introduced to further improve the model performance. In our experiment, we use proposed introspective model to monitor several typical segmentation models, and evaluate on Cityscapes and BDD100k segmentation datasets. The result shows that our approach significantly outperforms all existing methods in three critical performance metrics (AUPR, AUROC, FPR95). In addition, our model adopts a lightweight and efficient design, which reduces the inference time by 40% than the state-of-the-art framework, and the computing speed can achieve 29fps on an embedded computer (NVIDIA JETSON AGX XAVIER).