使用深層卷積網路實現多重極低解析熱影像之超解析影像重建

Abstract

隨著世界人口步入高齡化社會，遠距長照系統的需求逐漸增加，而 系統中常使用攝影機作為工具，其中紅外線攝影機比起一般彩色攝影 機能夠不受光照的影響，更適用於觀測年長者在夜晚時的活動及危險 情況 。雖然高解析度的紅外線攝影機擁有較多特徵可以解析影像的內 容，但高解析度紅外線攝影機的價格相較於彩色相機昂貴許多，且在 重視資料傳輸速率與儲存空間的物聯網系統中，長時間收集的高解析 度熱影像資料過於龐大。根據以上兩個原因，我們採用低解析度的紅 外線攝影機做為我們的主要感測設備。 在這份研究中，我們使用多個極低解析度的熱影像感測器，藉由 紅外線攝影機之間的位置關係重建出較清楚的熱影像。由於低解析度 紅外線攝影機的偵測距離有限，我們固定攝影機與偵測環境之間的距 離，使攝影機能夠看到受測者完整的身形。接著以高解析度紅外線 影像作為標籤，使用深層卷積網路強化多重低解析度熱影像之解析 度。我們的貢獻是能夠使用多個低解析度熱影像重建出受測者的溫 度分佈以及身體和軀幹的形狀，經過超解析度熱影像的PSNR可以達 到20.506dB，且使用超解度影像達到的動作辨識率相較於使用低解析 度熱影像得到12%的改善，另外以高解析度熱影像之動作辨識率為最 佳表現，我們的準確率可以達到最佳辨識率的96.74%。

As the world population enters an aging society, the demand for (remote) long-term care systems is gradually increasing. In the long-term care sys- tems, cameras are commonly used for observing the activity of the subject and detecting the dangerous circumstances. However, RGB camera may re- veal the privacy and infrared camera is more suitable devices than RGB cam- era because it is not sensitive to illumination. Although the high-resolution thermal camera can collect more features to analyze, it is also much more ex- pensive than RGB camera. Furthermore, high-resolution thermal image data collected over a long period of time is unbearable burden for the IoT system which may suffer from low transmission rate and limited storage space. For the two reasons above, we adopt the low-resolution infrared camera in our research. In this work, we used multiple low-resolution thermal images to fuse a clear thermal image through the positional relationship between the infrared cameras. Then we use high-resolution thermal images as labels to train the deep convolutional network. Due to the limitation of the detecting distance of low-resolution thermal camera, in this work the distance between the suubject and cameras is fixed. Our goal is to reconstruct the shape and the tempera- ture distribution of human body in low-resolution thermal images. For the super-resolution result, the PSNR is 20.506 dB on the testing set. For the ac- tivity recognition, the accuracy of SR thermal image has 12% improvement compare to fused thermal image and achieve 96.74% of the optimal accuracy with high-resolution thermal image.