友善於記憶體內運算的深度學習三維重建

Abstract

現今有許多應用使用到三維模型，像是智慧型手機使用的人臉辨識或虛擬實境所使用的角色模型，三維模型可以提供比二維模型更好的安全性以及更生動的表現。然而三維模型往往需要許多的運算資源消耗許多的能量，這使得三維模型的使用無法是永遠開啟的。為了改善能量消耗的問題，運算記憶體的架構被提出，透過運算記憶體即可使能量消耗大幅減少，使三維感測器可以永遠開啟達到實時的效果。而要在記憶體內運算有許多限制，包括只能使用卷積運算、只提供整數的運算以及類比電路所產生的非線性誤差。為了解決這些記憶體內運算所帶來的限制，本論文，可以訓練出適用於運算記憶體的深度學習模型，包括輸入、輸出以及權重的量化、每層網路輸出的標準化、網路整數權重的更新以及解決類比電路所帶來的誤差。在訓練出能在記憶體內運算的網路後，三維重建的深度網路便可使用在記憶體內運算的系統上。本篇論文的三維重建結果誤差能在1釐米以內，此方法亦能在較為複雜的面具模型上重建三維模型，在此誤差範圍下運算記憶體的三維模型可以適用於各種應用上，像是智慧型手機的人臉辨識系統。有了運算記憶體的特性可以實現低能耗以及永遠開啟的感測裝置。

Many 3D recognized systems use the 3D points cloud as input to improve the accuracy and security, such as face recognition used in smartphones or avatar models used in virtual reality. However, processing the 3D models often requires much computation, which consumes much energy and makes using 3D models impractical to be always-on. The Computing in Memory (CIM) architecture is proposed to reduce the energy consumption of the convolution operation. Through the CIM chip, the energy consumption can be significantly reduced so that the 3D sensor can be always-on. However, CIM operations have many limitations, including convolution-only operations, integer-only operations, and non-linearity errors caused by analog circuits. In order to solve the CIM limitations, this work proposes a set of procedures to train a deep learning model that is friendly to the CIM chip, including quantization of inputs, outputs, and weights, normalization of the network's output, weight updating in integer precision, and overcoming the error caused by analog circuits. After training the CIM-friendly network, the 3D reconstructed deep learning network can be used on the CIM system. The distance errors of the 3D reconstruction results in this work can achieve less than 1 cm. This method can reconstruct the 3D model on the more complex mask model. Within this error range, CIM 3D reconstruction can be applied to various applications, such as face ID systems for smartphones. The 3D reconstruction network for CIM can achieve low power consumption and be always-on sensing so that the application of 3D models can be more widely used in various devices.