請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87289
標題: | 分析神經網路推論於Nvidia Jetson AGX Xavier之能源並以自適應頻率調節優化能源效率 Analyzing the Inference of Neural Network on Nvidia Jetson AGX Xavier and Optimizing the Energy Efficiency through Self-Adaptive Frequency Scaling |
作者: | 洪崗竣 Kang-Chun Hung |
指導教授: | 楊佳玲 Chia-Lin Yang |
關鍵字: | Nvidia Jetson AGX Xavier,能量耗損,神經網絡模型,頻率調節, Nvidia Jetson AGX Xavier,Energy Consumption,Neural Network Model,Frequency Modulation, |
出版年 : | 2023 |
學位: | 碩士 |
摘要: | 近年來,人工智慧在科技發展中扮演重要的角色,而在這其中最為重要的是神經網路。許多網站以及機器都會倚賴神經網路來增進其功能,為人類帶更多的方便。然而,因為神經網路的深度,造成劇增的計算量以及過多的能量耗損。
在神經網路推論的階段,GPU計算以及DRAM資料搬移在Nvidia Jetson AGX Xavier上佔據了約莫67%的總能源耗損。因此,這篇論文將以調節GPU以及DRAM的頻率來達到能源效率。此篇論文中,我們使用多層感知器來當作我們「頻率預測器」的模型架構去預測最節能的頻率設定。再者,我們在推論目標神經網路前(offline),對其做層層分析。依據不同種的網路層,我們提供訓練完成的「頻率預測器」,頻率預測器會根據網路層不同的設定預測出最有能源效率的頻率。之後,我們將所有網路層中預測的結果,取最高的頻率當作上限值,取最低的當作下限值,最後作為目標神經網路推論時,GPU以及DRAM可以浮動的區間。 在Nvidia Jetson AGX Xavier上,此機制在總能量耗損中達到平均20.0%的減少,其中,GPU達到平均29.8%,DRAM達到平均23.7%的能量耗損減少。 Artificial intelligence has played an important role in technology development in recent years, and the essential items in this path are neural networks. Recently, more and more devices and websites have counted on neural networks to improve their functionalities, bringing much more convenience to people in this era. However, thanks to the deeper depth of the recent neural network, the model inference will have massive computations and excessive energy consumption. During the inference stage, GPU computation and DRAM memory access occupy approximately 67% of overall inference energy consumption on Nvidia Jetson AGX Xavier. Therefore, we focus on the frequency modulations of GPU and DRAM in this work. Moreover, in this work, we utilized the multi-layer perceptron models (MLP models) as the frequency setting predictors to predict the most energy-efficient frequency setting. First, we offline layer-wisely analyze the target model, providing frequency setting predictors to determine frequency settings that can achieve maximum energy efficiency. Afterward, among all predicted frequency settings, we select the maximum frequency setting as the upper bound and the minimum frequency setting as the lower bound. Eventually, we set this limitation as the range in which GPU and DRAM frequency can fluctuate during the target model inference stage. This work achieves an average 20.0% overall energy consumption reduction and an average 29.8% and 23.7% energy consumption reduction for GPU and DRAM, respectively, on Nvidia Jetson AGX Xavier during the target model inference stage. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87289 |
DOI: | 10.6342/NTU202300620 |
全文授權: | 未授權 |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-111-1.pdf 目前未授權公開取用 | 2.58 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。