利用後綴樹以及事件軌跡分析深度學習系統之行為與效能

Abstract

近年來深度學習因為加速器的普及而得以發展迅速，眾多開放原始碼的深度學習框架提供使用者簡潔易用的開發環境，往往只需要簡短的程式碼描述神經網路模型，就能以深度學習解決許多領域的問題。然而深度學習之應用程式非常仰賴加速器來提高運算能力、減少訓練的時間，簡短的程式碼所產生的繁複執行過程，包括在CPU上執行資料前處理管線、在加速器上執行神經網路模型所定義的計算、以及CPU與加速器之間互相溝通傳遞資料的過程，不易被一般使用者所理解與分析。目前常用的深度學習框架與系統效能分析工具雖然提供若干與效能相關的資訊，但在使用上仍然有相當大的門檻，一般使用者難以駕馭。本篇論文探討針對深度學習之應用程式與框架設計一套分析方法，運用實驗室現有的分析工具SOFA所收集事件軌跡(event traces)，但若直接以原始事件軌跡作為分析基石，則會占用龐大效能資源，造成分析時間過久，本文利用特徵擷取與後綴樹找出深度學習應用中最重要的疊代(iteration)片段，再此工具偵測出的疊代資訊以SOFA自動產生出效能報告，以利使用者分析效能瓶頸。最後探討以效能報告之方向，舒緩瓶頸後的改善結果，在其中最好的案例下，能達到6.25倍的速度提升，由此可知這套工具能有效的幫助開發者找出效能之瓶頸，並加以改善。

In recent years, Because of the popularization of the accelerator, the development of Deep learning also rise rapidly. Many open source deep learning framework provides users with a simple and easy-to-use development environment. Often only use a short program code to describe the neural network model, can solve many areas of problems. However, deep learning applications rely heavily on accelerators to improve computing power and Reduce training time. The tedious execution of short program code, including processing pipelines before data is executed on the CPU, performing the calculations defined by the neural network model on the accelerator, and the process of communicating data between the CPU and the accelerator, not easy to be understood and analyzed by the general user. At present, the commonly used deep learning framework and system efficiency analysis tool provides some information related to the performance, but there is still a considerable gap in the use, the general user is difficult to master. This paper aims at the deep learning application and framework design a set of analytical methods, using the existing analytical tools of our lab 'SOFA' to record the event traces, but if the original event trace is used as the cornerstone of analysis, it will take up a large number of efficiency resources, resulting in massive analysis time, In this paper, feature extraction and suffix tree are used to find the most important iteration phases in the deep learning application, and the iteration information detected by this tool can automatically generate performance report by return the value to SOFA to analyze the performance bottleneck. Finally, the paper discusses the suggestion of performance reporting, alleviate the bottleneck of the improvement results, in the best case, can achieve 6.25 speedup, Proven that this tool can effectively help developers identify bottlenecks in performance and improve.