在基於非揮發性記憶體的儲存設備上通過輕量級的隨機洗牌方法實現高效率的機器學習

Abstract

支援向量機（Support Vector Machine, SVM）與深度神經網絡（Deep Neural Network, DNN）機器學習演算法在近幾年受到大家的關注。在 訓練機器學習演算法時，對所有訓練資料進行隨機洗牌(Random shuffling) 可以提高測試準確度(Testing accuracy) 與收斂速度(Convergence rate)。然而，由於硬式磁碟機（Hard disk drive, HDD）中的隨機存取 (Random access) 速度慢，在實際系統中實現訓練資料的隨機洗牌並不 是一個簡單的過程。為了避免頻繁地對硬式磁碟機的隨機存取，現有 的解決方法通常會限制隨機洗牌的效果。由於新興的基於非揮發性記 憶體的儲存裝置(Non-volatile memory-based storage) 提供快速的隨機存 取，例如的Intel Optane SSD，我們提出一個輕量級的隨機洗牌方法 LIRS，透過隨機洗亂整個訓練數據集的索引，並直接從儲存裝置中讀 取選定的訓練資料並組成批量(Batch) 以達到隨機洗牌的效果。實驗結 果顯示，採用LIRS 可以使SVM 和DNN 的總訓練時間平均減少49.9% 和43.5%，並使在DNN 上的測試準確度平均提高1.01%。

Machine learning algorithms, such as Support Vector Machine (SVM) and Deep Neural Network (DNN), have gained a lot of interests recently. When training a machine learning algorithm, randomly shuffle all the training data can improve the testing accuracy and boost the convergence rate. Nevertheless, realizing training data random shuffling in a real system is not a straightforward process due to the slow random accesses in hard disk drive (HDD). To avoid frequent random disk access, the effect of random shuffling is often limited in existing approaches. With the emerging non-volatile memory-based storage device, such as Intel Optane SSD, which provides fast random accesses, we propose a lightweight implementation of random shuffling (LIRS) to randomly shuffle the indexes of the entire training dataset, and the selected training instances are directly accessed from the storage and packed into batches. Experimental results show that LIRS can reduce the total training time of SVM and DNN by 49.9% and 43.5% on average, and improve the final testing accuracy on DNN by 1.01%.