用於伴奏分離的輕量化深度學習模型

Abstract

音樂聲源分離旨在分離出一首歌曲當中的不同音軌，本篇研究著重於對伴奏軌的即時分離。過去的音樂聲源分離模型傾向於提升分離品質，但這使得模型的大小和延遲時間增加，難以在邊緣裝置上進行運算。此外，大多數方法在降低輸入秒數時會明顯降低分離品質。本論文改進Sony早期提出的輕量化模型MMDenseNet，希望在分離品質、延遲時間及空間資源三者間達成平衡。儘管MMDenseNet的參數量很低，但分離品質不理想，且在低延遲時情況下表現不佳。因此，本研究提出了三個改進方向，分別為訓練目標調整、模型架構調整、以及訓練及測試方法改進，試圖在維持空間資源表現的情形下改善模型分離品質與延遲時間。我們使用MUSDB18資料集進行訓練及測試，並使用SDR作為分離品質評估指標，計算延遲時間作為延遲評估指標，使用參數量作為空間資源評估指標。根據實驗結果，調整模型的訓練目標能夠在維持空間資源與延遲時間下提高分離品質(median SDR從11.162提升至13.951)。此外，提出的多種自注意力架構使MMDenseNet在稍微增加空間資源及延遲時間的情況下提升分離品質(median SDR從13.951提升至15.011)。最後，我們提出的漸進式訓練及測試方法使得模型在低延遲下能夠保持良好的分離品質(延遲時間1.19秒時，RTF為0.4031、median SDR 由 13.951提升至14.394)。

Music source separation aims to separate different tracks within a song, and this study focuses on the real-time separation of the accompaniment track. Previous music source separation models have prioritized improving separation quality, but this has resulted in increased model size and latency, making it challenging to perform computations on edge devices. Additionally, most methods exhibit a significant decrease in separation quality when reducing the input duration. This paper improves the lightweight model MMDenseNet proposed by Sony in the early stages, aiming to achieve a balance between separation quality, latency, and spatial resources. Although MMDenseNet has a low number of parameters, its separation quality is not ideal and it performs poorly in low-latency scenarios. Therefore, this research proposes three improvement directions, including adjustments to the training objectives, model architecture, as well as improvements in the training and testing methods, aiming to enhance the separation quality and latency while maintaining the parameter count. We utilize the MUSDB18 dataset for training and testing, using SDR as the evaluation metric for separation quality, measuring latency as the delay evaluation metric, and considering parameter count as an indicator of spatial resources. Based on the experimental results, adjusting the model's training objectives improves the separation quality while maintaining spatial resources and latency (median SDR improves from 11.162 to 13.951). Furthermore, the proposed various self-attention architectures enable MMDenseNet to enhance the separation quality with only a slight increase in spatial resources and latency (median SDR improves from 13.951 to 15.011). Finally, the progressive training and testing methods proposed in this study allow the model to maintain good separation quality at low latency (at a delay of 1.19 seconds, RTF is 0.4031, and median SDR improves from 13.951 to 14.394).