使用情緒向量於音樂檢索及曲風描述

Abstract

音樂情緒動態能以二維情緒平面(2-D valence-arousal plane)上的兩點來表示。相較於傳統的單點音樂情緒表示法，這種向量表示法是否能使音樂檢索(music retrieval)更有效，是否能用來展現或描述曲風特性(genre characteristic)是值得探討的兩個基本問題。在這篇論文，我們設計一系列的實驗來回答這兩個問題。首先，我們讓受試者使用這兩種表示法搜尋音樂後以下列七個衡量標準進行評分:功能多樣性(affordance)、學習簡易度(learnability)、易用性(ease of use)、實用性(usefulness)、娛樂性(joyfulness)、新穎度(novelty)及整體滿意度(overall satisfaction)。整體實驗結果顯示向量表示法比傳統的單點表示法在學習簡易度方面略遜一籌，但在功能多樣性、娛樂性和新穎度等方面皆獲得顯著的好評。其次，我們使用向量表示法來分析六種主流曲風的情緒特性，包括藍調音樂(blues)、鄉村音樂(country)、民謠(folk)、爵士樂(jazz)、流行音樂(pop)、以及搖滾樂(rock)。由於向量表示法比起單點表示法可以表示音樂的情緒動態，使得曲風特性的描述更加細緻。實驗中我們使用情緒向量在正負度及激昂度方向上的長度，呈現六種曲風的音樂情緒動態特性。結果顯示在正負度方向上，藍調及鄉村音樂的情緒向量較短、爵士及流行樂的情緒向量則較長；在激昂度方向上，鄉村音樂的情緒向量較短，搖滾樂的情緒向量較長。此外，我們也發現曲風的歌曲情緒向量長度和歌曲結構的複雜度有正向關係。

The dynamics of music emotion can be instantly visualized by a couple of points in a two dimensional valence-arousal plane. However, the effectiveness of this vector representation for music retrieval and genre characterization remains to be explored. In this thesis, we conduct a series of experiments for the effectiveness evaluation of vector representation. First, we build a music retrieval system enabling a subject to search music through either the conventional point representation or the vector representation. The effectiveness for music retrieval is evaluated using seven metrics: learnability, ease of use, affordance, usefulness, joyfulness, novelty, and overall satisfaction. Overall, the vector representation outperforms the point representation in affordance, novelty, and joyfulness, although the subjects need some introduction to get familiar with it. Second, we use both the point and vector representations to characterize the emotion flow of blues, country, folk, jazz, pop, and rock songs. Since the vector representation captures music emotion dynamics, it can be used to analyze the characteristics of emotion for each genre more elaborately. The characteristics of the music emotion dynamics are expressed in terms of the length of emotion vector, in both valence and arousal dimensions. We find that in the valence dimension, the emotion vectors of blues or country music tend to be short, whereas the emotion vectors of jazz and pop music tend to be long. Likewise, in the arousal dimension, the emotion vectors of country music tend to be short, and the emotion vectors of rock music tend to be long. We also find that the length of the emotion vector of a genre generally depends on the complexity of song structure.