音樂與演唱影像的整合對於情緒判斷的影響

Abstract

音樂經驗向來被視為是純聽覺的感受，導致視覺影像對音樂經驗的影響常被忽略。本文操弄音樂的大小調，探討影像是否會影響聆聽音樂時的正負向情緒強度。實驗一播放速度與節奏相同的大、小調音樂給參與者聆聽，確立台灣的參與者如同西方的參與者，會將大調樂曲評定為傳達正向情緒而小調樂曲傳達負向情緒。實驗二操弄音樂與影像在大小調的一致性與否，發現影音在大小調上一致時，對於音樂所感受到的正負情緒較強烈（一致性效果）。實驗三的結果進一步顯示，欣賞情緒一致的影音片段比單獨聽音樂還能感受到更強烈的情緒訊息（增強效果）。這些結果顯示表演者的視覺影像可以調節對音樂情緒的感知;當影音兩者的情緒意涵一致時可以增強所知覺到的情緒強度。Overy與Molnar-Szakacs (2009)所提出的SAME模型可適度解釋本研究的結果。

The conventional view that perceived emotion in music is derived mainly from auditory signals has led to neglect of the contribution of the visual image. In this study, we manipulated mode (major vs. minor) and examined the influence of a video image on the music. Melodies in either major or minor mode were controlled for tempo and rhythm and played to the participants. We found that Taiwanese participants, like Westerners, judged major melodies as expressing positive, and minor melodies negative, emotions (Experiment 1). The major or minor melodies were then paired with video images of the singers, which were either emotionally congruent or incongruent with their modes. Results showed that participants perceived stronger positive or negative emotions with congruent audio-visual stimuli (i.e., the congruency effect, Experiment 2). Compared to listening to music alone, stronger emotions were perceived when an emotionally congruent video image was added (i.e., the enhancement effect) and weaker emotions were perceived when an incongruent image was added (i.e., the attenuation effect) (Experiment 3). These results can be explained by the SAME model of Overy and Molnar-Szakacs (2009).