請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/74435
標題: | 透過序列到序列模型翻譯多模態情感 Translating Multimodal Emotion through Sequence-to-sequence Model |
作者: | Devina Ekawati 王廉花 |
指導教授: | 許永真(Jane Yung-jen Hsu) |
關鍵字: | Emotion Recognition,Emotion Translation,Sequence-to-sequence,Attention Mechanism,Residual Connection, |
出版年 : | 2019 |
學位: | 碩士 |
摘要: | Emotion plays a big role in our daily life. When we try to perceive emotion, we do not only rely on one modality, but rely on several modalities. Psychology studies show that our human sensories perceives several signals from our environment and translate them to codes that are similar across people.
In our work, we formulate the emotion recognition as emotion translation task using Sequence-to-sequence (Seq2seq) models which are widely used in neural machine translation task. Additionally, we add attention mechanism as this mechanism can help the model to remember long sequences. Motivated by Google Neural Machine Translation (GNMT), we also try to add residual connection to resolve the decreasing performance when the models have several stacks of hidden layers. We use CMU-MOSEI dataset to train and evaluate our models. Experiment shows that our proposed Seq2seq architecture outperforms the baseline model on emotion translation task. Moreover, the models that use several modalities achieve better performance than the models that only use one modality. This observation proves that multimodal representation escalates the performance of emotion translation or emotion recognition. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/74435 |
DOI: | 10.6342/NTU201902572 |
全文授權: | 有償授權 |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-108-1.pdf 目前未授權公開取用 | 2.54 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。