請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/51054
標題: | 基於GRU的序列對序列自動編碼器的神經元功能之分析 Exposing the Functionalities of Neurons for Gated Recurrent Unit Based Sequence-to-Sequence Autoencoder |
作者: | Yi-Ting Lee 李漪莛 |
指導教授: | 林守德(Shou-De Lin) |
關鍵字: | GRU,序列對序列模型,自動編碼器,神經元功能, Gated Recurrent Unit,Sequence-to-Sequence Model,Autoencoder,Neurons functionalities, |
出版年 : | 2020 |
學位: | 碩士 |
摘要: | 本文的目的在報告有關Seq2Seq模型的科學發現。眾所周知,由於RNN本質上具有遞歸機制,因此在神經元級別的分析會比分析DNN或CNN模型更具挑戰性。本文旨在提供神經元級的分析,以解釋為什麼基於單純GRU的Seq2Seq模型不需attention的機制即可成功地以很高的正確率、照順序輸出正確的token。我們發現了兩種神經元集合:存儲神經元和倒數神經元,分別存儲token和位置信息,通過分析這兩組神經元在各個時間點如何轉變以及它們的相互作用,我們可以揭開模型如何在正確位置產生正確token的機制。 The goal of this paper is to report certain scientific discoveries about a Seq2Seq model. It is known that analyzing the behavior of RNN-based models at the neuron level is considered a more challenging task than analyzing a DNN or CNN models due to their recursive mechanism in nature. This paper aims to provide neuron-level analysis to explain why a vanilla GRU-based Seq2Seq model without attention can successfully output correct tokens in the correct order with a very high accuracy. We found two types of neurons set, storage neurons and count-down neurons, storing token and position information respectively. By analyzing how these two group of neurons transform through the time step and how they interact, we can uncover the mechanism of how to produce the right tokens in the right positions. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/51054 |
DOI: | 10.6342/NTU202002828 |
全文授權: | 有償授權 |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
U0001-1008202016174700.pdf 目前未授權公開取用 | 5.66 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。