應用遷移學習與文字探勘分析致股東報告書

Abstract

本研究先以自然語言處理方法中的BERT (Bidirectional Encoder Representation from Transformers) 建立文字探勘模型，並利用致股東報告書對BERT進行微調 (fine-tuning)。接著探討BERT是否解決過往文字探勘方法的缺點，最後以情緒分析 (Sentiment Analysis) 剖析致股東報告書的語調，研究致股東報告書語調對於公司未來績效的影響。 實證結果顯示，致股東報告書須針對中英夾雜問題做前處理，而經過驗證資料集表現篩選超參數 (hyperparameter) 後，BERT模型分類準確率高達86%。經過視覺化BERT模型的運作，發現其能捕捉否定詞修飾的詞彙，且同樣能成功捕捉形容詞所修飾的名詞。語境測試結果顯示，將文字順序隨機打亂後，BERT表現大幅下滑，因此可知BERT確實有學習到語言結構。 然而關於語調對公司未來績效的影響，從實證結果發現，當年(t)的致股東報告書情緒對隔年(t+1)的盈餘並無顯著影響，推論原因可能是樣本篩選不夠具代表性，或是台灣致股東報書本身與美國的MD A資訊含量有差異，導致台灣的致股東報告書與未來盈餘並無呈顯著關聯。

First, this study applies BERT (Bidirectional Encoder Representation from Transformers) to construct a text mining model, and uses Report to Shareholders to fine-tune BERT. Next, we will discuss whether BERT can overcome some weaknesses of traditional text mining techniques. Finally, this study tries to assess the impact of the tone in Report to Shareholders on company’s future performance by using Sentiment Analysis. The empirical result shows that the problem of mixing Chinese and English in Report to Shareholders must be tackled, and after choosing the best hyperparameter based on validation performance, the classification accuracy reaches up to 86%. By visualizing the operation of BERT, we find that BERT can not only capture the relation between the word and its negation, but also capture the relation between the adjective and the noun successfully. The result from the context test also shows that the performance of BERT drop significantly after the text sequence is randomly shuffled, so it is considered that the language structure of Chinese is learned by BERT. However, regarding to the impact of tone in Report to Shareholders on the company’s future performance, the empirical result shows that the sentiment in Report to Shareholders has no significant impact on the next year’s earnings. The results suggest that the sample may not be representative enough or Taiwan’s Report to Shareholders has less information values than the US’s MD A information content so that there is no significant relation between the tone and the next year’s earnings.