基於大型語言模型的外語報刊數據意見挖掘

Abstract

在日益緊密相連的世界中，跨越語言邊界理解事件如何被報導是一項重要的挑戰。本論文以《人民日報》（中國共產黨的官方報章）為例，探討大型語言模型（LLMs）對多語言意見挖掘之使用。雖然基於LLM的方法在近年來已被證實對意見挖掘任務非常有效，但其在多語言任務應用方面的研究仍然相對有限。

本論文的目標是評估基於LLM的方法在單語和多語語境下進行意見挖掘與問答的表現，並檢測內文與提示語言不相符是否會影響結果。第一個實驗聚焦於使用零樣本提示詞識別實體層級的情感，並比較中文、英文及德文的提示詞結果表現。然而，儘管情感分析提供了重要的洞察，卻無法提供關於文本內容的有效資訊。為了彌補此一缺陷，第二個探索實驗使用檢索增強生成（RAG）進行問答，並比較了三種不同架構在不同問題類型上的表現。

在這兩個實驗中，多語言LLM（如GPT-4、Gemini）均表現出穩健的性能，即使在數據資料與查詢語言不一致的情況下，表現差異也非常微小。零樣本提示在情感分析中展現了強大的潛力，對「日本」一詞的可視化情感分析反映了中日關係在一些重要事件期間的預期變化。而在問答任務中，RAG架構的選擇則對性能表現影響顯著，不同架構在處理不同類型的問題時各有所長，突顯了根據任務調整方法的必要性。

這些研究結果表明了基於LLM的方法在多語言任務中的靈活性，即便數據資料和查詢語言有所不同，仍能為情感分析和問答任務提供有效的解決方案。

Understanding how events are reported across linguistic boundaries is a significant challenge in an increasingly interconnected world. This thesis explores the use of large language models (LLMs) for multilingual opinion mining, using The People's Daily, the official newspaper of the Communist Party of China, as a sample use case. While LLM-based methods have proven to be highly effective for opinion mining tasks in recent years, there is still relatively little research on their application to multilingual tasks.

The overall goal of this thesis was to assess the performance of LLM-based methods for opinion mining and question answering in both monolingual and multilingual contexts, evaluating whether mismatches between content and prompt languages impact outcomes. The first experiment focused on identifying entity-level sentiment using zero-shot prompting, comparing the performance of Chinese, English and German prompts. Although sentiment analysis provides valuable insights, it offers no information about the content of the texts. To address this gap, the second experiment explored question answering using Retrieval-Augmented Generation (RAG), comparing the performance of three different architectures across different question types.

Across both experiments, multilingual LLMs, such as GPT-4 and Gemini, showed robust performance, with minimal differences observed when data and query languages did not match. Zero-shot prompting demonstrated strong potential for sentiment analysis, with visualizations of sentiment toward Japan revealing expected shifts during key events in Sino-Japanese relations. For question answering, the choice of RAG architecture significantly influenced performance, with different architectures excelling at different types of questions, underscoring the need to tailor the approach to the task.

These findings underscore the versatility of LLM-based methods for multilingual tasks, offering effective solutions for sentiment analysis and question answering, even in cases where data and queries are in different languages.