以大型語言模型處理災情文本之嚴重度分類：強化救災應變

Abstract

近年來，快速應對自然災害後的災情成為重要課題，受到越來越多的研究關注。以往，大量災情通報訊息傳至應變中心時，救災人員常因判斷耗時或誤判嚴重程度，影響救災效率。本研究透過大型語言模型 (Large Language Model, LLM) 微調，建立災情嚴重程度分類模型以辨識災情通報，並根據結果進行後續救災規劃。研究收集了應變管理資訊系統 (EMIC)、農業部農村發展及水土保持署 (ARDSWC)、災情新聞文本與生成式人工智慧模擬文本四個來源的災難通報訊息，建立訓練和測試資料集。整合資料集後，進行兩種類型的標註：第一種針對道路阻塞、建築物倒塌、人員受傷、死亡、失蹤五個項目分類；第二種則根據人員傷亡情況，以台灣現行「大型傷亡事件」定義進行分類，並加入無人傷亡與無法確認傷亡情況兩個分類。實驗中，使用Bidirectional Encoder Representations from Transformers (BERT)和Generative Pre-trained Transformer (GPT)進行模型微調。結果顯示，微調後的GPT模型在五個分類器的表現均優於BERT模型，且訓練時間更短。因此，後續標註與模型微調均採用OpenAI平台的GPT模型。經改進資料量與不平衡問題，微調模型取得更佳結果。最終，微調GPT模型準確辨識大型傷亡事件，並在救災派遣最佳化模型中應用災難嚴重程度作為事件權重、處理時間、派遣次數等參數。參考戴至佑(2018)的高速公路事件應變最佳化模型，本研究設計多站點車輛路徑問題模型，處理應變隊伍對城市內災難事件的指派任務，並最小化系統總應變時間與單一應變隊伍總應變時間。以2018花蓮地震案例分析，微調GPT模型成功辨識所有災難事件，救災派遣模型使整體應變時間減少22%，重大傷亡事件應變時間減少34%。敏感性分析顯示，加入微調模型辨識結果的情境在特定參數設置下，重大傷亡事件應變時間下降最高16%。

In recent years, how to respond quickly to natural disasters has become an important issue and has received increasing research attention. Disaster relief assignments based on personnel judgment can be challenged when a large amount of disaster report messages are transmitted to the response center, which may take a prolonged time to judge or misjudge these messages. Hence, this research seeks to develop a model that can automatically identify the severity of disasters and provide accurate countermeasures based on the disaster information transmitted to the system. This study collected disaster-related texts to create a dataset for training and testing a disaster severity classification model. The primary sources include the Emergency Management Information Cloud (EMIC) and the Taiwan Agency of Rural Development and Soil and Water Conservation (ARDSWC). To address data imbalance in these government datasets, additional disaster descriptions from news sources were collected, and more data were generated through generative AI simulations based on descriptions in the EMIC dataset. The integrated dataset is labeled differently in two types. The first type of labeling is divided into five disaster categories, including blocked roads, building damage, personal injuries, deaths, and missing, according to their respective severity. The second type of data labeling focuses on the casualties. Therefore, the labeling of casualties is based on Taiwan’s current definition of “mass casualty events”, which uses more than 15 casualties as the classification standard, and adds two additional categories: no casualties and unknown casualties. In the experiment of fine-tuning LLMs, firstly, the two selected LLMs BERT and GPT are fine-tuned on the five disaster categories labeled with the first type of labeling. It can be found in the results that the fine-tuned GPT model trained through the OpenAI platform performs better on the five classifiers than the fine-tuned model combining BERT and neural networks, probably because it is easier for the OpenAI platform to produce more stable fine-tuning results when the dataset is not large. Further, the time required for fine-tuning training in the OpenAI platform is less. Therefore, the OpenAI platform is finally used to fine-tune the GPT model to continue testing with the second type of data labeling. The results show that a fine-tuned model can achieve better results by increasing the amount of training data and improving the problem of data imbalance. Because of the different writing patterns of disaster news and those on EMIC, generative AI is used to rewrite them in a similar manner to generating additional texts to improve the consistency within the data, which also leads to better predictions and recall rates. The fine-tuned GPT model with the highest accuracy for mass casualty events is used to identify incident severity and develop the disaster relief dispatch optimization model. Incident severity parameters include weight, processing time, and required dispatches. Referring to Tai's (2018) optimization model for highway incident response, this study designs a multi-depot vehicle routing problem for dispatching disaster response teams. The model aims to minimize both the total system response time and the response time of individual teams. For mass casualty incidents, multiple teams are assigned simultaneously, with additional teams called for support as needed. This study uses the 2018 Earthquake in Hualien City as a case study that considers the reported disaster impact 40 minutes after the earthquake occurred. Through the fire station dispatch records, we can learn the description text of the disaster incidents, the assigned teams and other information. GPT’s fine-tuned model can correctly identify all disaster incidents in the case, and after being incorporated into the disaster relief dispatch model, the overall response time is reduced by 22% compared with the response time in the original case, and the response time for mass casualty events is reduced by 34%. Moreover, in the sensitivity analysis, it is found that the response time for mass casualty events is reduced by up to 16% with certain parameter settings when the severity classification results of the fine-tuned model are added as event weights compared to the scenario without event weights.