使用類別映射的零樣本文本分類

Abstract

現有基於大型的預訓練模型並加入提示進行零樣本文本分類的方法，具有模型自身強大的表示能力和擴展性，但商業可用性相對較差。利用類標簽和已有資料集微調較小的模型進行零樣本分類的方法相對簡便，但存在模型泛化能力較弱 等問題。本文使用了三種方法來提高預訓練模型在零樣本文本分類任務上的的準 確性和泛化能力：1. 使用預訓練語言模型，將其輸入整理成統一的多項選擇格式；2.利用維基百科文本數據構建文本分類訓練集，對預訓練模型進行微調；3.提出了基於GloVe 文本相似度的零樣本類別映射方法，使用維基百科類別代替文本類別。不使用待分類標簽進行微調的情況下，該方法取得了與使用待分類標簽進行微調的最佳模型相當的效果。

The existing method of using large pre-trained models with prompts for zero-shot text classification has powerful representation ability and scalability. However, its commercial availability is relatively poor. The method of using class labels and existing datasets to fine-tune smaller models for zero-shot classification is relatively simple, but it may suffer from weaker model generalization ability. This paper proposes three methods to improve the accuracy and generalization ability of pre-trained models in zero-shot text classification tasks: 1) using pre-trained language models and formatting inputs into a unified multiple-choice format; 2) constructing a text classification training set using Wikipedia text data and fine-tuning the pre-trained model; and 3) proposing a zero-shot category mapping method based on GloVe text similarity, using Wikipedia categories to replace textual categories. Without using labeled samples for fine-tuning, the proposed method achieves results comparable to the best models fine-tuned with labeled samples.