用於二元分類的量子自注意力網路

Abstract

在近年機器學習領域，自注意力機制在發掘數據關聯性和提取重要訊息方面皆有極佳表現。在提供良好計算效果的同時，注意力分數能直觀地呈現輸入序列中 Token 間的關聯性，提供模型決策的可解釋性。使其在自然語言處理、圖像辨識和語音識別等多種機器學習任務中被廣泛應用。 由於量子電腦的希爾伯特空間與自注意力機制中的高維內積計算契合，適合作為其運算平台。因此本研究提出一種結合古典自注意力機制與量子特性的模型架構，利用量子電路調控權重矩陣，生成疊加態的查詢向量(Query Vector)與鍵向量(Key Vector)，並在希爾伯特空間中計算內積以量化 Token 間的關聯性生成注意力分數。該分數衡量輸入資訊的重要性，進一步加權值向量(Value Vector)完成注意力機制，能有效捕捉局部與全局聯繫。由於權重矩陣的維度不再與嵌入向量的維度相關，而是取決於量子電路中的可訓練參數，該架構在降低模型參數數量的同時保留了注意力機制的核心優勢。為驗證模型效果，我們將其應用於Pima Indians Diabetes 二元分類任務和 Netflix 電影推薦系統。實驗結果顯示，該模型在參數降低的同時性能可媲美古典演算法，並且在注意力權重矩陣中展現出兩種編碼方式對於 Token 間的關聯性的不同表現，為模型提供了可解釋性，能幫助理解每個輸入 Token 對模型決策的重要程度，有利於資料分析。

In recent years, self-attention mechanisms have demonstrated exceptional performance in uncovering data relationships and extracting critical information within the field of machine learning. By delivering strong computational results, attention scores intuitively represent the relationships between tokens in the input sequence, providing interpretability for model decisions. This makes self-attention widely applicable in various machine learning tasks, such as natural language processing, image recognition, and speech recognition. Given the compatibility between the Hilbert space of quantum computers and the high-dimensional inner product calculations in self-attention mechanisms, quantum computing offers a suitable platform for such computations. We propose a hybrid model architecture that integrates classical self-attention mechanisms with quantum properties. Leveraging quantum circuits to adjust weight matrices, the model generates superposition states of Query Vectors and Key Vectors. It computes their inner products in Hilbert space to quantify the relationships between tokens and generate attention scores. These scores measure the importance of input information, further weighting the Value Vectors to complete the attention mechanism, effectively capturing both local and global dependencies. As the dimensions of the weight matrices are no longer tied to the dimensions of the embedding vectors but instead depend on the trainable parameters within the quantum circuits, this architecture reduces the number of model parameters while retaining the core advantages of the attention mechanism. To evaluate the performance of the proposed model, it was applied to the binary classification task of the Pima Indians Diabetes dataset and the Netflix movie recommendation system. Experimental results indicate that the model achieves performance comparable to classical algorithms while significantly reducing the number of parameters. Additionally, the attention weight matrix reveals different patterns between the two encoding methods in capturing the relationships between tokens, providing interpretability for the model. This helps in understanding the importance of each input token in the model’s decision-making process, which is beneficial for data analysis.