使用全局到局部對齊和 ViT 增強全參考圖像質量評估

Abstract

圖像質量評估(IQA)在理解和改善視覺體驗方面起著至關重要的作用。近年來，已經有大量的 IQA 研究在現有的資料集中有了顯著的結果。這些模型中，特徵融合的部分幾乎是基於合成資料集的圖像而設計的。因此，訓練圖像通常僅限於合成資料集，也就是完全對齊的圖像，這也導致這些模型難以適應真實世界的圖像。在本文中，我們提出了一種強化的全參考 IQA 模型，此模型可以將未完全對齊的輸入圖像，包括位置、視野，經過全局到局部的對齊處理後做特徵融合，並評估圖像質量。 我們的模型分為四個主要部分：特徵提取、交叉注意力視覺變換器、全局對齊和局部對齊。 實驗結果顯示，我們提出的模型在模擬的資料集媲美其他模型的性能。而在真實資料集上，我們提出的模型不僅體現了對齊的重要性，與其他方法相比，我們也取得了非常出色的結果。

Image quality assessment (IQA) plays a vital role in understanding and improving visual experience. In recent years, there has already been plenty of research in IQA that built notable results in existing datasets. The performance of these models is often compromised by the fully aligned datasets constraint in feature fusion. To deal with this, the training images are usually limited to synthetic datasets, causing difficulties to fit in real world. In this paper, we propose an enhanced full-reference IQA model with global-to-local alignment and vision transformer to deal with the input images that are not fully aligned, including position, filed of view. There are four major components in our model: feature extraction, cross-attention vision transformer, spatial transformer network, and the local patch alignment. The experimental results show that our proposed model has a good performance for the standard $synthetic$ IQA datasets. On the $real$ dataset, our proposed model reflects the importance of alignment. Moreover, compared with other methods, we achieve very outstanding results.