利用圖片美學的學習進行濾鏡推薦

Abstract

近幾年，隨著智慧型手機以及3G網路的普及，社群網路平台的使用人數大幅地上升。因此，使用者也逐漸習慣在這些平台上分享圖片，並藉此吸引更多關注。為了讓使用者能在圖片上增加不同的效果或改善圖片的品質，許多社群網路平台提供濾鏡給使用者使用。透過濾鏡的使用，使用者可以在沒有任何修圖基礎下，輕易地改變圖片的外貌。但是隨著濾鏡數量的成長，如何在短時間選擇最好的濾鏡就成為使用上最大的問題。因此，在這種圖片品質的排名問題中，學習圖片的美學就成為很重要的一部份。在最近的研究中顯示，卷積神經網路在圖片美學的分類上的表現已經超越過去傳統的做法。在這篇論文中，我們將介紹一種新的方法來學習圖片的美學，同時利用圖片的比較來建立一個新的數據集。此外，我們使用不同的卷積神經網路架構以及圖片配對比較的損失函數幫助圖片美學的學習。藉由兩張圖片間的比較結果，我們的訓練模型能夠將圖片的美學反應嵌入隱藏層中。除此之外，我們也利用圖片的主題類別幫助美學排名的學習。為了訓練並評估我們的方法，我們建立了一個新的數據集。裡面包含多於三萬張套用過濾鏡的圖片，以及四萬多個經過人工標註的圖片配對。就我們所知，這是第一個使用濾鏡過的圖片及其美學標註的數據集。我們的實驗結果也顯示利用圖片配對的比較方法能夠比傳統的美學分類方法達到更好的表現。

Nowadays, social media have become popular platforms for the public to share photos. To apply effects on a photo or improve its quality, most social media provide filters by which users can change the appearance of their photos without domain knowledge. However, due to the growing number of filter types, it becomes a major issue for users to choose the best filter type instantly. For this purpose, learning image aesthetics takes an important role in image quality ranking problems. In these years, several research has proved that Convolutional Neural Networks (CNNs) outperform traditional methods in image aesthetic categorization, which classifies images into high or low quality. In this paper, we introduce a new method for image quality learning and a dataset of filtered images with comparison. Instead of binarizing image quality, we use different CNN architectures and a pairwise comparison loss function to learn the aesthetic response for an image. By utilizing pairwise image comparison, the models embed aesthetic responses in the hidden layers. Moreover, to improve the aesthetic ranking, the image category is integreated into the aesthetic-oriented models. To train our models and evaluate our method, we introduce a new dataset called Filter Aesthetic Comparison Dataset (FACD). The dataset contains more than 30,000 filtered images based on the AVA dataset and more than 40,000 image pairs with quality comparison annotations using Amazon Mechanical Turk. To our best knowledge, it is the first dataset containing filtered images and the user preference labels. The experimental results show that our method which learns aesthetic ranking by pairwise comparison outperforms the traditional aesthetic classification methods.