利用文字與視覺字的共現關係幫助基於草圖的圖像搜尋

Abstract

由於觸控式螢幕裝置日趨盛行，利用手繪草圖來檢索圖像已然成為一種趨勢。手繪草圖可以簡單地表達出一些複雜的使用者意圖，如物體的形狀。然而，草圖有時會因為使用者繪畫技巧的差異或不同種類物體形狀的相似而無法明確表達出使用者意圖。儘管添加文字查詢來增加語意資訊可以幫助去除草圖的模糊性，但這需要花費大量的人力和時間為所有資料庫圖片標記文字標籤。我們提出了一個利用文字（words）與視覺字（visual words）的共現（co-occurrence）關係直接針對文字和圖像的關係進行建模的方法，在所有資料庫圖片都沒有文字標籤的情況下，仍然可以加入文字查詢來輔助傳統的基於草圖的圖像搜尋（sketch-based image retrieval）改善其搜尋結果。實驗結果顯示，我們的方法確實可以幫助基於草圖的圖像搜尋得到更相關的搜尋結果，因為它確實從文字與視覺字的共現關係中學習到了其中的語意。

As the increasing popularity of touch-screen devices, retrieving images by hand-drawn sketch has become a trend. Human sketch can easily express some complex user intention such as the object shape. However, sketches are sometimes ambiguous due to different drawing styles and inter-class object shape ambiguity. Although adding text queries as semantic information can help removing the ambiguity of sketch, it requires a huge amount of efforts to annotate text tags to all database images. We propose a method directly model the relationship between text and images by the co-occurrence relationship between words and visual words, which improves traditional sketch-based image retrieval (SBIR), provides a baseline performance and obtains more relevant results in the condition that all images in database do not have any text tag. Experimental results show that our method really can help SBIR to get better retrieval result since it indeed learned semantic meaning from the ``word-visual word' (W-VW) co-occurrence relationship.