電商商品圖像與文本的解耦表征之學習

Abstract

深度學習擅於生成分佈式表征，然而這類表征通常不易於被解釋。 解耦式表征(Disentangled Representation)是一個新近被討論的概念， 擁有模塊性、緊湊性和明晰性等特性，而且可以通過生成因素來解 讀。本篇論文將利用來自於電商商品的平行化文本與圖像資料，訓練 一個可以將分佈式商品標題表征轉換至解耦式表征的模型，並使轉後 的表征可以被拆解為兩個模塊，其中一個模塊用於編碼商品圖像和標 題共同傳遞的訊息，而另一個模塊則用於編碼無法從商品圖像，只 能從商品標題中知曉或推斷的訊息。我們通過引入變分式隨機忽略 (Variational Dropout) 的方法來達成我們的目標，該方法同時也可以為 我們提供從資料中學習得到的有價值的隨機忽略機率 (Dropout Rate)。 實驗和評估的結果表明，轉換后的解耦式表征更善於判讀不同商品標 題之間的相似度，而且不同的模塊在評估過程中展現了相對不同的特 性，這也有利於發展更多的應用。我們也初步驗證了轉換后的表征能 夠基本滿足解耦式表征所需要的特性。

Deep learning is good at generating distributed representaions, but they cannot be well interpreted. While disentangled representation is a recently discussed concept that features modularity, compactness and explicitness, which is explainable via the generating factors. This thesis makes use of aligned text and image data of E-commerce products to learn a model that can transfrom a product title representation to a disentangled one, which can be divided into two modules, one of them encodes the information that is commonly conveyed by the title and the image, while the other encodes the rest information cannot inferred from the image but only known from the title. We achieve our goal by injecting variational dropout, which also provides us meaningful dropout rates learned from the data. The experiment and evaluation results show that the transformed disentangled representations are good at calculating the similarity between different product titles, meanwhile, different sections of the representation show different patterns when doing the evaluation tasks, which might be useful for more applications. We also show that the properties of disentanlgement can be basically satisfied by our learning methods.