基於使用者生成多媒體內容之巨量資料分析

Abstract

數量持續成長的社群媒體用戶基於共享和社交的目的，貢獻大量人物照片。這些人物多媒體資料（如旅行照片和家人影片等）保有豐富的人群活動資料，對行動推薦系統，個人化，廣告和更多以人為中心的應用非常有利。有鑒於這些強烈需求，我們提出利用影像中自動偵測獲得的人物訊息（如人臉屬性，人物群體類別，視覺情感概念）來幫助社交多媒體分析。本計畫進一步結合計算社會學和認知心理學來了解社群使用者所提供的視覺資料中所挖掘出的知識訊息。最後，我們並展示利用百萬規模的社群影像及其周邊資訊（如地理位置，時間，標籤和評論）來幫住人物特徵分析，人口統計調查和社群情感運算。就我們所知，這是第一個利用大規模社交視覺資料來幫助分析使用者行為的研究工作。

A growing population of the Internet users are contributing a huge amount of photos and videos to social media for the purpose of sharing and social communication. These big human-centric media collections such as travel photos and family videos retain abundant people activities inherently beneficial for mobile recommender system, personalization, advertisement and more people-related applications. Witnessing these strong needs, we propose to exploit the human-centric contexts automatically detected from visual content, e.g., people attributes, social group types and visual concepts, for social multimedia analytics. The proposed approach further incorporates computational sociology and cognitive psychology to understand the knowledge mined from the visual content contributed by real users. Finally, we demonstrate its effectiveness for user profiling, demographic investigation and social affective computing by using million-scale social images and the associated metadata (i.e., geo-locations, time stamps, tags and comments) crawled from social media. To the best of our knowledge, this is the first work addressing how large-scale visual contexts can help user profiling and improve user behavior analysis.