觀眾留言應用於直播影片精彩擷取之深度學習模型

Abstract

現今直播已經成為人們學習新事物時隨手可得的管道之一。儘管直播影片吸引了大量的觀眾，但在長時間的影片內容中，只包含少部分的精彩片段，其他大部分時間則為相對無趣的內容。基於此現象，大量的人工智慧研究者開始開發深度學習模型來自動擷取直播影片的精彩片段。相關研究中，大部分都是基於影像畫面的視覺分析來擷取精彩片段，鮮少有研究使用觀眾留言。因此，在這篇論文中，我們提出一個新的深度學習模型來分析觀眾留言內容，藉由觀眾留言傳達出的訊息來擷取影片中的精彩片段。我們將模型應用在收集自數個不同Twitch直播頻道的影片庫上來評估模型表現，最終得到模型的精確度為51.3%，優於我們所比較的基線模型。

Live streaming has become a ubiquitous channel for people to learn new happenings. Although live streaming videos generally attract a large audience of watchers, their contents are long and contain relatively unexciting stretches of knowledge transmission. This observation has prompted artificial intelligence researchers to establish advanced models that automatically extract highlights from live streaming videos. Most streaming highlight extraction research has been based on visual analysis of video frames, and seldom have studies considered the messages posted by the viewer-audience. In this paper, we propose a deep learning model that examines the messages posted by streaming audiences. The video segments whose messages reveal audience excitement are extracted to compose the highlights of a streaming video. We evaluate our model in terms of multiple Twitch streaming channels. The precision of our highlight extraction model is 51.3% and is superior to several baseline methods.