藉由視覺注意力來處理視頻摘要

Abstract

視頻摘要主要是從一部影片藉由挑選出真正重要的片段來縮短影片長度，到目前為止，視頻摘要仍然是一項在電腦視覺領域中值得研究的題目。在本篇論文中，我們提出了一個新的架構試圖去解決包含各式內容的影片。我們提出的多樣化專注層面的視頻摘要模型。

Video summarization is among challenging tasks in computer vision, which aims at identifying highlight frames or shots over lengthy video inputs. In this paper, we propose an attention-based model for video summarization and to handle complex video data. A novel deep learning the framework of multi-head multi-layer video self-attention (M2VSA) is presented to identify informative regions across spatial and temporal video features, which jointly exploit context diversity over space and time for summarization purposes. Together with visual concept consistency enforced in our framework, both video recovery and summarization can be preserved. More importantly, our developed model can be realized in both supervised/unsupervised settings. Finally, our experiments quantitative and qualitative results demonstrate the effectiveness of our model and our superiority over state-of-the-art approaches.