基於時序金字塔之第一人稱影像行為辨識

Abstract

在本篇論文中, 我們提出了針對於拍攝自第一人稱攝影機的影片, 進 行主角執行中的行為的辨識方法. 我們將此問題轉換為鏈狀條件隨機場 (Linear-chain Conditional Random Fields) 的序列標註問題. 在本方法中 使用高階視覺線索, 也就是畫面中物件偵測的結果, 來當做辨識特徵. 另 外也使用了時序金字塔(Temporal Pyramid) 來實現在時間軸上的多重解 析度, 並證明其可以改善現行的物件偵測結果. 另外也針對在日常生活 中常會發生的事件交錯情況, 提出在時序金字塔中找尋可能解的辦法. 最後我們利用目前最新研究提供的資料[1] 進行實驗, 得出可匹敵的結 果. 再利用自行拍攝的影片資料, 比較有無進行交錯事件搜尋的差別.

We present a simple but effective online recognition system for detecting interleaved activities of daily life (ADLs) in first-person-view videos. The two major difficulties in detecting ADLs are interleaving and variability in duration. We use temporal pyramid in our system to attack these difficulties, and this means we can use relatively simple models instead of time dependent probability ones such as Hidden semi-Markov model or nested models. The proposed solution includes the combination of conditional random fields (CRF) and an online inference algorithm, which explicitly considers multiple interleaved sequences by inferencing multi-stage activities on temporal pyramid. Although our system only uses linear chain-structured CRF model, which can be easily learned without a large amount of training data, it still recognizes complicated activity sequences. The system is evaluated on a data set provided by the work from state-of-the-art, and the result is comparable to their method. We also provide some experiment result using a customized dataset.