請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/62908
標題: | 一大腦啟發和視覺皮層感知之互動立體視訊處理系統 A Brain-Inspired and Visual-Cortex-Aware Interactive 3D Video Processing System |
作者: | Chung-Te Li 李宗德 |
指導教授: | 陳良基(Liang-Gee Chen) |
關鍵字: | 大腦啟發,視覺皮層感知,互動,立體視訊處理, Brain-Inspired,Visual-Cortex-Aware,Interactive 3D Video, |
出版年 : | 2013 |
學位: | 博士 |
摘要: | 本論文描述了一大腦啟發和視覺皮層感知的立體視訊處理系統。與目前的智慧3D電視(Smart 3DTV)相比,本文提出的系統強調立體視覺品質的提高和自然智慧立體互動。我們將此系統稱為超智慧3D電視(Ultra Smart 3DTV)。 本系統通過分析人類視覺系統感知來增強立體視覺品質。根據心理學家們的研究,人類具有透過各種單眼和雙眼深度線索產生立體視覺的能力。在觀看3D立體影像的同時,深度線索之間的衝突,讓觀眾感到不舒服或不自然。然而,缺乏3D立體內容,是一個現有立體電視系統的最主要問題。因此,本文提出了一個無深度線索衝突2D影像到3D立體影像之轉換,以產生視覺品質更好的3D影像。我們試圖通過模仿大腦如何分析深度,來計算影像中各物體的深度,並透過模仿大腦的機制,在轉換中,避免因深度線索之衝突而造成立體影像品質不佳。由於科學家已經發現了人類大腦處理的深度線索,可以用貝氏機率模型來解釋,我們將2D影像到3D立體影像的轉換問題,化歸為深度線索間之貝氏分析,從而可得到最佳化的深度估計,而提供更好的立體影像品質。比起傳統的轉換方法,我們的方法不論在主觀上或是客觀上都有更佳的表現。
此外,我們所提出的自然智慧立體互動透過(1)偵測觀看者的意向與(2)其觀看電視時之立體感知,並考慮兩者之間的相互作用來實現。對於使用者來說,透過雙手及肢體來表達意向,是最自然的方式。因此,我們所提出的自然智慧立體互動讓使用者以雙手來表達其意向。然而,在透過雙手與電視中顯示的物體互動時,立體感不能被扭曲,這是一個好的立體互動系統的必要條件。為了避免立體感被扭曲,我們嘗試了解視覺皮層中的早期視覺,並用影像處理的演算法模擬之。值得注意的是,視網膜上的影像在早期視覺中,是很重要的。因此,在立體互動系統中,我們提出了一個基於視覺皮層中的早期視覺及視網膜影像的立體影像校正方法。而視網膜影像與觀看電視的視角,距離等等有很強的關係。因此,我們的互動系統將會針對觀看者與電視之間的幾何關系來對立體影像做相對應的補償。我們也將智慧立體互動與大腦啟發2D到3D立體影像之轉換做結合,提出兩個展示系統:互動視角立體電視及立體互動窗。根據我們的實驗結果,比起目前的智慧3D電視,本文所提出的3D立體互動展示讓使用者有較佳的使用體驗。 在本論文中,我們所提出的大腦啟發和視覺皮層感知的立體視訊處理系統,透過大腦啟發之2D-3D轉換,提供更多高視覺品質的3D內容。並透過對視覺皮層的分析,使所提出的自然智慧立體互動系統讓觀眾能享受更好的使用體驗。 This dissertation describes a brain-inspired and visual-cortex-aware interactive 3D video processing system. Compare to current smart 3DTV, the proposed system focuses on 1) the enhancement of 3D visual quality and 2) natural and smart 3D interaction. We enhance the 3D visual quality by analyzing the perception in the human visual system. Psychologists have explored that human beings perceive 3D effects by various monocular and binocular depth cues. While watching a 3D video, the conflicts between those depth cues make the viewer feel unnatural or uncomfortable. In contrast, lacking of 3D contents is a well-known fundamental problem for current 3DTV system. Therefore, we propose a depth cue conflict-free 2D-to-3D conversion to generate 3D videos with higher visual quality. To eliminate the potential conflicts between the depth cues while watching the converted 3D videos, we try to compute the depth from conventional 2D videos by mimicking how the brain analyzes the depth. Since neural scientists have found that the depth perception is generated by dealing with all the depth cues in Bayesian way, so-called “Bayesian brain”, we convert 2D videos to 3D videos by solving a Bayesian inference problem of the depth cues. We call the proposed methods as “brain-inspired 2D-to-3D conversion.” From the subjective viewpoint, the brain-inspired 2D-to-3D conversion outperforms earlier conversion methods by preserving more reliable depth cues. Moreover, an enhancement of 0.70-3.14 dB and 0.0059-0.1517 in the perceptual quality of the videos is realized in terms of the objective-modified peak signal-to-noise ratio and disparity distortion measure, respectively. Besides, the natural and smart 3D interaction is performed between the intention of the viewer and the corresponding 3D perception. In our proposed system, we assume that the viewers perform their intentions by hands, which is one of the most natural ways of the interaction. However, the 3D perception cannot be distorted during the process of the interaction. This is a fundamental and necessary condition of a smart 3D interaction system. In the proposed system, we model the early vision in the visual cortex to make sure there are no distortions in the 3D perception. Psychologists have also found that the images on the retinas are the inputs of the visual cortex; therefore, we propose an interactive 3D video retargeting method on the basis of estimating the retinal images and the responses of the early vision in the visual cortex, called as “visual-cortex-aware interactive 3D retargeting”. Notably, scientists have explored that the retinal images will be pre-processed on the basis of viewing angle in the early vision while watching television. Hence, our proposed visual-cortex-aware interactive 3D retargeting considers the pre-processing of the retinal images for preserving the intensive 3D perception. Several demonstrations of 3D interactions, including 3D viewpoint-interactive video and 3D interactive window, are also designed in this dissertation. From the subjective viewpoint, the proposed interactive 3D demonstrations are much more immersive and also preferred than current non-interactive 3D since the perceptual distortion is quite reduced. In this dissertation, we describe a brain-inspired and visual-cortex-aware interactive 3D video processing system for 3D video processing. Our proposed brain-inspire 2D-to-3D conversion provides more 3D contents with enhanced visual quality. The proposed visual-cortex-aware interactive 3D retargeting let viewers be able to enjoy their 3D experiences during the natural and smart 3D interaction. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/62908 |
全文授權: | 有償授權 |
顯示於系所單位: | 電機工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-102-1.pdf 目前未授權公開取用 | 12.6 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。