視聽互動機制：以時間知覺為例

Abstract

本研究以時間知覺為對象，檢驗跨感官處理的兩種假說。感官適切性假說預測在視聽訊息皆呈現的情況下，時間向度的處理會以聽覺為判斷來源；最大似然率估計假說則以感官信號可靠性為權重標準，以不同信號間的線性組合使形成多重感官最佳整合為解釋方向，因此預測視覺與聽覺皆會影響時間知覺，而影響的程度依個別感官的可靠性而定，愈可靠的感官在整合時具有愈高的權重。實驗一採用一前一後出現具兩種時間長度的刺激，標準刺激感官呈現的方式為視覺或聽覺，比較刺激則為視覺、聽覺、或二者皆有，令參與者依據某一感官判斷何者的時間較長。結果顯示視覺刺激時間長度的判斷會因聽覺刺激出現而延展，但聽覺刺激時間長度則不受視覺刺激出現的影響，支持感官適切性假說。實驗二操弄標準刺激的時間長度，發現僅在中間的幾個時間長度，聽覺刺激會展延視覺刺激的時間長度，但視覺刺激對聽覺刺激的時間長度皆無影響。實驗三操弄在比較刺激與標準刺激皆共同呈現視聽刺激，發現在最長時間長度下，符合最大似然率估計假說的預測，可由視覺和聽覺刺激單獨呈現的表現來預測。實驗四操弄聽覺刺激時間長度的模糊程度（可靠性），發現聽覺刺激的權重遠大於視覺刺激的權重，而在聽覺刺激較不可靠時，才符合最大似然率估計假說的預測。實驗五增強視覺刺激的時間可用性，發現視覺刺激權重亦可能大於聽覺刺激，而以視覺為判斷來源。綜觀本研究的五個實驗結果，皆指出聽覺偏好為時間知覺的基礎，但當聽覺信號可靠性降低，或視覺信號具時間判斷的可用性時，聽覺偏好現象便產生改變，使得視覺訊息影響時間長度的判斷。因此感官適切性假說與最大似然率估計假說皆無法解釋本文的實驗結果。本文提出整合的理論模型：在時間知覺中，聽覺偏好為其先前假設，因此人類在面對多重感官刺激以判斷時間長度時，基於先前假設會使聽覺訊息的貢獻增加，而視覺訊息的貢獻則被減少。基於此先前假設整合訊息時，則以個別感官信號的可靠性產生各自感官的權重，依此權重做線性組合，產生視聽訊息整合後的時間感。

We focus on time perception to test two approaches about multisensory perception. The modality appropriateness hypothesis states that audition determines the temporal judgment of audiovisual stimuli whereas the maximum likelihood estimation (MLE) model proposes an optimal cue-combination so that visual and auditory signals are integrated based on the weighting scheme that is proportional to the estimate reliability of a signal. In Experiment 1 observers compared the duration between two intervals based on the common modality. The standard stimulus was presented in visual or auditory modality, and the comparison stimulus was presented in visual, auditory, or both modalities. The results support the modality appropriateness hypothesis in that the sound expanded the perceived visual duration whereas the disk did not affect the perceived auditory duration. In Experiment 2 the standard stimulus was set in different durations, and the sound effect on the perceived visual duration was observed only at the intermediate four durations whereas the disk did not affect the perceived auditory duration at all. In Experiment 3 when both the standard and comparison stimuli were presented bimodally, only with the longest duration the bimodal performance was predictable from unimodal performances according to the MLE model. In Experiment 4 the auditory signal was manipulated into different extents of reliability. The contribution of auditory modality to the bimodal estimate was greater than the visual modality, and the prediction of the MLE model only applied to the condition with less reliable auditory signal. In Experiment 5 a looming disk was used to make vision more temporally related. The results showed that visual modality can determine the bimodal judgment. Across five experiments an auditory bias in bimodal time perception was observed. However, with higher reliability of the auditory signal or more temporally related visual signal, the prior auditory bias declined and more contribution of visual signal came into effect. In conclusion, neither the modality appropriateness hypothesis nor the MLE model can explain our results, and a hybrid model was proposed: The auditory bias acts as a prior assumption held by observers in time perception to combine sensory signals, and the signals integrate based on the weighting scheme according to the reliability of signals.