聽覺意義訊息對視知覺和後設認知的影響

Abstract

Visual sensitivity (d’) of a picture can be enhanced crossmodally by the presentation of an auditory cue that is semantically congruent rather than incongruent. However, it remains unknown whether such crossmodal semantic congruency can modulate metacognitive sensitivity of picture processing (meta-d’, the ability to discriminate whether one’s own perceptual judgment is correct). We examined this issue by measuring the d’ and meta-d’ in a picture detection task, and their quotient (meta-d’/d’, called M-ratio) is an index of metacognitive efficiency which controls the influence of task difficulty. The auditory cue (a naturalistic sound or a spoken word) and the object picture (presented for 37 ms, sandwiched by two 13-ms masks) were either congruent (e.g., a dog barking or the spoken word “dog” paired with a dog picture) or incongruent (e.g., a piano note or the spoken word “piano” paired with a dog picture). Auditory cues were presented at -1000, -350, 0 and 500 ms stimulus onset asynchronies (SOAs, negative values indicate auditory leading cue is leading). Participants had to detect the presence of an object picture, and then to rate their confidence regarding the detection judgment. When a naturalistic sound or spoken word was presented earlier than or simultaneously with the picture, the d’ was higher in the congruent than in the incongruent condition. However, only the spoken word elicited the semantic congruency effects on d’ even at 500 ms SOA. Spoken words also induced semantic congruency effects on meta-d’ across all the SOAs. By contrast, naturalistic sounds only elicit semantic congruency effects on meta-d’ at the -350 ms SOA. Interestingly, the metacognitive efficiency was higher in the congruent than in the incongruent condition only when spoken words were presented simultaneously or later than the picture. Hence, hearing a semantically-congruent (as compared to incongruent) auditory cue can facilitate not only visual perception but also metacognition. Moreover, spoken words elicit more pronounced semantic congruency effects on both visual perception and metacognition. We provide a unified framework to explain the different congruency effects of spoken words and naturalistic sounds.