Language processing does not happen in isolation, but is often embedded in a rich

non-linguistic visual and social context. Yet, although many psycholinguistic studies

have investigated the close interplay between language and the visual context, the role

of social aspects and listener characteristics in real-time language processing remains

largely elusive. The present thesis aims at closing this gap.<br />

Taking extant literature regarding the incrementality of language processing, the

close interplay between visual and linguistic context and the relevance for and effect

of social aspects on language comprehension into account, we argue for the necessity

to extend investigations on the influence of social information and listener

characteristics on real-time language processing. Crucially, we moreover argue for the

inclusion of social information and listener characteristics into real-time language

processing accounts. Up-to-date, extant accounts on language comprehension remain

elusive about the influence of social cues and listener characteristics on real-time

language processing. Yet a more comprehensive approach that takes these aspects

into account is highly desirable given that psycholinguistics aims at describing how

language processing happens in real-time in the mind of the comprehender.

In 6 eye-tracking studies, this thesis hence investigated the effect of two distinct

visual contextual cues on real-time language processing and thematic role assignment

in emotionally valenced non-canonical German sentences. We are using emotional

facial expressions of a speaker as a visual social cue and depicted actions as a visual

contextual cue that is directly mediated by the linguistic input. Crucially, we are also

investigating the effect of the age of the listener as one type of listener characteristics

in testing children and older and younger adults.<br />

In our studies, participants were primed with a positive emotional facial

expression (vs. a non-emotional / negative expression). Following this they inspected

a target scene depicting two potential agents either performing or not performing an

action towards a patient. This scene was accompanied by a related positively valenced

German Object-Verb-Adverb-Subject sentence (e.g.,: The ladybug(accusative

object, patient) tickles happily the cat(nominative object, agent).). Anticipatory eye-movements to

the agent of the action, i.e., the sentential subject in sentence end position (vs.

distractor agent), were measured in order to investigate if, to what extent and how

rapidly positive emotional facial expressions and depicted actions

can facilitate thematic role assignment in children and older and younger adults.

Moreover, given the complex nature of emotional facial expressions, we also

investigated if the naturalness of the emotional face has an influence on the

integration of this social cue into real-time sentence processing. We hence used a

schematic depiction of an emotional face, i.e., a happy smiley, in half of the studies

and a natural human emotional face in the remaining studies.<br />

Our results showed that all age groups could reliably use the depicted actions as a

cue to facilitate sentence processing and to assign thematic roles even before the

target agent had been mentioned. Crucially, only our adult listener groups could also

use the emotional facial expression for real-time sentence processing. When the

natural human facial expression instead of the schematic smiley was used to portray

the positive emotion, the use of the social cue was even stronger. Nevertheless, our

results have also suggested that the depicted action is a stronger cue than the social

cue, i.e., the emotional facial expression, for both adult age groups. Children on the

other hand do not yet seem to be able to also use emotional facial expressions as

visual social cues for language comprehension. Interestingly, we also found time

course differences regarding the integration of the two cues into real-time sentence

comprehension. Compared to younger adults, both older adults and children were

delayed by one word region in their visual cue effects.<br />

Our on-line data is further supported by accuracy results. All age groups answered

comprehension questions for ‘who is doing what to whom’ more accurately when an

action was depicted (vs. was not depicted). However, only younger adults made use

of the emotional cue for answering the comprehension questions, although to a lesser

extent than they used depicted actions.<br />

In conclusion, our findings suggest for the first time that different non-linguistic

cues, i.e., more direct referential cues such as depicted actions and more indirect

social cues such as emotional facial expressions, are integrated into situated language

processing to different degrees. Crucially, the time course and strength of the

integration of these cues varies as a function of age.<br />

Hence our findings support our argument regarding the inclusion of social cues

and listener characteristics into real-time language processing accounts. Based on our

own results we have therefore outlined at the end of this thesis, how an account of

real-time language comprehension that already takes the influence of visual context

such as depicted actions into account (but fails to include social aspects and listener

characteristics) can be enriched to also include the effects of emotional facial

expressions and listener characteristics such as age.