Localizing, describing, interpreting: effects of different audio text structures on attributing meaning to digital pictures

Glaser, M (通讯作者),Leibniz Inst Wissensmedien, Schleichstr 6, D-72076 Tubingen, Germany.;Glaser, M (通讯作者),Bruckenschlegel 7, D-70186 Stuttgart, Germany.
2022-10
Based on previous research on multimedia learning and text comprehension, an eye-tracking study was conducted to examine the influence of audio text coherence on visual attention and memory in a multimedia learning situation with a focus on picture comprehension. Audio text coherence was manipulated by the type of LDI structure, that is, whether localization, description, and interpretation followed in immediate succession for each pictorial detail or whether localizations and description of details were separated from their interpretation. Results show that with a LDI integrated structure compared to a LDI separated structure the referred-to picture elements were fixated longer during interpretation parts, and linkages between descriptions and interpretations were better recalled and recognized. The effects on recall and recognition of linkages were fully mediated by fixation times. This pattern of results can be explained by an interplay between audio text coherence and dual coding processes. It points out the importance of local coherence and the provision of localization information in audio explanations as well as visual attention to allow for dual coding processes that can be used to better attribute meaning to picture details. Practical implications for the design of educational videos, audio texts on websites, and audio guides are discussed.
INSTRUCTIONAL SCIENCE
卷号:50|期号:5|页码:729-748
ISSN:0020-4277|收录类别:SSCI
语种
英语
来源机构
Leibniz Institut fur Wissensmedien
资助信息
This work was supported by the German Research Foundation [grant numbers 638221]. The German Research Foundation was not involved in the study design, in the collection, analysis and interpretation of data, in the writing of the report, or in the decision to submit the article for publication. Open Access funding enabled and organized by Projekt DEAL.
被引频次(WOS)
0
被引频次(其他)
0
180天使用计数
7
2013以来使用计数
9
EISSN
1573-1952
出版年
2022-10
DOI
10.1007/s11251-022-09593-6
学科领域
循证教育学
关键词
Audio text coherence Multimedia Dual coding Picture comprehension Visual attention
资助机构
German Research Foundation(German Research Foundation (DFG)) Projekt DEAL
WOS学科分类
Education & Educational Research Psychology, Educational