-
Beyond Close Reading: An Empirical Approach for Annotation and Classification of Multimodal Texts
- Author(s):
- Kenzie Burchell, Asen Ivanov (see profile)
- Date:
- 2020
- Group(s):
- CSDH-SCHN 2020
- Subject(s):
- Archives, Corpora (Linguistics), Mass media and war
- Item Type:
- Conference paper
- Conf. Title:
- CSDH/SCHN Digital Humanities Conference 2020
- Conf. Org.:
- CSDH/SCHN
- Conf. Loc.:
- Congress 2020, virtual
- Conf. Date:
- June 1-5, 2020
- Tag(s):
- Corpus linguistics, Images, Media and conflict, Multimodality
- Permanent URL:
- http://dx.doi.org/10.17613/gcx1-zx65
- Abstract:
- While a range of approaches and techniques for linguistic annotation and classification are currently available, they have not been designed to handle multimodal texts with a pronounced visual dimension such as posters, webpages, or moving images (i.e., film, TV). In this paper, we present an approach for annotation and classification of multimodal texts that could facilitate their computational analysis. The approach builds upon prior work on multimodal discourse analysis and social semiotics that is unified by its common roots in the tenets of systemic functional linguistics (SFL). By combing insights from this work, the approach we present provides techniques for annotating and classifying key communicative elements commonly found in multimodal texts including (1) layout and composition, (2) image motif and aesthetics, (3) image-text relations, (3) navigation, (4) visual rhythm, and (5) visual sequencing. To illustrate the approach, we discuss the key ideas informing its design—namely, ideas developed within the framework of the Genre and Multimodality (GeM) model (Bateman, 2008), and its subsequent elaboration in the study of online political communication (Seizov, 2014). We also describe the context of application within which we developed the approach—i.e., to systematically classify and analyze the communicative elements of news coverage of the Syrian war. We then present an overview of the data classification procedures the approach presupposes and the classification schema and data dictionary we have developed to aid annotation and classification.
- Notes:
- Please find attached PP slides and transcript (in the notes). You can view a recording of our presentation at https://youtu.be/MXLmSV9Fqj0 I have recorded an additional video to go along with the presentation at https://youtu.be/oZ-D0OoYPlc The reason for the second video is that I had a tech hiccup. I explain that in the video. I’m looking forward to the conference! best, Asen
- Metadata:
- xml
- Status:
- Published
- Last Updated:
- 3 years ago
- License:
- Attribution-ShareAlike
- Share this:
-
Beyond Close Reading: An Empirical Approach for Annotation and Classification of Multimodal Texts