• Beyond Close Reading: An Empirical Approach for Annotation and Classification of Multimodal Texts

    Author(s):
    Kenzie Burchell, Asen Ivanov (see profile)
    Date:
    2020
    Group(s):
    CSDH-SCHN 2020
    Subject(s):
    Archives, Corpora (Linguistics), Mass media and war
    Item Type:
    Conference paper
    Conf. Title:
    CSDH/SCHN Digital Humanities Conference 2020
    Conf. Org.:
    CSDH/SCHN
    Conf. Loc.:
    Congress 2020, virtual
    Conf. Date:
    June 1-5, 2020
    Tag(s):
    Corpus linguistics, Images, Media and conflict, Multimodality
    Permanent URL:
    http://dx.doi.org/10.17613/gcx1-zx65
    Abstract:
    While a range of approaches and techniques for linguistic annotation and classification are currently available, they have not been designed to handle multimodal texts with a pronounced visual dimension such as posters, webpages, or moving images (i.e., film, TV). In this paper, we present an approach for annotation and classification of multimodal texts that could facilitate their computational analysis. The approach builds upon prior work on multimodal discourse analysis and social semiotics that is unified by its common roots in the tenets of systemic functional linguistics (SFL). By combing insights from this work, the approach we present provides techniques for annotating and classifying key communicative elements commonly found in multimodal texts including (1) layout and composition, (2) image motif and aesthetics, (3) image-text relations, (3) navigation, (4) visual rhythm, and (5) visual sequencing. To illustrate the approach, we discuss the key ideas informing its design—namely, ideas developed within the framework of the Genre and Multimodality (GeM) model (Bateman, 2008), and its subsequent elaboration in the study of online political communication (Seizov, 2014). We also describe the context of application within which we developed the approach—i.e., to systematically classify and analyze the communicative elements of news coverage of the Syrian war. We then present an overview of the data classification procedures the approach presupposes and the classification schema and data dictionary we have developed to aid annotation and classification.
    Notes:
    Please find attached PP slides and transcript (in the notes). You can view a recording of our presentation at https://youtu.be/MXLmSV9Fqj0 I have recorded an additional video to go along with the presentation at https://youtu.be/oZ-D0OoYPlc The reason for the second video is that I had a tech hiccup. I explain that in the video. I’m looking forward to the conference! best, Asen
    Metadata:
    xml
    Status:
    Published
    Last Updated:
    4 years ago
    License:
    Attribution-ShareAlike
    Share this:

    Downloads

    Item Name: pptx ivanov_burchell_slides_csdh_2020.pptx
      Download
    Activity: Downloads: 73

    Back to Deposits