Story Apps as Didactic Multimodal Resources for Multiliteracies Practices

A Case in Point

Authors

  • Íris Susana Pires Pereira Centro de Investigação em Educação (CIEd), Universidade do Minho, Braga / Portugal https://orcid.org/0000-0003-0647-2319
  • Maitê Moraes Gil Centro de Investigação para o Desenvolvimento Humano (CEDH) | Centro de Estudos Filosóficos e Humanístico (CEFH), Faculdade de Educação e Psicologia, Universidade Católica Portuguesa, Porto / Portugal https://orcid.org/0000-0002-2814-2540
  • Cristina Maria dos Santos Moreira da Silva Sylla Centro de Investigação em Estudos da Criança (CIEC), Interactive Technologies Institute (ITI/LARSyS), Universidade do Minho, Braga / Portugal ##default.groups.name.translator## https://orcid.org/0000-0003-2159-7566

DOI:

https://doi.org/10.17851/2237-2083.31.4.1916-1951

Keywords:

multiliteracies, multimodality, story app, didactic resources, agency

Abstract

This article aims to contribute to the understanding of the use of interactive story apps as didactic resources for fostering the learning and teaching of multiliteracies. It presents the multimodal discourse analysis of an interactive story app showing a dense net of intermodal meaning relations, such as convergence, complementarity, and divergence, in the representation of ideational and interpersonal meanings. The results allow the characterization of the story app as a complex multimodal ensemble and point to the need of a renewed readers and teacher’s agency in the pedagogical use of interactive story apps in the learning and teaching of multiliteracies. The article finishes by identifying the limitations of the study and some future developments.

References

AL-YAQOUT, G.; NIKOLAJVA, M. Re-conceptualising picturebook theory in the digital age. Barnelitterært Forskningstidsskrift, Oslo, v. 6, n. 1, p. 1-7, 2015. DOI: https://doi.org/10.3402/blft.v6.26971

BAL, M. Narratology. Introduction to the Theory of Narrative. 4. ed. London: University of Toronto Press, 2017.

BEZEMER, J.; KRESS, G. Changing text: A Social Semiotic Analysis of Textbooks. Designs for Learning, Stockholm, v. 3, n.1-2, p. 10-29, 2010. DOI: https://doi.org/10.16993/dfl.26

BEZEMER, J.; KRESS, G. The Textbook in a Changing Multimodal Landscape. In: KLUG, N.; STÖCKL, H. (eds.). Language in Multimodal Contexts. New York/Berlin: De Gruyter, 2016a. p. 476-498.

BEZEMER, J.; KRESS, G. Multimodality, learning and communication. A social semiotic frame. New York: Routledge, 2016b.

BUS, A.; TAKACS, Z.; KEGEL, C. Affordances and limitations of electronic storybooks for young children’s emergent literacy. Developmental Review, Fribourg, v. 35, p. 79-97, 2015. DOI: https://doi.org/10.1016/j.dr.2014.12.004

COPE, B.; KALANTZIS, M. Multiliteracies: New Literacies, New Learning. Pedagogies, Singapore, v. 4, n. 3, p. 164-195, 2009. DOI: http://doi.org/10.1080/15544800903076044

GEE, J. P. A personal retrospective on the New London Group and its formation. In: SERAFINI, F.; GEE, E. (eds.). Remixing multiliteracies. Theory and practice from New London to New times. New York: Teachers College Press, 2017. p. 19-30.

GIL, M.; PEREIRA, Í. S.; SYLLA, C. La construcción del posicionamiento lector en una narrativa digital interactiva. Una discusión a la luz de la teoría de las multiliteracidades. Cultura, Lenguaje Y Representación, Castellón de la Plana, v. 30, p. 79-106, 2023. DOI: https://doi.org/10.6035/clr.6773

GUALBERTO, C. L.; SANTOS, Z. B. Multimodalidade no contexto brasileiro: um estado de arte. DELTA: Documentação de Estudos em Linguística Teórica e Aplicada, São Paulo, v. 35, n. 2, p. 1-30, 2019. DOI: https://doi.org/10.1590/1678-460X2019350205

HAGEN, A. The Potential for Aesthetic Experience in a Literary App. An analysis of The Fantastic Flying Books of Mr. Morris Lessmore. Barnelitterært forskningstidsskrift, v. 11, n. 1, p. 1-10, 2020. DOI: https://doi.org/10.18261/issn.2000-7493-2020-01-02

HALLIDAY, M.; MATTHIESSEN, C. An Introduction to Functional Grammar. London: Routledge, 2004.

JEWITT, C. Multimodal, “Reading”, and “Writing” for the 21st Century. Discourse: studies in the cultural politics of education, London, v. 26, n. 3, p. 315–332, 2005. DOI: https://doi.org/10.1080/01596300500200011

JEWITT, C. Multimodality and Literacy in School Classrooms. Review of Research in Education, Washington, v. 32, n. 1, p. 241-267, 2008. DOI: https://doi.org/10.3102/0091732X07310586

KALANTZIS, M; COPE, B. New learning. Elements of a science of education. 2. ed. Cambridge: Cambridge University Press, 2012.

KRESS, G. Multimodality. A social semiotic approach to Contemporary Communication. New York: Routledge, 2010.

KRESS, G.; VAN LEEUWEN, T. Reading images. The grammar of visual design. 2. ed. New York: Routledge, 2006.

LOW, D. E.; PANDYA, J. Z. Issues of Validity, Subjectivity, and Reflexivity in Multimodal Literacy Research and Analysis. Journal of Language and Literacy Education, Athens, v. 15, n. 1, p. 1-22, 2019.

MARTIN, J. R.; ROSE, D. Working with Discourse. Meaning Beyond Clause. London and New York: Bloomsbury, 2007.

MENEGAZZI D.; SYLLA C. Touch to Read: Investigating the Readers’ Interaction Experience in Mediated Reading to Design Story Apps. In: BROOKS A., BROOKS E. (eds). Interactivity, Game Creation, Design, Learning, and Innovation. ArtsIT 2019, DLI 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Springer, 2020. p. 588-600. 328 v. DOI: https://doi.org/10.1007/978-3-030-53294-9_43

NEW LONDON GROUP. A Pedagogy of Multiliteracies: Designing Social Futures. Harvard Educational Review, v. 66, n. 1, p. 60-92, 1996.

NEW LONDON GROUP. A pedagogy of multiliteracies. In: COPE, B; KALANTZIS, M. (eds.), Multiliteracies. London: Routledge, 2000. p.19-37.

O’BRIEN, A. Using Focalisation Choices to Manipulate Audience Viewpoint in 3-D Animation Narratives: What Do Student Authors Need to Know? In: UNSWORTH, L.; THOMAS, A. (eds). English Teaching and New Literacies Pedagogy. Interpreting and Authoring Digital Multimodal Narratives. New York: Peter Lang Gmbh, 2014. p. 123-150.

PAINTER, C.; MARTIN, J. R.; UNSWORTH, L. Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox, 2012.

PEREIRA, Í. S. P. Para uma pedagogia da leitura realizada on-line para construir conhecimento. Texto Digital, v. 15, n. 2, p. 28-56, 2019. DOI: https://doi.org/10.5007/1807-9288.2019v15n2p28

PEREIRA, Í.; GIL, M.; SYLLA, C. Para a construção da Teoria dos Multiletramentos. Dimensões ideacionais (e implicações pedagógicas) da multimodalidade textual numa StoryApp para crianças. Calidoscopio, v. 20, n. 1, p. 65-89, 2022. https://revistas.unisinos.br/index.php/calidoscopio/article/view/24438

PEREIRA, Í. S. P.; GIL, M.; SYLLA, C. Meeting the challenges of designing multimodal texts to enact the multiliteracies pedagogy = Enfrentando os Desafios do Design de Textos Multimodais Para Promover a Pedagogia das Multiliteracias. Comunicação e Sociedade, v. 43, p. 1-31, 2023. DOI: https://doi.org/10.17231/comsoc.43(2023).4481

RESNICK, M.; MARTIN, F.; BERG, R.; BOROVOY, R.; COLELLA, V.; KRAMER, K.; SILVERMAN, B. Digital Manipulatives: New Toys to Think With. In: CHI ‘98. Human Factors in Computer Systems, 1998, Los Angeles. Conference Proceedings. Steven Pemberton, 1998. p. 281-287.

ROWSELL, J.; KRESS, G.: PAHL, K.; STREET, B. The Social Practice of Multimodal Reading: A New Literacy Studies–Multimodal Perspective on Reading. In: ALVERMANN, D. E.; UNRAU, N. J.; RUDDEL, R. B. (eds.). Theoretical Models and Processes of Reading. Newark: International Reading Association, 2013. p. 1182–1207. 978 v.

STAKE, R. Case Studies. In: DENZIN, N.; LINCOLN, Y. (eds.). Handbook on qualitative Studies. 2 ed. London: Sage Publications Inc., 2000. p. 435-454.

STEIN, P. Multimodal Pedagogies in Diverse Classrooms. Representations, rights and resources. London: Routledge, 2008.

SYLLA, C.; GIL, M.; PEREIRA, Í. Untangling the complexity of designing tools to support tangible and digital intercultural storytelling in troubled times: a case in point. Literacy, London, v. 56, n. 1, p. 3-17, 2022. DOI: https://doi.org/10.1111/lit.12263

TAKACS, Z.; BUS, A. Benefits of Motion in Animated Storybooks for Children’s Visual Attention and Story Comprehension. An Eye-Tracking Study. Frontiers in Psychology, London, v. 7, p. 1-12, 2016. DOI: https://doi.org/10.3389/fpsyg.2016.01591

TAKACS, Z.; SWART, E.; BUS, A. Benefits and Pitfalls of Multimedia and Interactive Features in Technology-Enhanced Storybooks: A Meta-Analysis. Review of Educational Research, London, v. 85, n. 4, p. 698-739, 2015. DOI: https://doi.org/10.3102/0034654314566989

UNSWORTH, L. Towards a Metalanguage for Multiliteracies Education: Describing the Meaning-Making Resources of Language-Image Interaction. English Teaching: Practice and Critique, Leeds, v. 5, n. 1, p. 55-76, 2006.

WALSH, M. The ‘textual shift’: Examining the reading process with print, visual and multimodal texts. Australian Journal of Language & Literacy, Crows Nest, v. 29, n. 1, p. 24-37, 2006.

WALSH, M. Worlds have collided and modes have merged: Classroom evidence of changed literacy practices. Literacy, London, v. 42, n. 2, p. 101-108, 2008. DOI: https://doi.org/10.1111/j.1741-4369.2008.00495.x

YOKOTA, J.; TEALE, W. Picture Books and the Digital World. Reading Teacher, Newark, v. 67, n. 8, p. 577-585, 2014. DOI: https://doi.org/10.1002/trtr.1262

Downloads

Published

2024-10-11

Issue

Vol. 31 No. 4 (2023): Multimodality through the lenses of Kress’ Social Semiotics

Section

Thematic issue 31:4 (2023): Multimodality through the lenses of Kress’ Social Semiotics

How to Cite

Story Apps as Didactic Multimodal Resources for Multiliteracies Practices: A Case in Point. Revista de Estudos da Linguagem, [S. l.], v. 31, n. 4, p. 1916–1951, 2024. DOI: 10.17851/2237-2083.31.4.1916-1951. Disponível em: https://periodicos.ufmg.br/index.php/relin/article/view/55254. Acesso em: 26 dec. 2024.