A multimodal social semiotic rethinking of agency in AI-assisted designs for language learning and assessment: Examples from Duolingo

Sophia Diamantopoulou; Sigrid Ørevik

doi:10.1590/1983-3652.2026.64269

Autores/as

Sophia Diamantopoulou University College London https://orcid.org/0000-0001-8564-8743
Sigrid Ørevik University of Bergen https://orcid.org/0000-0002-0812-4572

DOI:

https://doi.org/10.1590/1983-3652.2026.64269

Palabras clave:

Agency, Designs for learning, Multimodal social semiotics, English language learning, Artificial intelligence

Resumen

This article addresses issues of design and agency, focusing on Artificial Intelligence (AI)-generated tools for speaking practice and assessment in the field of English language learning. It uses examples from the educational technology company Duolingo as agentive designs for enabling and assessing learning. Drawing on multimodal social semiotics (MMSS) as a theoretical and methodological lens, the article analyses selected screenshots from the Duolingo examples as multimodal texts-in-action, showcasing how these AI-assisted learning designs make visible the agency of the designer and the role of AI, and how the design facilitates the user’s agency. In these designs, AI is described as enacting a form of devolved agency, given that it interacts with the user as a proxy for the human designers who shape the affordances of the AI-assisted learning design. Among the implications of the study is an identified need to make more visible the limitations and potential of MMSS theory to account for the role of AI in contexts of learning and communication.

Descargas

Los datos de descarga aún no están disponibles.

Biografía del autor/a

Sophia Diamantopoulou, University College London

Associate Professor, Institute of Education

Referencias

ABRAMS, Sandra; HANGHØJ, Torkild. Reflexive AI Literacies: Conceptualizing a Model for Learners’ Critical Engagement with Generative AI. Pedagogies: An International Journal, 2025. DOI: 10.1080/1554480X.2025.2561972.

ÅGERFALK, Pär. Artificial Intelligence as Digital Agency. European Journal of Information Systems, v. 29, n. 1, p. 1–8, 2020. DOI: 10.1080/0960085X.2020.1721947.

BENNETT, Jane. Vibrant Matter: A Political Ecology of Things. Durham, NC: Duke University Press, 2010.

BEZEMER, Jeff; KRESS, Gunther. Multimodality, Learning and Communication: A Social Semiotic Frame. London: Routledge, 2016.

BROD, Garvin; KUCIRKOVA, Natalia; SHEPHERD, Joshua; JOLLES, Dietsje; MOLENAAR, Inge. Agency in Educational Technology: Interdisciplinary Perspectives and Implications for Learning Design. Educational Psychology Review, v. 35, 2023. DOI: 10.1007/s10648-023-09749-x.

BURRISS, Sarah; LEANDER, Kevin. Critical Posthumanist Literacy: Building Theory for Reading, Writing, and Living Ethically with Everyday Artificial Intelligence. Reading Research Quarterly, v. 59, n. 4, p. 560–569, 2024. DOI: 10.1002/rrq.565.

BURSTEIN, Jill; CARDWELL, Ramsey; CHUANG, Ping-Lin; MICHALOWSKI, Allison; NYDICK, S. Exploring AI-Enabled Test Practice, Affect, and Test Outcomes in Language Assessment. In: PROCEEDINGS of the Artificial Intelligence in Measurement and Education Conference (AIME-Con). [S. l.: s. n.], 2025.

BURSTEIN, Jill; LAFLAIR, Geoffrey; KUNNAN, Antony John; DAVIER, Alina von. A Theoretical Assessment Ecosystem for a Digital-First Assessment—The Duolingo English Test. [S. l.], 2022.

CASTILLO, L. Explain My Answer Is Now Free for All Learners. Duolingo Blog. 1 Jan. 2026. Available from: https://blog.duolingo.com/explain-my-answer-now-free/.

COPE, Bill; KALANTZIS, Mary. Artificial Intelligence in the Long View: From Mechanical Intelligence to Cyber-Social Systems. Discover Artificial Intelligence, v. 2, n. 13, 2022. DOI: 10.1007/s44163-022-00029-1.

COPE, Bill; KALANTZIS, Mary. A Multimodal Grammar of Artificial Intelligence: Measuring the Gains and Losses in Generative AI. Multimodality & Society, v. 4, n. 2, p. 123–152, 2024. DOI: 10.1177/26349795231221699.

COPE, Bill; KALANTZIS, Mary. On Cyber-Social Learning: A Critique of Artificial Intelligence in Education. In: KORKOULOU, Despina; TZIRIDES, Orestis A.; COPE, Bill; KALANTZIS, Mary (eds.). Trust and Inclusion in AI-Mediated Education: Where Human Learning Meets Learning Machines. [S. l.]: Springer, 2024.

COPE, Bill; KALANTZIS, Mary; SEARSMITH, Duane. Artificial Intelligence for Education: Knowledge and Its Assessment in AI-Enabled Learning Ecologies. Educational Philosophy and Theory, v. 53, n. 12, p. 1229–1245, 2021. DOI: 10.1080/00131857.2020.1728732.

DAI, Yun; LAI, Sichen. A Theoretical Framework of Student Agency in AI-Assisted Learning: A Grounded Theory Approach. In: KHAN, L. (ed.). Oxford Intersections: Social Media in Society and Culture. [S. l.]: Oxford University Press, 2025.

DAVIER, Alina von; DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid; BERGER, Cindy. Designing for Assessment in Multimodal English Language Learning: Examples from Duolingo. Paper presented at the 12th International Conference on Multimodality (12ICOM), 29–31 October 2025. [S. l.], 2025.

DIAMANTOPOULOU, Sophia. Student Agency in the Creation of Multimodal Texts at the Ancient Agora of Athens: A Political Project. In: STARC, Susanne; KOMNINOS, Nicos (eds.). Multimodal Literacy in Education: Perspectives from Global Practices. [S. l.]: Peter Lang, 2024. p. 63–94.

DIAMANTOPOULOU, Sophia; FLORÉN, Henrika. Social Semiotics and Multimodality. In: INTERNATIONAL Encyclopedia of Language and Linguistics. 3. ed. [S. l.: s. n.], 2026. p. 254–259. DOI: 10.1016/B978-0-323-95504-1.00539-1.

DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid (eds.). Multimodality in English Language Learning. [S. l.]: Routledge, 2022.

DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid. The Concept of Design in the Field of Multimodal English Language Learning: Recognizing the Distribution of Agency in Semiotic Work. In: PROCEEDINGS of the 8th Designs for Learning Conference. [S. l.: s. n.], 2024. Paper presented at the 8th Designs for Learning Conference: Conceptualizing Design in Research and Development Work, Stockholm University, Stockholm,

Sweden, August 28–30.

DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid. Multimodality and English Language Learning (MELL). In: CHAPELLE, Carol A. (ed.). Encyclopedia of Applied Linguistics. 2. ed. [S. l.]: Wiley, 2025. DOI: 10.1002/9781405198431.wbeal20822.

DONDERO, Maria Giulia; ALONSO, Juan; LEONE, Massimo. Aspects of AI Semiotics: Enunciation, Agency, and Creativity. Semiotica, n. 262, p. 1–3, 2025. DOI: 10.1515/sem-2025-0014.

DUOLINGO. Practice Speaking with Video Call. Video, 33 s. YouTube. 2024. Available from: https://www.youtube.com/watch?v=vBjfn2Pb6DI. Visited on: 15 June 2026.

DUOLINGO. Duolingo English Test: Test Format. Duolingo English Test. 2025. Available from: https://englishtest.duolingo.com/prepare/format. Visited on: 15 June 2026.

DUOLINGO. Interactive Speaking Practice Hub. Duolingo English Test. 2025. Available from: https://englishtest.duolingo.com/test/practiceHub/interactive-speaking?isReview=true. Visited on: 15 June 2026.

DUQUE-PEREIRA, Ives da Silva; MOURA, Sergio de. AI Chatbots and the Simulation of Dialog: What Does Bakhtinian Theory Have to Say? SciELO in Perspective. 30 Apr. 2025. Available from: https://blog.scielo.org/en/2025/04/30/ai-chatbots-and-the-simulation-of-dialog-what-does-bakhtinian-theory-have-to-say/.

GARZÓN, Juan; PATIÑO, Eddy; MARULANDA, Camilo. Systematic Review of Artificial Intelligence in Education: Trends, Benefits, and Challenges. Multimodal Technologies and Interaction, v. 9, n. 8, p. 84, 2025. DOI: 10.3390/mti9080084.

GELL, Alfred. Art and Agency: An Anthropological Theory. Oxford: Clarendon Press, 1998.

HENRY, Parker. Get to Know the AI Behind Every Video Call with Lily. Duolingo Blog. 22 Apr. 2025. Available from: https://blog.duolingo.com/ai-and-video-call/.

JEWITT, Carey; ADAMI, Elisabetta; ARCHER, Arlene. Editorial. Multimodality & Society, v. 1, n. 1, p. 3–7, 2021. DOI: 10.1177/2634979521992902.

JEWITT, Carey; HENRIKSEN, Berit. Social Semiotic Multimodality. In: O’HALLORAN, Kay L.; TAN, Sabine (eds.). Social Semiotics and Socially Situated Sign Making: Essays in Honour of Theo van Leeuwen. [S. l.]: De Gruyter Mouton, 2016.

JIANG, Lianjiang; LAI, Chun. How Did the Generative Artificial Intelligence-Assisted Digital Multimodal Composing Process Facilitate the Production of Quality Digital Multimodal Compositions: Toward a Process-Genre Integrated Model. TESOL Quarterly, v. 59, n. 1, p. 52–85, 2025. DOI: 10.1002/tesq.3390.

KARATZA, Styliani; ADAMI, Elisabetta. Multimodal Communication with Technology in the Classroom: From TELL to TECL (Technology-Enhanced Communication Learning). In: HAMPEL, Regine; STICKLER, Ursula (eds.). The Bloomsbury Handbook of Language Learning and Technology. [S. l.]: Bloomsbury Academic, 2024. p. 98–114.

KARIMOVA, Gulnara. The Dialogic Evolution of AI-Based Products: A Polyphonic Analysis of Temporal Transformations. International Journal of Human–Computer Interaction, v. 41, n. 5, p. 3625–3638, 2025. DOI: 10.1080/10447318.2024.2338661.

KOSTROMITINA, Maria. From 2020 to 2025: A Look at the Evolution of the DET. [S. l.], 2025. Available from: https://go.duolingo.com/l/208162/2025-07-01/5rc2kc/208162/1751387004rz2mVDcq/evolutionupdate2025_063025.pdf.

KRESS, Gunther. Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge, 2010.

KRESS, Gunther; LEEUWEN, Theo van. Reading Images: The Grammar of Visual Design. 3. ed. London: Routledge, 2021.

LATOUR, Bruno. Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford University Press, 2005.

LIM, Fei Victor; GILJE, Øystein; DJONOV, Emilia. Editorial for Special Issue: Digital Multimodal Composing in the Era of Artificial Intelligence. Computers and Composition, v. 75, p. 102911, 2025. DOI: 10.1016/j.compcom.2024.102911.

MATTHEWS, Stephanie Walsh; DANESI, Marcel. AI: A Semiotic Perspective. Chinese Semiotic Studies, v. 15, n. 2, p. 199–216, 2019. DOI: 10.1515/css-2019-0012.

OPENAI. Duolingo: GPT-4 Deepens the Conversation on Duolingo. OpenAI. 2026. Available from: https://openai.com/index/duolingo/?video=806129298. Visited on: 15 June 2026.

PANSY_PIX. I Recently Got Duolingo Max for the Roleplay Option and It’s Giving Me Three Stars for Lessons I Haven’t Even Done Yet. Why Is That? Reddit. 22 Sept. 2024. Available from: https://www.reddit.com/r/duolingo/comments/1fn3c02/i_recently_got_duolingo_max_for_the_roleplay/. Visited on: 15 June 2026.

PARK, Yena; CARDWELL, Ramsay; GOODWIN, Sarah; NAISMITH, Ben; LAFLAIR, Geoffrey; YANCEY, Kevin; ZHANG, Xiaowan. Assessing Speaking on the Duolingo English Test. [S. l.], 2025. Available from: https://englishtest.duolingo.com/research.

PURBA, Siska; SILITONGA, Bertha; YANG, John. AI-Assisted Learning: A Systematic Review. Turkish Online Journal of Distance Education, v. 26, n. 4, p. 77–93, 2025. DOI: 10.17718/tojde.1591404.

SELANDER, Staffan; KRESS, Gunther. Design för lärande: Ett Multimodalt Perspektiv. Lund: Studentlitteratur, 2017.

SEO, Kyongwon; TANG, Joyce; ROLL, Ido; FELS, Sidney; YOON, Dongwook. The Impact of Artificial Intelligence on Learner–Instructor Interaction in Online Learning. International Journal of Educational Technology in Higher Education, v. 18, p. 54, 2021. DOI: 10.1186/s41239-021-00292-9.

SWANEPOEL, Danielle; CORKS, Daniel. Artificial Intelligence and Agency: Tie-Breaking in AI Decision-Making. Science and Engineering Ethics, v. 30, p. 56, 2024. DOI: 10.1007/s11948-024-00476-2.

TANG, Kok-Sing; PUTRA, Gde Buana. Generative AI as a Dialogic Partner: Enhancing Multiple Perspectives, Reasoning, and Argumentation in Science Education with Customized Chatbots. Journal of Science Education and Technology, 2025. DOI: 10.1007/s10956-025-10240-1.

TOMALIN, Marcus. Multimodal Social Semiotics and the Challenge of Artificial Intelligence. Multimodality & Society, v. 5, n. 2, p. 223–244, 2025. DOI: 10.1177/26349795251327939.

TUFEKCI, Zeynep. Algorithmic Harms Beyond Facebook and Google: Emergent Challenges of Computational Agency. Colorado Technology Law Journal, v. 13, n. 2, 2015.

WALLINGFORD, Priya. How Duolingo Call Works: Features, Benefits and Usage. Duolingo Guides. 2 Mar. 2025. Available from: https://duolingoguides.com/how-duolingo-call-works/.

WODZAK, Sophie. No More Tradeoffs: Interactive Speaking. Duolingo English Test Blog. 5 Apr. 2025. Available from: https://blog.englishtest.duolingo.com/interactive-speaking/.

ZAPPAVIGNA, Michele; DORAN, Y. J. Emoji as Interpersonal Resources in LLM Chatbot Conversations: A Social Semiotic Analysis of Tenor and Affiliation in Human–AI Interaction. Social Semiotics, 2025. DOI: 10.1080/10350330.2025.2570317.

ZENML. Structured LLM Conversations for Language Learning Video Calls. ZenML. 2025. Available from: https://www.zenml.io/llmops-database/structured-llm-conversations-for-language-learning-video-calls.