A multimodal social semiotic rethinking of agency in AI-assisted designs for language learning and assessment: Examples from Duolingo

Sophia Diamantopoulou; Sigrid Ørevik

doi:10.1590/1983-3652.2026.64269

Autores

Sophia Diamantopoulou University College London, Institute of Education, London, United Kingdom https://orcid.org/0000-0001-8564-8743
Sigrid Ørevik University of Bergen, Department of Foreign Languages, Bergen, Norway https://orcid.org/0000-0002-0812-4572

DOI:

https://doi.org/10.1590/1983-3652.2026.64269

Palavras-chave:

Agência, Designs para aprendizagem, Semiótica social multimodal, Aprendizagem de língua inglesa, Inteligência artificial

Resumo

Este artigo aborda questões de design e agência, com foco em ferramentas geradas por Inteligência Artificial (IA) para a prática oral e sua avaliação no campo da aprendizagem de língua inglesa. Utilizam-se exemplos da empresa de tecnologia educacional Duolingo como designs agentivos para facilitar e avaliar a aprendizagem. Tendo como base teórica e metodológica a semiótica social multimodal (multimodal social semiotics – MMSS), o artigo analisa capturas de tela selecionadas dos exemplos do Duolingo como textos multimodais em ação, mostrando como esses designs de aprendizagem assistidos por IA tornam visível a agência do designer e o papel da IA, e como o design facilita a agência do usuário. Nesses designs, a IA é descrita como desempenhando uma forma de agência delegada, uma vez que interage com o usuário como uma representante dos designers humanos, os quais moldam as capacidades (affordances) do design de aprendizagem assistido por IA. Entre as implicações do estudo, identifica-se a necessidade de tornar mais visíveis as limitações e o potencial da teoria MMSS para explicar o papel da IA em contextos de aprendizagem e comunicação.

Downloads

Os dados de download ainda não estão disponíveis.

Biografia do Autor

Sophia Diamantopoulou, University College London, Institute of Education, London, United Kingdom

Associate Professor, Institute of Education

Referências

ABRAMS, Sandra; HANGHØJ, Torkild. Reflexive AI Literacies: Conceptualizing a Model for Learners’ Critical Engagement with Generative AI. Pedagogies: An International Journal, 2025. DOI: 10.1080/1554480X.2025.2561972.

ÅGERFALK, Pär. Artificial Intelligence as Digital Agency. European Journal of Information Systems, v. 29, n. 1, p. 1–8, 2020. DOI: 10.1080/0960085X.2020.1721947.

BENNETT, Jane. Vibrant Matter: A Political Ecology of Things. Durham, NC: Duke University Press, 2010.

BEZEMER, Jeff; KRESS, Gunther. Multimodality, Learning and Communication: A Social Semiotic Frame. London: Routledge, 2016.

BROD, Garvin; KUCIRKOVA, Natalia; SHEPHERD, Joshua; JOLLES, Dietsje; MOLENAAR, Inge. Agency in Educational Technology: Interdisciplinary Perspectives and Implications for Learning Design. Educational Psychology Review, v. 35, 2023. DOI: 10.1007/s10648-023-09749-x.

BURRISS, Sarah; LEANDER, Kevin. Critical Posthumanist Literacy: Building Theory for Reading, Writing, and Living Ethically with Everyday Artificial Intelligence. Reading Research Quarterly, v. 59, n. 4, p. 560–569, 2024. DOI: 10.1002/rrq.565.

BURSTEIN, Jill; CARDWELL, Ramsey; CHUANG, Ping-Lin; MICHALOWSKI, Allison; NYDICK, S. Exploring AI-Enabled Test Practice, Affect, and Test Outcomes in Language Assessment. In: PROCEEDINGS of the Artificial Intelligence in Measurement and Education Conference (AIME-Con). [S. l.: s. n.], 2025.

BURSTEIN, Jill; LAFLAIR, Geoffrey; KUNNAN, Antony John; DAVIER, Alina von. A Theoretical Assessment Ecosystem for a Digital-First Assessment—The Duolingo English Test. [S. l.], 2022.

CASTILLO, L. Explain My Answer Is Now Free for All Learners. Duolingo Blog. 1 Jan. 2026. Available from: https://blog.duolingo.com/explain-my-answer-now-free/.

COPE, Bill; KALANTZIS, Mary. Artificial Intelligence in the Long View: From Mechanical Intelligence to Cyber-Social Systems. Discover Artificial Intelligence, v. 2, n. 13, 2022. DOI: 10.1007/s44163-022-00029-1.

COPE, Bill; KALANTZIS, Mary. A Multimodal Grammar of Artificial Intelligence: Measuring the Gains and Losses in Generative AI. Multimodality & Society, v. 4, n. 2, p. 123–152, 2024. DOI: 10.1177/26349795231221699.

COPE, Bill; KALANTZIS, Mary. On Cyber-Social Learning: A Critique of Artificial Intelligence in Education. In: KORKOULOU, Despina; TZIRIDES, Orestis A.; COPE, Bill; KALANTZIS, Mary (eds.). Trust and Inclusion in AI-Mediated Education: Where Human Learning Meets Learning Machines. [S. l.]: Springer, 2024.

COPE, Bill; KALANTZIS, Mary; SEARSMITH, Duane. Artificial Intelligence for Education: Knowledge and Its Assessment in AI-Enabled Learning Ecologies. Educational Philosophy and Theory, v. 53, n. 12, p. 1229–1245, 2021. DOI: 10.1080/00131857.2020.1728732.

DAI, Yun; LAI, Sichen. A Theoretical Framework of Student Agency in AI-Assisted Learning: A Grounded Theory Approach. In: KHAN, L. (ed.). Oxford Intersections: Social Media in Society and Culture. [S. l.]: Oxford University Press, 2025.

DAVIER, Alina von; DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid; BERGER, Cindy. Designing for Assessment in Multimodal English Language Learning: Examples from Duolingo. Paper presented at the 12th International Conference on Multimodality (12ICOM), 29–31 October 2025. [S. l.], 2025.

DIAMANTOPOULOU, Sophia. Student Agency in the Creation of Multimodal Texts at the Ancient Agora of Athens: A Political Project. In: STARC, Susanne; KOMNINOS, Nicos (eds.). Multimodal Literacy in Education: Perspectives from Global Practices. [S. l.]: Peter Lang, 2024. p. 63–94.

DIAMANTOPOULOU, Sophia; FLORÉN, Henrika. Social Semiotics and Multimodality. In: INTERNATIONAL Encyclopedia of Language and Linguistics. 3. ed. [S. l.: s. n.], 2026. p. 254–259. DOI: 10.1016/B978-0-323-95504-1.00539-1.

DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid (eds.). Multimodality in English Language Learning. [S. l.]: Routledge, 2022.

DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid. The Concept of Design in the Field of Multimodal English Language Learning: Recognizing the Distribution of Agency in Semiotic Work. In: PROCEEDINGS of the 8th Designs for Learning Conference. [S. l.: s. n.], 2024. Paper presented at the 8th Designs for Learning Conference: Conceptualizing Design in Research and Development Work, Stockholm University, Stockholm,

Sweden, August 28–30.

DIAMANTOPOULOU, Sophia; ØREVIK, Sigrid. Multimodality and English Language Learning (MELL). In: CHAPELLE, Carol A. (ed.). Encyclopedia of Applied Linguistics. 2. ed. [S. l.]: Wiley, 2025. DOI: 10.1002/9781405198431.wbeal20822.

DONDERO, Maria Giulia; ALONSO, Juan; LEONE, Massimo. Aspects of AI Semiotics: Enunciation, Agency, and Creativity. Semiotica, n. 262, p. 1–3, 2025. DOI: 10.1515/sem-2025-0014.

DUOLINGO. Practice Speaking with Video Call. Video, 33 s. YouTube. 2024. Available from: https://www.youtube.com/watch?v=vBjfn2Pb6DI. Visited on: 15 June 2026.

DUOLINGO. Duolingo English Test: Test Format. Duolingo English Test. 2025. Available from: https://englishtest.duolingo.com/prepare/format. Visited on: 15 June 2026.

DUOLINGO. Interactive Speaking Practice Hub. Duolingo English Test. 2025. Available from: https://englishtest.duolingo.com/test/practiceHub/interactive-speaking?isReview=true. Visited on: 15 June 2026.

DUQUE-PEREIRA, Ives da Silva; MOURA, Sergio de. AI Chatbots and the Simulation of Dialog: What Does Bakhtinian Theory Have to Say? SciELO in Perspective. 30 Apr. 2025. Available from: https://blog.scielo.org/en/2025/04/30/ai-chatbots-and-the-simulation-of-dialog-what-does-bakhtinian-theory-have-to-say/.

GARZÓN, Juan; PATIÑO, Eddy; MARULANDA, Camilo. Systematic Review of Artificial Intelligence in Education: Trends, Benefits, and Challenges. Multimodal Technologies and Interaction, v. 9, n. 8, p. 84, 2025. DOI: 10.3390/mti9080084.

GELL, Alfred. Art and Agency: An Anthropological Theory. Oxford: Clarendon Press, 1998.

HENRY, Parker. Get to Know the AI Behind Every Video Call with Lily. Duolingo Blog. 22 Apr. 2025. Available from: https://blog.duolingo.com/ai-and-video-call/.

JEWITT, Carey; ADAMI, Elisabetta; ARCHER, Arlene. Editorial. Multimodality & Society, v. 1, n. 1, p. 3–7, 2021. DOI: 10.1177/2634979521992902.

JEWITT, Carey; HENRIKSEN, Berit. Social Semiotic Multimodality. In: O’HALLORAN, Kay L.; TAN, Sabine (eds.). Social Semiotics and Socially Situated Sign Making: Essays in Honour of Theo van Leeuwen. [S. l.]: De Gruyter Mouton, 2016.

JIANG, Lianjiang; LAI, Chun. How Did the Generative Artificial Intelligence-Assisted Digital Multimodal Composing Process Facilitate the Production of Quality Digital Multimodal Compositions: Toward a Process-Genre Integrated Model. TESOL Quarterly, v. 59, n. 1, p. 52–85, 2025. DOI: 10.1002/tesq.3390.

KARATZA, Styliani; ADAMI, Elisabetta. Multimodal Communication with Technology in the Classroom: From TELL to TECL (Technology-Enhanced Communication Learning). In: HAMPEL, Regine; STICKLER, Ursula (eds.). The Bloomsbury Handbook of Language Learning and Technology. [S. l.]: Bloomsbury Academic, 2024. p. 98–114.

KARIMOVA, Gulnara. The Dialogic Evolution of AI-Based Products: A Polyphonic Analysis of Temporal Transformations. International Journal of Human–Computer Interaction, v. 41, n. 5, p. 3625–3638, 2025. DOI: 10.1080/10447318.2024.2338661.

KOSTROMITINA, Maria. From 2020 to 2025: A Look at the Evolution of the DET. [S. l.], 2025. Available from: https://go.duolingo.com/l/208162/2025-07-01/5rc2kc/208162/1751387004rz2mVDcq/evolutionupdate2025_063025.pdf.

KRESS, Gunther. Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge, 2010.

KRESS, Gunther; LEEUWEN, Theo van. Reading Images: The Grammar of Visual Design. 3. ed. London: Routledge, 2021.

LATOUR, Bruno. Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford University Press, 2005.

LIM, Fei Victor; GILJE, Øystein; DJONOV, Emilia. Editorial for Special Issue: Digital Multimodal Composing in the Era of Artificial Intelligence. Computers and Composition, v. 75, p. 102911, 2025. DOI: 10.1016/j.compcom.2024.102911.

MATTHEWS, Stephanie Walsh; DANESI, Marcel. AI: A Semiotic Perspective. Chinese Semiotic Studies, v. 15, n. 2, p. 199–216, 2019. DOI: 10.1515/css-2019-0012.

OPENAI. Duolingo: GPT-4 Deepens the Conversation on Duolingo. OpenAI. 2026. Available from: https://openai.com/index/duolingo/?video=806129298. Visited on: 15 June 2026.

PANSY_PIX. I Recently Got Duolingo Max for the Roleplay Option and It’s Giving Me Three Stars for Lessons I Haven’t Even Done Yet. Why Is That? Reddit. 22 Sept. 2024. Available from: https://www.reddit.com/r/duolingo/comments/1fn3c02/i_recently_got_duolingo_max_for_the_roleplay/. Visited on: 15 June 2026.

PARK, Yena; CARDWELL, Ramsay; GOODWIN, Sarah; NAISMITH, Ben; LAFLAIR, Geoffrey; YANCEY, Kevin; ZHANG, Xiaowan. Assessing Speaking on the Duolingo English Test. [S. l.], 2025. Available from: https://englishtest.duolingo.com/research.

PURBA, Siska; SILITONGA, Bertha; YANG, John. AI-Assisted Learning: A Systematic Review. Turkish Online Journal of Distance Education, v. 26, n. 4, p. 77–93, 2025. DOI: 10.17718/tojde.1591404.

SELANDER, Staffan; KRESS, Gunther. Design för lärande: Ett Multimodalt Perspektiv. Lund: Studentlitteratur, 2017.

SEO, Kyongwon; TANG, Joyce; ROLL, Ido; FELS, Sidney; YOON, Dongwook. The Impact of Artificial Intelligence on Learner–Instructor Interaction in Online Learning. International Journal of Educational Technology in Higher Education, v. 18, p. 54, 2021. DOI: 10.1186/s41239-021-00292-9.

SWANEPOEL, Danielle; CORKS, Daniel. Artificial Intelligence and Agency: Tie-Breaking in AI Decision-Making. Science and Engineering Ethics, v. 30, p. 56, 2024. DOI: 10.1007/s11948-024-00476-2.

TANG, Kok-Sing; PUTRA, Gde Buana. Generative AI as a Dialogic Partner: Enhancing Multiple Perspectives, Reasoning, and Argumentation in Science Education with Customized Chatbots. Journal of Science Education and Technology, 2025. DOI: 10.1007/s10956-025-10240-1.

TOMALIN, Marcus. Multimodal Social Semiotics and the Challenge of Artificial Intelligence. Multimodality & Society, v. 5, n. 2, p. 223–244, 2025. DOI: 10.1177/26349795251327939.

TUFEKCI, Zeynep. Algorithmic Harms Beyond Facebook and Google: Emergent Challenges of Computational Agency. Colorado Technology Law Journal, v. 13, n. 2, 2015.

WALLINGFORD, Priya. How Duolingo Call Works: Features, Benefits and Usage. Duolingo Guides. 2 Mar. 2025. Available from: https://duolingoguides.com/how-duolingo-call-works/.

WODZAK, Sophie. No More Tradeoffs: Interactive Speaking. Duolingo English Test Blog. 5 Apr. 2025. Available from: https://blog.englishtest.duolingo.com/interactive-speaking/.

ZAPPAVIGNA, Michele; DORAN, Y. J. Emoji as Interpersonal Resources in LLM Chatbot Conversations: A Social Semiotic Analysis of Tenor and Affiliation in Human–AI Interaction. Social Semiotics, 2025. DOI: 10.1080/10350330.2025.2570317.

ZENML. Structured LLM Conversations for Language Learning Video Calls. ZenML. 2025. Available from: https://www.zenml.io/llmops-database/structured-llm-conversations-for-language-learning-video-calls.