Semântica de Frames Multimodal
expandindo as categorias de análise dos datasets multimodais da FrameNet Brasil
DOI:
https://doi.org/10.35699/2317-2096.2025.57570Keywords:
dêixis, atenção conjunta, multimodalidade, Semântica de Frames, narrativa fílmicaAbstract
Neste artigo, proponho uma extensão dos modelos de cena de atenção conjunta para incluir cenários comunicativos multimodais mediados. A partir dessa extensão, discuto como o modelo analítico da FrameNet pode ser usado para a anotação de gêneros multimodais, de modo a incluir categorias capazes de explicar as mudanças convidadas na atenção conjunta em narrativas visuais. Revisito a literatura sobre dêixis e afirmo que, apesar de grande parte da pesquisa fundamental sobre esse tópico propor correlações entre dêixis na linguagem verbal e outros modos semióticos, o foco da pesquisa está no material linguístico no qual a expressão de dêixis está fundamentada. Ao reunir contribuições dos estudos de comunicação humana, da análise de dêixis em narrativas e da Semântica de Frames, proponho que dispositivos semióticos de diferentes tipos podem ser mobilizados em narrativas fílmicas para convidar os espectadores a promover mudanças no centro da cena de atenção conjunta. Para ilustrar a tipologia proposta, apresento exemplos de análises de deslocamentos do centro dêitico em filmes e descrevo a metodologia de anotação incorporada ao modelo FrameNet para explicar esses deslocamentos.
References
BAKHTIN, M. M. Esthétique de la création verbale. Paris: Gallimard, 1984.
BATEMAN, J. Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. Berlin: Springer, 2008.
BATEMAN, J.; WILDFEUER, J.; HIIPALA, T. Multimodality: Foundations, Research and Analysis – A Problem-Oriented Introduction. Berlin: Walter de Gruyter GmbH & Co, 2017.
BELCAVELLO, F. FrameNet Annotation for Multimodal Corpora: Devising a Methodology for the Semantic Representation of Text-Image Interactions in Audiovisual Productions. 2023. 134 f. Thesis (Doctorate in Linguistics) – Faculdade de Letras, Universidade Federal de Juiz de Fora, Juiz de Fora, 2023.
BELCAVELLO, F.; VIRIDIANO, M.; COSTA, A. D. da; MATOS, E. E. da S.; TORRENT, T. T. Frame-Based Annotation of Multimodal Corpora: Tracking (A) Synchronies in Meaning Construction. In: INTERNATIONAL FRAMENET WORKSHOP: TOWARDS A GLOBAL, MULTILINGUAL FRAMENET, 2020, Marseille. Proceedings […]. Marseille: European Language Resources Association (ELRA), 2020. p. 23-30.
BELCAVELLO, F.; VIRIDIANO, M.; MATOS, E.; TORRENT, T. T. Charon: A FrameNet Annotation Tool for Multimodal Corpora. In: LINGUISTIC ANNOTATION WORKSHOP (LAW-XVI), 16. Marseille. Proceedings […]. Workshop presented at LREC2022. Marseille: ELRA, 2022. p. 91-96.
BORDWELL, D.; THOMPSON, K. A arte do cinema: uma introdução. Tradução: Roberta Gregoli. Campinas, SP: Editora da Unicamp; São Paulo: Editora da USP, 2013.
BÜHLER, K. Sprachtheorie. Jena: Fischer, 1934.
CIENKI, A. The Study of Gesture in Cognitive Linguistics: How It Could Inform and Inspire Other Research in Cognitive Science. Cognitive Science, v. 13, n. 6, e1623, 2022.
COHN, N. A Multimodal Parallel Architecture: A Cognitive Framework for Multimodal Interactions. Cognition, v. 146, p. 304-323, 2016a.
COHN, N. From Visual Narrative Grammar to Filmic Narrative Grammar: The Narrative Structure of Static and Moving Images. In: WILDFEUER, J.; BATEMAN, J. (ed.). Film Text Analysis: New Perspectives on the Analysis of Filmic Meaning. London: Routledge, 2016b. p. 94-117.
COHN, N. Visual Narrative Structure. Cognitive Science, v. 37, n. 3, p. 413-452, 2013.
DANNÉLLS, D.; TORRENT, T. T.; SIGILIANO, N. S.; DOBNIK, S. Beyond Strings of Characters: Resources Meet NLP – Again. In: VOLODINA, E.; DANNÉLLS, D.; BERDICEVSKIS, A.; FORSBERG, M.; VIRK, S. (ed.). Live and Learn: Festschrift in Honor of Lars Borin. Gothenburg: Institutionen för Svenska, Flerspråkighet och Språkteknologi – Göteborgs Universitet, 2022. p. 29-36.
DUTRA, L. V.; SIGILIANO, N. S. Ferramenta linguístico-computacional como facilitadora para o ensino de gramática na escola. In: SIMPÓSIO BRASILEIRO DE TECNOLOGIA DA INFORMAÇÃO E DA LINGUAGEM HUMANA (STIL), 2021, [S. l.]. Anais […]. [S. l.]: SBC, 2021. p. 432-436.
FILLMORE, C. J. Frame Semantics and the Nature of Languages. In: ANNALS OF THE NEW YORK ACADEMY OF SCIENCES. Conference on the Origin and Development of Language and Speech, v. 280, p. 20-32, 1976. Work presented at the Conference on the Origin and Development of Language and Speech, 1976, New York.
FILLMORE, C. J. Frame Semantics. In: THE LINGUISTIC SOCIETY OF KOREA (ed.). Linguistics in the Morning Calm. Seoul: Hanshin Publishing Company, 1982. p. 111-137.
FILLMORE, C. J. Frames and the Semantics of Understanding. Quaderni di Semantica, v. 6, n. 2,
p. 222-254, 1985.
FILLMORE, C. J. Santa Cruz Lectures on Deixis, 1971. Bloomington, IN: Indiana University Linguistics Club, 1975.
FILLMORE, C. J. The Case for Case Reopened. In: COLE, P.; SADOCK, J. M. (ed.). Syntax and Semantics: Grammatical Relations. New York: Academic Press, 1977. v. 8, p. 59-81.
FILLMORE, C. J. The Merging of Frames. In: ROSSINI FAVRETTI, R. (ed.). Frames, Corpora, and Knowledge Representation. Bologna: Bononia University Press, 2008. p. 1-12.
FILLMORE, C. J.; PETRUCK, M. R.; RUPPENHOFER, J.; WRIGHT, A. FrameNet in Action: The Case of Attaching. International Journal of Lexicography, v. 16, n. 3, p. 297-332, 2003.
FRAMENET BRASIL LAB. WebTool. c2014-2025. Available at: http://webtool.frame.net.br/. Accessed on: 31 Jan. 2025.
GOLDIN-MEADOW, S. Pointing Sets the Stage for Learning Language – and Creating Language. Child Development, v. 78, n. 3, p. 741-745, 2007.
GRICE, H. P. Meaning. The Philosophical Review, v. 66, n. 3, p. 377-388, 1957.
HANKS, W. F. Língua como prática social: das relações entre língua, cultura e sociedade a partir de Bourdieu e Bakhtin. Textos selecionados por Hanks, traduzidos por Anna Christina Bentes, Marco Antônio Rosa Machado, Marcos Rogério Cintra e Renato C. Rezende. São Paulo: Cortez, 2008.
JACKENDOFF, R. Foundations of Language. Oxford: Oxford University Press, 2002.
JACKENDOFF, R.; LERDAHL, F. The Capacity for Music: What is It, and What’s Special About It? Cognition, v. 100, p. 33-72, 2006.
KALANTZIS, M.; COPE, B.; PINHEIRO, P. Letramentos. Campinas: Unicamp, 2020.
KITA, S. Pointing: Where Language, Culture and Cognition Meet. Mahwah, NJ:
Lawrence Erlbaum, 2003.
LEMKE, J. Multiplying Meaning. In: MARTIN, J. R.; VEEL, R. (ed.). Reading Science: Critical and Functional Perspectives on Discourses of Science. London: Routledge, 1998. p. 87-114.
LEVINSON, S. C. Deixis. In: HORN, L. R.; WARD, G. (ed.). The Handbook of Pragmatics. Malden, MA: Blackwell Publishing, 2004. p. 97-121.
LISZKOWSKI, U.; BROWN, P.; CALLAGHAN, T.; TAKADA, A.; DE VOS, C. A. Prelinguistic Gestural Universal of Human Communication. Cognitive Science, v. 36, p. 698-713, 2012.
RODRÍGUEZ, C.; MORENO-NÚNEZ, A.; BASILIO, M.; SOSA, N. Ostensive Gestures Come First: Their Role in the Beginning of Shared Reference. Cognitive Development, v. 36, p. 142-149, 2015.
SAUSSURE, F de. Cours de linguistique générale. Paris: Payot, 1916.
SEARLE, J. R. Collective Intentions and Actions. In: COHEN, O.; MORGAN, J.; POLLACK, M. (ed.). Intentions in Communication. Cambridge, MA: MIT Press, 1990.
SEGAL, E. M. Narrative Comprehension and the Role of Deictic Shift Theory. In: DUCHAN, J.; BRUDER, G. A.; HEWITT, L. E. (ed.). Deixis in Narrative: A Cognitive Science Perspective. London: Psychology Press, 1995. p. 3-18.
STEEDMAN, M. J. Plans, Affordances, and Combinatory Grammar. Linguistics and Philosophy, v. 25, n. 5-6, p. 723-753, 2002.
STEEN, F.; HOUGAARD, A.; JOO, J.; OLZA, I.; CÁNOVAS, C.; PLESHAKOVA, A.; RAY, S.; UHRIG, P.; VALENZUELA, J.; WOŹNY, J.; TURNER, M. Toward an Infrastructure for Data-Driven Multimodal Communication Research. Linguistics Vanguard, v. 4, n. 1, e20170041, 2018.
SWEETSER, E. Looking at Space to Study Mental Spaces: Co-Speech Gesture as a Crucial Data Source in Cognitive Linguistics. In: GONZALEZ-MARQUEZ, M.; SPIVEY, M. J.; COULSON, S.; MITTELBERG, I. (ed.). Methods in Cognitive Linguistics. Amsterdam: John Benjamins, 2007. p. 201-224.
THE NEIGHBORS Window. Written and directed by Marshall Curry. New York: Marshall Curry Productions, 2019. Available at: http://www.theneighborswindow.com/. Accessed on: 31 Jan. 2025.
THE NEW LONDON GROUP. A Pedagogy of Multiliteracies: Designing Social Futures. Harvard Educational Review, v. 66, n. 1, p. 60-93, 1996.
TOMASELLO, M. Becoming Human: A Theory of Ontogeny. Cambridge, MA: The Belknap Press of Harvard University Press, 2018.
TOMASELLO, M. Constructing a Language: A Usage-Based Theory of Language Acquisition. Cambridge, MA: Harvard University Press, 2003.
TOMASELLO, M. Origins of Human Communication. Cambridge, MA: MIT Press, 2008.
TOMASELLO, M. The Cultural Origins of Human Cognition. Cambridge, MA: MIT Press, 1999.
TOMASELLO, M.; CARPENTER, M.; LISZKOWSKI, U. A New Look at Infant Pointing. Child Development, v. 78, n. 3, p. 705-722, 2007.
TORRENT, T. T.; MATOS, E. E. S.; BELCAVELLO, F.; VIRIDIANO, M.; GAMONAL, M. A.; COSTA, A. D. da; MARIM, M. C. Representing Context in FrameNet: A Multidimensional, Multimodal Approach. Frontiers in Psychology, v. 13, e838441, 2022.
TURNER, M. Multimodal Form-Meaning Pairs for Blended Classic Joint Attention. Linguistics Vanguard, v. 3, n. s1, e20160043, 2017.
TURNER, M. The Role of Creativity in Multimodal Construction Grammar. Zeitschrift für Anglistik und Amerikanistik, v. 66, n. 3, p. 357-370, 2018.
VIRIDIANO, M.; TORRENT, T. T.; CZULO, O.; LORENZI, A.; MATOS, E.; BELCAVELLO, F. The Case for Perspective in Multimodal Datasets. In: WORKSHOP ON PERSPECTIVIST APPROACHES TO NLP, 1., 2022, Marseille. Proceedings […]. Workshop presented at LREC2022. Marseille: European Language Resources Association, 2022. p. 108-116.
WILKINS, D. Why Pointing with the Index Finger is not a Universal (in Sociocultural and Semiotic Terms). In: KITA, S. (ed.). Pointing: Where Language, Culture and Cognition Meet. Mahwah, NJ: Lawrence Erlbaum, 2003. p. 171-216.
WITTGENSTEIN, L. Philosophische Untersuchungen. London: Kegan Paul, 1953.
ZUBIN, D. A.; HEWITT, L. E. The Deictic Center: A Theory of Deixis in Narrative. In: DUCHAN J. F.; BRUDER G. A.; HEWITT, L. E. (ed.). Deixis in Narrative: A Cognitive Science Perspective. New York: Routledge: Taylor & Francis Group, 1995. p. 129-155.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Natália Sathler Sigiliano
This work is licensed under a Creative Commons Attribution 4.0 International License.
