Pronunciation Assessment
Relating the CEFR’s Phonological Control Scale to Intelligibility and Comprehensibility Measures
DOI:
https://doi.org/10.17851/2237-2083.33.1.195-219Palabras clave:
speech assessment, intelligibility, comprehensibility, phonological controlResumen
Research in pronunciation learning and teaching over the past years has advanced the idea that working with the intelligibility and comprehensibility constructs is fundamental to developing pronunciation for successful communication. This study investigates the relationship between the Common European Framework of Reference for Languages - CEFR phonological scale (2018 version), and two measures commonly used in second language (L2) speech research, namely, intelligibility and comprehensibility rates. Speech samples from 16 Brazilian speakers of English were collected. Their speech samples were assessed by 14 listeners, teachers of English, in terms of intelligibility, comprehensibility and phonological control. The raters transcribed the speech samples, which generated an intelligibility score (number of words correctly transcribed); for comprehensibility, the raters assigned a level of difficulty in understanding the speech through a nine-point scale; finally, they used the CEFR scale to rate the level of phonological control for each speaker. Results show a highly significant correlation between intelligibility and comprehensibility, but not significant correlation between these two speech dimension variables and phonological control.
Referencias
ALLAN, D., Oxford Placement Test (1 or 2). Oxford: Oxford University Press, 2004.
BENT, T.; BRADLOW, A. R. The interlanguage speech intelligibility benefit. Journal of the Acoustical Society of America, Melville, v. 114, p. 1600–1610, 2003.
COUNCIL OF EUROPE. Common European Framework of Reference for Languages - Companion Volume. 2018. Retrieved from https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2018/1680787989
COUNCIL OF EUROPE. Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Press Syndicate of the University of Cambridge. 2001.
DERWING, T.; MUNRO, M. Accent, Intelligibility, And Comprehensibility: Evidence from Four L1s. Studies in Second Language Acquisition, Cambridge, v. 19, n.1, p. 1–16. 1997.
DERWING, T.; MUNRO, M. Second Language Accent and Pronunciation Teaching: A Research-Based Approach. TESOL Quarterly, Alexandria,v. 39, n.3, p. 379–397. 2005.
DERWING, T.; MUNRO, M. Pronunciation fundamentals: evidence-based perspectives for L2 teaching and research. Amsterdam: Benjamins. 2015.
DERWING, T., MUNRO, M.; THOMSON, R. A Longitudinal Study of ESL Learners’ Fluency and Comprehensibility Development. Applied Linguistics, Oxford,v. 29, n.3, p. 359-380. 2008.
DEYGERS, B., ZEIDLER, B., VILCU, D.; CARLSEN, C. H. One Framework to Unite Them All? Use of the CEFR in European University Entrance Policies. Language Assessment Quarterly, Philadelphia,v. 15, n.1, p. 3–15. 2018.
FIGUERAS, N. The impact of the CEFR. Elt Journal, [S.L.], v. 66, n. 4, p. 477-485, 17 set. 2012. Oxford, Oxford University Press.
FOOTE, J. A.; TROFIMOVICH, P. Is it because of my language background? A study of language background influence on comprehensibility judgments. Canadian Modern Language Review, Toronto, v. 74, n.2, p. 253–278. 2018.
HARDING, L. What do raters need in a pronunciation scale? The user’s view. In ISAACS, T.; TROFIMOVICH, P. (eds) Second Language Pronunciation Assessment. Bristol: Multilingual Matters, 2017. p. 12-34.
ISAACS, T. Assessing pronunciation. In KUNNAN, A.J. (org.) The Companion to Language Assessment. Hoboken: Wiley, 2013. p. 140–155.
ISAACS, T.; THOMSON, R. Rater experience, rating scale length, and judgments of L2 pronunciation: revisiting research conventions. Language Assessment Quarterly, [s. l], v. 2, n. 10, p. 135-159, 2012.
KANG, O.; KERMAD, A. Assessment in second language pronunciation. In KANG, O. THOMSON, R.; MURPHY, J. (orgs.) The Routledge Handbook of Contemporary English Pronunciation. New York: Routledge, 2018. p. 511-526.
KANG, O., RUBIN, D; KERMAD, A. Effect of training and rater individual differences on oral proficiency assessment. Language Testing, London, v. 36, n.4, p. 481-504, 2019.
KANG, O., THOMSON, R. I.; MORAN, M. Empirical Approaches to Measuring the Intelligibility of Different Varieties of English in Predicting Listener Comprehension. Language Learning, Ann Arbor,v. 68, n.1, p. 115–146. 2017.
KHABBAZBASHI, N.; GALACZI, E. D. A comparison of holistic, analytic, and part marking models in speaking assessment. Language Testing, London, v. 37, n.3, p. 333–360. 2020.
KUIKEN, F., VEDDER, I. Rating written performance: What do raters do and why? Language Testing, London, v. 31, n.3, p. 329–348. 2014.
LARSON-HALL, J. A Guide to Doing Statistics in Second Language Research Using SPSS. New York: Routledge, 2010.
LEVIS, J. M. Changing Contexts and Shifting Paradigms in Pronunciation Teaching. TESOL Quarterly, Alexandria, v. 39, n.3, p. 369–377. 2005.
LEVIS, J. M. Revisiting the intelligibility and nativeness principles. Journal of Second Language Pronunciation, Amsterdam,v. 6, n.3, p. 310-328. 2020.
MARTINS, T.S. Using CEFR’s phonological control scale to assess L2 learners’ intelligibility and comprehensibility. 2022. 95 f. Dissertação (Mestrado Letras Inglês) - Universidade Federal de Santa Catarina, Centro de Comunicação e Expressão, Programa de Pós-Graduação em Inglês: Estudos Linguísticos e Literários, Florianópolis, 2022. Disponível em: https://bu.ufsc.br/teses/PPGI0225-D.pdf. Acesso em: 12 maio 2023.
MUNRO, M. J.; DERWING, T. M. Processing Time, Accent, and Comprehensibility in the Perception of Native and Foreign-Accented Speech. Language and Speech, Thousand Oaks, v. 38, n.3, p. 289–306. 1995a.
MUNRO, M. J.; DERWING, T. M. Foreign Accent, Comprehensibility, and Intelligibility in the Speech of Second Language Learners. Language Learning, Ann Arbor,v. 45, n.1, p. 73–97. 1995b.
MUNRO, M. J.; DERWING, T. M. A prospectus for pronunciation research in the 21st century: A point of view. Journal of Second Language Pronunciation, Amsterdam,v. 1, n.1, p. 11–42. 2015.
QUEVEDO-CAMARGO, G. Breve história da evolução do construto proficiência em línguas. Em Aberto, Brasília, v. 32, n. 104, p. 27-44, 2019.
SAITO, K., TROFIMOVICH, P.; ISAACS, T. Second language speech production: Investigating linguistic correlates of comprehensibility and accentedness for learners at different ability levels. Applied Psycholinguistics, Cambridge, v. 7, n.2, p. 217–240. 2015.
SILVEIRA, R.; MARTINS, T. S. Assessing second language oral proficiency development with holistic and analytic scales. Ilha do Desterro, Florianópolis, v. 73, p. 227-250, 2020.
SILVEIRA, R.; SILVA, T. C. L2 Speech intelligibility: effects of coda modification, degree of semantic information and listeners’ background. Revista Brasileira de Linguística Aplicada, Belo Horizonte, v. 18, p. 639-664, 2018.
TOFFOLI, S.; ANDRADE, D.; BORNIA, A.C.; QUEVEDO-CAMARGO, G. Avaliação com itens abertos: validade, confiabilidade, comparabilidade e justiça. Educação e Pesquisa, São Paulo, v. 42, n. 2, p. 343-358, abr. 2016.
TOPAL, İ. H. CEFR-oriented probe into pronunciation: Implications for language learners and teachers. Journal of Language and Linguistic Studies, Konya,v. 15, n.2, p. 420–436. 2019
TROFIMOVICH, P.; ISAACS, T. Disentangling accent from comprehensibility. Bilingualism, Cambridge, v. 15, n.4, p. 905–916. 2012.
