ARTICLE-OPINION: THREE-TIER TEST VALIDATION FOR MAPPING EPISTEMOLOGICAL DENSITY PROFILES
Artigo-parecer: Validação de teste em três camadas para mapear perfis epistemológicos de densidade
DOI:
https://doi.org/10.1590/1983-21172022240142Palavras-chave:
inventários conceituais, modelo de Rasch, Ciência Aberta, Dados AbertosResumo
Este trabalho é um artigo-parecer derivado do processo de avaliação do artigo original de Melo e Amantes (2022b), que usa um teste de três camadas para avaliar os perfis epistemológicos dos alunos sobre o conceito de densidade, o primeiro artigo publicado na revista Ensaio, cujos dados suplementares de pesquisa foram disponibilizados publicamente no repositório Scielo Dataverse Ensaio. Começamos o debate explicando o paradigma da Ciência Aberta, que é um tema fundamental deste artigo-parecer. Após uma breve revisão de diferentes perspectivas para investigar como os alunos aprendem conceitos científicos, são abordadas a perspectiva sociocultural e cognitivista clássica da aprendizagem, destacando as diferenças centrais entre elas nas investigações sobre o aprendizado de conceitos científicos, enfatizando a importância das formas modernas de inventários conceituais para avaliar concepções dos alunos em amostras maiores. Após uma discussão sobre o método adotado pelas autoras, o modelo de Rasch, e a (falsa) dicotomia entre pesquisas qualitativas e quantitativas, a política de Dados Abertos foi mais bem detalhada para esclarecer suas potencialidades. Para ilustrar essas noções, é dado um exemplo da relevância de variáveis contextuais em um bem conhecido programa de avaliação em larga escala.
Downloads
Referências
Ayala, R. J. de (2022). The theory and practice of Item Response Theory Guilford Publications.
Bayuni, T. C., Sopandi, W., & Sujana, A. (2018). Identification misconception of primary school teacher education students in changes of matters using a five-tier diagnostic test. Journal of Physics: Conference Series, 1013(1), 012086. https://doi.org/10.1088/1742-6596/1013/1/012086 » https://doi.org/10.1088/1742-6596/1013/1/012086
Bock, R. D., Gibbons, R., & Muraki, E. (1988). Full-Information Item Factor Analysis. Applied Psychological Measurement, 12(3), 261-280. https://doi.org/10.1177/014662168801200305 » https://doi.org/10.1177/014662168801200305
Borsboom, D. (2008). Latent Variable Theory. Measurement: Interdisciplinary Research and Perspectives, 6(1-2), 25-53. https://doi.org/10.1080/15366360802035497 » https://doi.org/10.1080/15366360802035497
Butler, D. (2003). Who will pay for open access? Nature, 425(6958), 554-555. https://doi.org/10.1038/425554a
» https://doi.org/10.1038/425554a
Cabrerizo, F. M. (2022). Open access in low-income countries - open letter on equity. Nature, 605, 620. https://doi.org/10.1038/d41586-022-01414-7
» https://doi.org/10.1038/d41586-022-01414-7
Cavalcanti, C. J. H., Nascimento, M. M., & Ostermann, F. (2018). A falácia da culpabilização do professor pelo fracasso escolar. Thema, 15(3), 1064-1088. https://doi.org/10.15536/thema.15.2018.1064-1088.1059
» https://doi.org/10.15536/thema.15.2018.1064-1088.1059
Chen, C.-W., Andersson, B., & Zhu, J. (2022). A Factor Mixture Model for Item Responses and Certainty of Response Indices to identify student knowledge profiles. Journal of Educational Measurement, n/a(n/a). https://doi.org/10.1111/jedm.12344 (por enquanto, apenas versão on-line disponível)
» https://doi.org/10.1111/jedm.12344
Crocker, L. M., & Algina, J. (2008). Introduction to Classical and Modern Test Theory Cengage Learning.
Directory of Open Access Journals - DOAJ. (2023). Directory of Open Access Journals - DOAJ Recuperado em 2 de fevereiro, de https://doaj.org/
Elsevier. (2018). What are mirror journals, and can they offer a new world of open access? Elsevier. Recuperado em 29 de janeiro, de https://www.elsevier.com/connect/what-are-mirror-journals-and-can-they-offer-a-new-world-of-open-access
Eryılmaz, A. (2010). Development and application of three-tier heat and temperature test: Sample of bachelor and graduate students. Eurasian Journal of Educational Research, 40, 53-76. https://ejer.com.tr/wp-content/uploads/2021/01/ejer_2010_issue_40.pdf
» https://ejer.com.tr/wp-content/uploads/2021/01/ejer_2010_issue_40.pdf
Gee, J. P., & Green, J. L. (1998). Discourse analysis, learning, and social practice: a methodological study. Review of Research in Education, 23(1), 119-169. https://doi.org/10.3102/0091732x023001119
» https://doi.org/10.3102/0091732x023001119
Góes, C. (2021). Pairwise difference regressions are just weighted averages. Scientific Reports, 11(1), 23044. https://doi.org/10.1038/s41598-021-02096-3
» https://doi.org/10.1038/s41598-021-02096-3
Halloun, I. A., & Hestenes, D. (1985a). Common sense concepts about motion. American Journal of Physics, 53(11), 1056-1065. https://doi.org/10.1119/1.14031
» https://doi.org/10.1119/1.14031
Halloun, I. A., & Hestenes, D. (1985b). The initial knowledge state of college physics students. American Journal of Physics, 53(11), 1043-1055. https://doi.org/10.1119/1.14030
» https://doi.org/10.1119/1.14030
Hestenes, D., & Halloun, I. (1995). Interpreting the force concept inventory: A response to March 1995 critique by Huffman and Heller. The Physics Teacher, 33(8), 502-502. https://doi.org/10.1119/1.2344278
» https://doi.org/10.1119/1.2344278
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141-158. https://doi.org/10.1119/1.2343497
» https://doi.org/10.1119/1.2343497
Huffman, D., & Heller, P. (1995). What does the force concept inventory actually measure? The Physics Teacher, 33(3), 138-143. https://doi.org/10.1119/1.2344171
» https://doi.org/10.1119/1.2344171
Humphry, S., & Montuoro, P. (2021). The Rasch Model cannot reveal systematic Differential Item Functioning in single tests: subset DIF analysis as an alternative methodology [Original Research]. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.742560
» https://doi.org/10.3389/feduc.2021.742560
Kaltakci-Gurel, D., Eryilmaz, A., & McDermott, L. C. (2017). Development and application of a four-tier test to assess pre-service physics teachers’ misconceptions about geometrical optics. Research in Science & Technological Education, 35(2), 238-260. https://doi.org/10.1080/02635143.2017.1310094
» https://doi.org/10.1080/02635143.2017.1310094
Kathawalla, U.-K., Silverstein, P., & Syed, M. (2021). Easing into Open Science: a guide for graduate students and their advisors. Collabra: Psychology, 7(1). https://doi.org/10.1525/collabra.18684
» https://doi.org/10.1525/collabra.18684
Lemke, J. (1988). Genres, semantics, and classroom education. Linguistics and Education, 1(1), 81-99. https://doi.org/10.1016/S0898-5898(88)80011-1
» https://doi.org/10.1016/S0898-5898(88)80011-1
Lemke, J. L. (1982). Talking physics. Physics Education, 17(6), 263. https://doi.org/10.1088/0031-9120/17/6/310
» https://doi.org/10.1088/0031-9120/17/6/310
Lemke, J. L. (1990). Talking science: language, learning, and values. Ablex Pub. Corp.
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.
Loehlin, J. C., & Beaujean, A. A. (2017). Latent Variable Models: an Introduction to Factor, Path, and Structural Equation Analysis. Routledge.
Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514-534. https://doi.org/10.1207/s15328007sem1104_2
» https://doi.org/10.1207/s15328007sem1104_2
Melo, V. F., & Amantes, A. (2022a). Validação de teste em três camadas para mapear perfis epistemológicos de densidade Versão 1 [Questionários e tabelas com respostas dicotômicas dos alunos]. SciELO Data. https://doi.org/10.48331/scielodata.G7PNYG
» https://doi.org/10.48331/scielodata.G7PNYG
Melo, V. F., & Amantes, A. (2022b). Validação de teste em três camadas para mapear perfis epistemológicos de densidade. Ensaio Pesquisa em Educação em Ciências, 24, e38980. https://doi.org/10.1590/1983-21172022240124
» https://doi.org/10.1590/1983-21172022240124
Mendonça, P. C. C., & Franco, L. G. (2021). A Ciência Aberta e a área de Educação em Ciências: perspectivas e diálogos. Ensaio Pesquisa em Educação em Ciências, 23 https://doi.org/10.1590/1983-21172021230102
» https://doi.org/10.1590/1983-21172021230102
Mendonça, P. C. C., Franco, L. G., & Massi, L. Coelho, G R.. (2023). Experiências da revista Ensaio Pesquisa em Educação em Ciências com Avaliação por Pares Aberta. Ensaio Pesquisa em Educação em Ciências, 25 https://doi.org/10.1590/1983-21172022240137
» https://doi.org/10.1590/1983-21172022240137
Meyerowitz‐Katz, G., Besançon, L., Flahault, A., & Wimmer, R. (2021). Impact of mobility reduction on COVID-19 mortality: absence of evidence might be due to methodological issues. Scientific Reports, 11(1), 23533. https://doi.org/10.1038/s41598-021-02461-2
» https://doi.org/10.1038/s41598-021-02461-2
Mislevy, R. J. (1986). Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11(1), 3-31. https://doi.org/10.2307/1164846
» https://doi.org/10.2307/1164846
MOOC - Massive Open Online Course. (2019). Homepage - Open Science MOOC Recuperado em 29 de janeiro, de https://www.elsevier.com/connect/what-are-mirror-journals-and-can-they-offer-a-new-world-of-open-access
Mortimer, E. F., Scott, P., Amaral, E. M. R., & El-Hani, C. N. (2014). Conceptual profiles: theoretical-methodological bases of a research program. In E. F. Mortimer & C. N. El-Hani(Eds.), Conceptual profiles: a theory of teaching and learning scientific concepts. Springer.
Nickerson, R. C., Varshney, U., & Muntermann, J. (2013). A method for taxonomy development and its application in information systems. European Journal of Information Systems, 22(3), 336-359. https://doi.org/10.1057/ejis.2012.26
» https://doi.org/10.1057/ejis.2012.26
Olejniczak, A. J., & Wilson, M. J. (2020). Who’s writing open access (OA) articles? Characteristics of OA authors at Ph.D.-granting institutions in the United States. Quantitative Science Studies, 1(4), 1429-1450. https://doi.org/10.1162/qss_a_00091
» https://doi.org/10.1162/qss_a_00091
Packer, M. (2010). The science of qualitative research Cambridge University Press. https://doi.org/10.1017/CBO9780511779947
» https://doi.org/10.1017/CBO9780511779947
Perry, M., & Elder, A. D. (1997). Knowledge in transition: adults’ developing understanding of a principle of physical causality. Cognitive Development, 12(1), 131-157. https://doi.org/10.1016/S0885-2014(97)90033-2
» https://doi.org/10.1016/S0885-2014(97)90033-2
Peşman, H., & Eryılmaz, A. (2010). Development of a three-tier test to assess misconceptions about simple electric circuits. The Journal of Educational Research, 103(3), 208-222. https://doi.org/10.1080/00220670903383002
» https://doi.org/10.1080/00220670903383002
PLOS. (2023a). Home - PLOS Recuperado em 29 de janeiro, de https://plos.org
PLOS. (2023b). Open Science - PLOS Recuperado em 29 de janeiro, de https://plos.org/open-science
» https://plos.org/open-science
Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: toward a theory of conceptual change. Science Education, 66(2), 211-227. https://doi.org/10.1002/sce.3730660207
» https://doi.org/10.1002/sce.3730660207
Pramesti, Y. S., Mahmudi, H., & Setyowidodo, I. (2021). Using three-tier test to diagnose students’ level of understanding. Journal of Physics: Conference Series, 1806(1), 012013. https://doi.org/10.1088/1742-6596/1806/1/012013
» https://doi.org/10.1088/1742-6596/1806/1/012013
Putra, A. S. U., Hamidah, I., & Nahadi. (2020). The development of five-tier diagnostic test to identify misconceptions and causes of students’ misconceptions in waves and optics materials. Journal of Physics: Conference Series, 1521(2), 022020. https://doi.org/10.1088/1742-6596/1521/2/022020
» https://doi.org/10.1088/1742-6596/1521/2/022020
Python. (2023). Python language reference (Versão 3.11.2). Python Software Foundation. https://www.python.org
R Core Team. (2023). R: A language and environment for statistical computing. (Versão 4.2.2). R Foundation for Statistical Computing. https://www.R-project.org
Rindskopf, D. (1984). Latent variable models: applications in education. Contemporary Educational Psychology, 9(2), 104-121. https://doi.org/10.1016/0361-476X(84)90013-4
» https://doi.org/10.1016/0361-476X(84)90013-4
Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Res, 6, 588. https://doi.org/10.12688/f1000research.11369.2
» https://doi.org/10.12688/f1000research.11369.2
Ross-Hellauer, T. (2022). Open science, done wrong, will compound inequities. Nature, 603(7901), 363. https://doi.org/10.1038/d41586-022-00724-0
» https://doi.org/10.1038/d41586-022-00724-0
Ross-Hellauer, T., Deppe, A., & Schmidt, B.(2017). Survey on open peer review: attitudes and experience amongst editors, authors and reviewers. PLOS ONE, 12(12), e0189311. https://doi.org/10.1371/journal.pone.0189311
» https://doi.org/10.1371/journal.pone.0189311
Ross-Hellauer, T., & Görögh, E. (2019). Guidelines for open peer review implementation. Research Integrity and Peer Review, 4(1), 4. https://doi.org/10.1186/s41073-019-0063-9
» https://doi.org/10.1186/s41073-019-0063-9
Saleem, H., Diola, B., & Ella, L. K. (1999). Misconceptions and the Certainty of Response Index (CRI). Physics Education, 34(5), 294. https://doi.org/10.1088/0031-9120/34/5/304
» https://doi.org/10.1088/0031-9120/34/5/304
Savaris, R. F., Pumi, G., Dalzochio, J., & Kunst, R. (2021). Stay-at-home policy is a case of exception fallacy: an internet-based ecological study. Scientific Reports, 11(1), 5313. https://doi.org/10.1038/s41598-021-84092-1
» https://doi.org/10.1038/s41598-021-84092-1
SciELO. (2018). Linhas prioritárias de ação 2019-2023 Disponível em: https://www.scielo20.org/redescielo/wp-content/uploads/sites/2/2018/09/L%C3%ADneas-prioritaris-de-acci%C3%B3n-2019-2023_pt.pdf
SciELO. (2023a). Ensaio Pesquisa em Educação em Ciências (Belo Horizonte). Recuperado em 29 de janeiro, de https://data.scielo.org/dataverse/brepec
» https://data.scielo.org/dataverse/brepec
SciELO. (2023b). SciELO Data Recuperado em 29 de janeiro, de https://data.scielo.org/dataverse/scielodata
» https://data.scielo.org/dataverse/scielodata
SciELO. (2023c). SciELO Livros Recuperado em 29 de janeiro, de https://books.scielo.org
SciELO. (2023d). SciELO Preprints Recuperado em 29 de janeiro, de https://preprints.scielo.org/index.php/scielo
» https://preprints.scielo.org/index.php/scielo
Scott, T. F., Schumayer, D., & Gray, A. R. (2012). Exploratory factor analysis of a Force Concept Inventory data set. Physical Review Special Topics - Physics Education Research, 8(2), 020105. https://doi.org/10.1103/PhysRevSTPER.8.020105
» https://doi.org/10.1103/PhysRevSTPER.8.020105
Silva, F. C. C. d., & Silveira, L. d. (2019). O ecossistema da Ciência Aberta. Transinformação, 31 https://doi.org/10.1590/2318-0889201931e190001
» https://doi.org/10.1590/2318-0889201931e190001
Silveira, F. L., Moreira, M. A., & Axt, R. (1986). Validação de um teste para detectar se o aluno possui a concepção newtoniana sobre força e movimento. Ciência e Cultura, 38(12), 2047-2055.
Simard, M.-A., Ghiasi, G., Mongeon, P., & Larivière, V. (2022). National differences in dissemination and use of open access literature. PLOS ONE, 17(8), e0272730. https://doi.org/10.1371/journal.pone.0272730
» https://doi.org/10.1371/journal.pone.0272730
Smith, A. C., Merz, L., Borden, J. B., Gulick, C. K., Kshirsagar, A. R., & Bruna, E. M. (2022). Assessing the effect of article processing charges on the geographic diversity of authors using Elsevier’s “Mirror Journal” system. Quantitative Science Studies, 2(4), 1123-1143. https://doi.org/10.1162/qss_a_00157
» https://doi.org/10.1162/qss_a_00157
Smith, J. I., & Tanner, K. (2010). The problem of revealing how students think: concept inventories and beyond. CBE Life Sciences Education, 9(1), 1-5. https://doi.org/10.1187/cbe.09-12-0094
» https://doi.org/10.1187/cbe.09-12-0094
Stemler, S. E., & Naples, A. (2021). Rasch Measurement v. Item Response Theory: knowing when to cross the l ine. Practical Assessment, Research, and Evaluation, 26(1), 11. https://doi.org/10.7275/v2gd-4441
» https://doi.org/10.7275/v2gd-4441
Stewart, J., Zabriskie, C., DeVore, S., & Stewart, G. (2018). Multidimensional item response theory and the Force Concept Inventory. Physical Review Physics Education Research, 14(1), 010137. https://doi.org/10.1103/PhysRevPhysEducRes.14.010137
» https://doi.org/10.1103/PhysRevPhysEducRes.14.010137
Symonds, J. E., & Gorard, S. (2010). Death of mixed methods? Or the rebirth of research as a craft. Evaluation & Research in Education, 23(2), 121-136. https://doi.org/10.1080/09500790.2010.483514
» https://doi.org/10.1080/09500790.2010.483514
Timans, R., Wouters, P., & Heilbron, J. (2019). Mixed methods research: what it is and what it could be. Theory and Society, 48(2), 193-216. https://doi.org/10.1007/s11186-019-09345-5
» https://doi.org/10.1007/s11186-019-09345-5
Traxler, A., Henderson, R., Stewart, J., Stewart, G., Papak, A., & Lindell, R. (2018). Gender fairness within the Force Concept Inventory. Physical Review Physics Education Research, 14(1), 010103. https://doi.org/10.1103/PhysRevPhysEducRes.14.010103
» https://doi.org/10.1103/PhysRevPhysEducRes.14.010103
Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science. International Journal of Science Education, 10(2), 159-169. https://doi.org/10.1080/0950069880100204
» https://doi.org/10.1080/0950069880100204
UNESCO. (2019). Launch of the Global Alliance of Open Access Scholarly Communication Platforms to democratize knowledge UNESCO. Recuperado em 29 de janeiro, de https://www.unesco.org/en/articles/launch-global-alliance-open-access-scholarly-communication-platforms-democratize-knowledge
UNESCO. (2022). Recomendação da UNESCO sobre Ciência Aberta https://unesdoc.unesco.org/ark:/48223/pf0000379949_por
» https://unesdoc.unesco.org/ark:/48223/pf0000379949_por
van Dijk, W., Schatschneider, C., & Hart, S. A. (2021). Open Science in education sciences. Journal of Learning Disabilities, 603(7901), 139-152. https://doi.org/10.1177/0022219420945267
» https://doi.org/10.1177/0022219420945267
van Dusen, B., White, J.-S. S., & Roualdes, E. A. (2016, July 20-21). The impact of learning assistants on inequities in physics student outcomes.PER Conference Physics Education Research Conference 2016, Sacramento.
Vosniadou, S. (2012). Reframing the classical approach to conceptual change: preconceptions, misconceptions and synthetic models. In B. J. Fraser, K. Tobin, & C. J. McRobbie (Eds.), Second International Handbook of Science Education (pp. 119-130). Springer Netherlands. https://doi.org/10.1007/978-1-4020-9041-7_10
» https://doi.org/10.1007/978-1-4020-9041-7_10
Wertsch, J. V. (1985). Vygotsky and the social formation of mind Harvard University Press.
Wickman, P.-O., & Östman, L. (2002). Learning as discourse change: A sociocultural mechanism. Science Education, 86(5), 601-623. https://doi.org/10.1002/sce.10036
