Avaliar Examinar um trabalho escrito avaliado na universidade: comparação entre a avaliação humana e a avaliação por um modelo de inteligência artificial generativa
##plugins.themes.bootstrap3.article.sidebar##
##plugins.themes.bootstrap3.article.main##
Resumo
A inteligência artificial generativa (IAG) pode apoiar o juízo avaliativo exercido pelo corpo docente sobre produções estudantis. No entanto, a sua utilização levanta questões que vão além, nomeadamente, da concordância das classificações. Para compreender melhor as condições de tal apoio por parte da IAG, este artigo apresenta uma experimentação exploratória realizada num curso universitário de mestrado em domínio do francês escrito. O estudo compara a avaliação humana e a avaliação pela IAG de uma mesma produção escrita com função diagnóstica. Um quadro de referência do juízo avaliativo estruturado em três eixos sustenta a análise: as funções da avaliação, as suas etapas e a sua dimensão ética, indissociável da responsabilidade da pessoa avaliadora. Os resultados mostram uma convergência global entre os dois modos de avaliação no que respeita às notas e ao número de erros identificados, mas revelam desvios significativos na identificação, na categorização dos erros e no encadeamento entre diagnóstico, classificação e retroação quando a complexidade dos erros aumenta. Este estudo demonstra que a IAG não pode substituir o juízo avaliativo. Contudo, constitui um apoio para certas operações do processo, desde que enquadrado por balizas metodológicas, institucionais e éticas claras, associadas a uma formação do corpo docente e a uma vigilância profissional constante. Abre igualmente perspetivas de investigação, entre outras, no sentido da comparação de diferentes modelos generativos.
Downloads
##plugins.themes.bootstrap3.article.details##
Este trabalho encontra-se publicado com a Licença Internacional Creative Commons Atribuição-NãoComercial-CompartilhaIgual 4.0.
Referências
Ahangama, N. (2026). Designing assessments in the generative AI era: A tailored assessment framework for ICT tertiary education. International Journal of Educational Technology in Higher Education, 23(9). https://doi.org/10.1186/s41239-026-00582-0 DOI: https://doi.org/10.1186/s41239-026-00582-0
Álvarez Méndez, J. M. (2001). Evaluar para conocer, examinar para excluir. Morata.
An, Y., Yu, J. H. et James, S. (2025). Investigating the higher education institutions’ guidelines and policies regarding the use of generative AI in teaching, learning, research, and administration. International Journal of Educational Technology in Higher Education, 22(10). https://doi.org/10.1186/s41239-025-00507-3 DOI: https://doi.org/10.1186/s41239-025-00507-3
Anistyasari, Y., Hidayati, S. C., Suparji, S., Ekohariadi, E. et Kusumaningtyas, D. A. (2025). Comparing AI and human assessment of academic writing skills: A kappa analysis. E3S Web of Conferences, 645. https://doi.org/10.1051/e3sconf/202564506014 DOI: https://doi.org/10.1051/e3sconf/202564506014
Audran, J. (2024). Cinq enjeux d’évaluation face à l’émergence des IA génératives en éducation. Mesure et évaluation en éducation, 47(1), 6-26. https://doi.org/10.7202/1114564ar DOI: https://doi.org/10.7202/1114564ar
Bardin, L. (2013). L’analyse de contenu. Presses Universitaires de France. https://doi.org/10.3917/puf.bard.2013.01 DOI: https://doi.org/10.3917/puf.bard.2013.01
Bittle, K. et El Gayar, O. (2025). Generative AI and academic integrity in higher education: A systematic review and research agenda. Information, 16(4), 296. https://doi.org/10.3390/info16040296 DOI: https://doi.org/10.3390/info16040296
Black, P. et Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. https://doi.org/10.1080/0969595980050102 DOI: https://doi.org/10.1080/0969595980050102
Black, P. et Wiliam, D. (2009) Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21, 5-31. https://doi.org/10.1007/s11092-008-9068-5 DOI: https://doi.org/10.1007/s11092-008-9068-5
Bloom, B. S. (1971). Handbook on formative and summative evaluation of student learning. McGraw-Hill.
Broadfoot, P. (1996). Education, assessment and society: A sociological analysis. Open University Press.
Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3-12. https://doi.org/10.1111/j.1745-3992.2010.00195.x DOI: https://doi.org/10.1111/j.1745-3992.2010.00195.x
Brookhart, S. M. (2013). Classroom assessment in the context of standards-based reform. Teachers College Record, 115(11), 1-16. DOI: https://doi.org/10.1177/016146811311500805
Dabis, A., et Csáki, C. (2024). AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI. Humanities and Social Sciences Communications, 11, 1006. https://doi.org/10.1057/s41599-024-03526-z DOI: https://doi.org/10.1057/s41599-024-03526-z
De Ketele, J.-M. (1986). L’évaluation : approche descriptive ou normative? De Boeck.
De Ketele, J.-M. (2006). La recherche en évaluation : propos synthétiques et prospectifs. Mesure et évaluation en éducation, 29(1), 99-118. https://doi.org/10.7202/1086970a DOI: https://doi.org/10.7202/1086970ar
De Ketele, J.-M. et Gérard, F.-M. (2007). La qualité et la validation des évaluations. De Boeck.
De Ketele, J.-M. (2010a). Ne pas se tromper d'évaluation. Revue française de linguistique appliquée, 15(1), 25-37. https://doi.org/10.3917/rfla.151.0025 DOI: https://doi.org/10.3917/rfla.151.0025
De Ketele, J.-M. (2010b). L’évaluation : une problématique complexe. De Boeck.
Collin, S. et Marceau, E. (2023). Enjeux éthiques et critiques de l’intelligence artificielle en enseignement supérieur. Éthique publique, 24(2). https://doi.org/10.4000/ethiquepublique.7619 DOI: https://doi.org/10.4000/ethiquepublique.7619
Conseil de l’innovation du Québec (2024). L’intelligence artificielle générative : comprendre, encadrer et agir. Gouvernement du Québec. https://www.ethique.gouv.qc.ca/publications/iagenerative/
Elshall, A. S. et Badir, A. (2025). Balancing AI-assisted learning and traditional assessment: The FACT assessment in environmental data science education. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1596462 DOI: https://doi.org/10.3389/feduc.2025.1596462
Enseigner à l’UQAM. (s. d.). La démarche évaluative et les valeurs qui s’y rattachent. https://enseigner.uqam.ca/evaluer/demarche-et-valeurs/
García-Varela, F., Nussbaum, M., Mendoza, M., Martínez-Troncoso, C. et Bekerman, Z. (2025). ChatGPT as a Stable and Fair Tool for Automated Essay Scoring. Education Sciences, 15(8), 946. https://doi.org/10.3390/educsci15080946 DOI: https://doi.org/10.3390/educsci15080946
Geçkin, V., Kızıltaş, E. et Çınar, Ç. (2023). Assessing second-language academic writing: AI vs. human raters. Journal of Educational Technology & Online Learning, 6(4), 1096-1108. http://doi.org/10.31681/jetol.1336599 DOI: https://doi.org/10.31681/jetol.1336599
Ghimire, A., Edwards, J. (2024). From Guidelines to Governance: A Study of AI Policies in Education. Dans Olney, A.M., Chounta, IA., Liu, Z., Santos, O.C., Bittencourt, I.I. (dir.) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2024. Communications in Computer and Information Science (vol. 2151, p.299-307). Springer, Cham. https://doi.org/10.1007/978-3-031-64312-5_36 DOI: https://doi.org/10.1007/978-3-031-64312-5_36
Gipps, C. V. (1994). Beyond testing: Towards a theory of educational assessment. Falmer Press.
Gipps, C. V. (1999). Socio-cultural aspects of assessment. Review of Research in Education, 24, 355-392. DOI: https://doi.org/10.2307/1167274
Hadji, C. (1997a). L’évaluation démystifiée. ESF.
Hadji, C. (1997b). Pour une éthique de l’agir évaluationnel. Mesure et évaluation en éducation, 20(2), 7-26. https://doi.org/10.7202/1091380ar DOI: https://doi.org/10.7202/1091380ar
Hadji, C. (2012). Faut-il avoir peur de l’évaluation? De Boeck. DOI: https://doi.org/10.3917/dbu.hadji.2012.01
Hannachi, R. et Araújo, S. (2025). Vers une évaluation humaine et automatisée de l’écrit avec l’intelligence artificielle générative. Les Langues modernes, 119(1). https://hal.science/hal-05220332v1
Huba, M. E. et Freed, J. E. (2000). Learner-centered assessment on college campuses. Allyn & Bacon.
Ilieva, G., Yankova, T., Ruseva, M. et Kabaivanov, S. (2025). A Framework for Generative AI-Driven Assessment in Higher Education. Information, 16(6), 472. https://doi.org/10.3390/info16060472 DOI: https://doi.org/10.3390/info16060472
Jorro, A. et Mercier-Brunel, Y. (2022). Évaluation. Dans A. Jorro (dir.), Dictionnaire des concepts de la professionnalisation (p. 197-200). De Boeck Supérieur. DOI: https://doi.org/10.3917/dbu.jorro.2022.01.0197
Kemal, S. et Liman Kaban, A. (2024). Comparative Analysis of Human Graders and AI in Assessing Secondary School EFL Journal Writing. Asian Journal of Distance Education, 20(1). https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/796
Kılınç, S. (2024). Comprehensive AI assessment framework: Enhancing educational evaluation with ethical AI integration. Journal of Educational Technology & Online Learning, 7(4), 521-540. https://doi.org/10.31681/jetol.1492695 DOI: https://doi.org/10.31681/jetol.1492695
Koo, T. K. et Li, M. Y. (2016). A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine, 15(2), 155-163. https://jcm.kglmeridian.com/view/journals/jcm/15/2/article-p155.xml DOI: https://doi.org/10.1016/j.jcm.2016.02.012
Leroux, J. L. et Bélair, L. (2015). Exercer son jugement professionnel. Dans J. L. Leroux (dir.), Évaluer les compétences (p. 65-107). Chenelière.
Li, H., Chen, C. H., Fan, K., Young-Johnson, C., Lim, S. ET Feng, Y. (2025). Agreement between large language models and human raters in essay scoring: A research synthesis. arXiv. https://doi.org/10.48550/arXiv.2512.14561
Lund, B. D., Lee, T. H., Mannuru, N. R., et Arutla, N. (2025). AI and academic integrity: Exploring student perceptions and implications for higher education. Journal of Academic Ethics, 23, 1545-1565. https://doi.org/10.1007/s10805-025-09613-3 DOI: https://doi.org/10.1007/s10805-025-09613-3
Lundgren, M. (2024). Large Language Models in Student Assessment: Comparing ChatGPT and Human Graders. arXiv. https://doi.org/10.48550/arXiv.2406.16510 DOI: https://doi.org/10.2139/ssrn.4874359
McDonald, N., Johri, A., Ali, A., Hingle, A., 2024.Generative artificial intelligence in higher education: Evidence from an analysis of institutional policies and guidelines. Computers in Human Behavior: Artificial Humans, 3. https://doi.org/10.1016/j.chbah.2025.100121 DOI: https://doi.org/10.1016/j.chbah.2025.100121
Messick, S. (1989). Validity. Dans R. L. Linn (dir.), Educational measurement (3e éd., p. 13-103). Macmillan.
Messick, S. (1994). The Interplay of Evidence and Consequences in the Validation of Performance Assessments. Educational Researcher, 23(2), 13-23. https://doi.org/10.3102/0013189X023002013 DOI: https://doi.org/10.3102/0013189X023002013
Microsoft (2024). Microsoft Copilot. https://www.microsoft.com/copilot
Miles, M. et Huberman, A. (2009). Analyse des données qualitatives. De Boeck.
Mpolomoka, D. L. (2025). Utilizing artificial intelligence for assessment. Pedagogical Research, 10(3). https://doi.org/10.29333/pr/16677 DOI: https://doi.org/10.29333/pr/16677
OpenAI. (2024). GPT 4 technical report. ArXiv. https://arxiv.org/abs/2303.08774
Organisation de coopération et de développement économiques (OCDE). (2021). Recommandation du Conseil sur l’intelligence artificielle. OCDE. https://legalinstruments.oecd.org/fr/instruments/OECD-LEGAL-0449
Page, E. B. (1968). The use of the computer in analyzing student essays. International Review of Education, 14, 210-225. https://doi.org/10.1007/BF01419938. DOI: https://doi.org/10.1007/BF01419938
Perrenoud, P. (1998). L’évaluation des élèves : de la fabrication de l’excellence à la régulation des apprentissages. De Boeck Université.
Perrenoud, P. (2004). Évaluer des compétences. De Boeck.
Peterson, S. (2025). Addressing student use of generative AI in schools and universities through academic integrity reporting. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1610836 DOI: https://doi.org/10.3389/feduc.2025.1610836
Posit team (2025). RStudio: Integrated Development Environment for R. Posit Software, PBC, Boston, MA. URL http://www.posit.co/
Roblez, A. (2025, 25 février). Intelligence évaluative et évaluation artificielle : réflexions sur la relation entre l’IA et l’évaluation [communication orale]. Webinaire du Réseau de la pédagogie, Institut catholique de Lille. https://hal.science/hal-04967764v1
Santos Guerra, M. A. (2010). La evaluación como aprendizaje. Kairós.
Scallon, G. (2015). Des savoirs aux compétences : exploration en évaluation des apprentissages. De Boeck.
Scriven, M. (1967). The methodology of evaluation. Dans R. W. Tyler, R. M. Gagne et M. Scriven (dir.), Perspectives of curriculum evaluation (p. 39-83). Rand McNally.
Scriven, M. (1991). Evaluation thesaurus (4e éd.). Sage.
Steiss, J., Tate, Tamara., Graham, S., Gruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., Olsan, C.B. (2024). Comparing human and ChatGPT feedback. Learning and Instruction, 91. https://doi.org/10.1016/j.learninstruc.2024.101894 DOI: https://doi.org/10.1016/j.learninstruc.2024.101894
Tremblay, C., Miklohoun, S. et Poellhuber, B. (2025). Recension des usages d’intelligences artificielles génératives (IAg) pour offrir de la rétroaction en enseignement supérieur. International Journal of Technologies in Higher Education, 22(1). https://doi.org/10.18162/ritpu-2025-v22n1-11 DOI: https://doi.org/10.18162/ritpu-2025-v22n1-11
UNESCO (2023). Guidance for generative AI in education and research. UNESCO.
Wanlin, P. (2007). L’analyse de contenu comme méthode d’analyse qualitative d’entretiens : une comparaison entre les traitements manuels et l’utilisation de logiciels. Recherches qualitatives, hors série, 3, 243-272. https://www.erudit.org/en/books/collection-hors-serie-les-actes-de-la-revue-recherches-qualitatives/bilan-et-prospectives-de-la-recherche-qualitative-en-sciences-humaines-et-sociales/5175co/
Zhao, J., Chapman, E. et Ghassemi Pour Sabet, P. (2024). Generative AI and educational assessments: A systematic review. Education Research & Perspectives, 51(31), 124-155. https://doi.org/10.70953/ERPv51.2412006 DOI: https://doi.org/10.70953/ERPv51.2412006