Evaluating the Efficacy of ChatGPT for Grammar Assessment and Personalized Feedback

Nur Aeni; Muhammad Tahir; Andi Sahtiani Jahrir

doi:10.53682/eclue.v13i2.12757

Authors

Nur Aeni Universitas Negeri Makassar
Muhammad Tahir Universitas Negeri Makassar
Andi Sahtiani Jahrir Universitas Negeri Makassar

DOI:

https://doi.org/10.53682/eclue.v13i2.12757

Keywords:

AI Chatbot ChaptGPT Personalized feedback Grammar Assessment

Abstract

This research investigates the role of the AI chatbot ChatGPT in grammar assessment and personalized feedback for second-grade students at SMKN 10 Makassar. It aims to evaluate ChatGPT’s accuracy in assessing students’ grammatical competence, the effectiveness of its feedback in enhancing grammar skills, and students’ perceptions of its usefulness as a grammar learning tool. Using a mixed-method explanatory sequential design, quantitative data were collected through a structured questionnaire involving 46 students, while qualitative insights were gained through interviews with six purposively selected participants. The findings revealed that a majority of students perceived ChatGPT as accurate and consistent in identifying grammatical errors, particularly appreciating the clarity and specificity of its feedback. Furthermore, the data indicated that ChatGPT contributed to improved grammar understanding and motivation, especially through personalized explanations and error correction. Qualitative responses supported these findings, highlighting students’ increased confidence and engagement in grammar learning. While some neutral responses suggested room for refinement in feedback fairness and bias perception, the overall results demonstrate ChatGPT’s potential as an effective supplementary tool in grammar instruction. This research contributes to the growing literature on AI-assisted language learning and offers practical implications for integrating technology into grammar teaching at the junior high school level.

References

Aeni, N., Winahyu, S. K., Muthmainnah, M., Mansyur, U., Syamsidar, Izmi, S. A., & Padli, F. (2024, September). An Analysis of Jenni Speak: How AI Can Improve Speaking Skills in Indonesian Higher Education?. In International Conference on Artificial Intelligence and Networking (pp. 291-301). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-96-2015-9_20

Aeni, N., Alsbbagh, Y. A. M. R., Taufik, M., Patandung, Y., Kurnia, R., Muthmainnah, ... & Al Yakin, A. (2024, June). AI Chatbots in Oral Communication for Higher Education Students: Threats or Opportunities. In International Conference on Data Analytics & Management (pp. 147-164). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-96-3358-6_12

Ali, A. M. (2023). An Intervention Study on the Use of Artificial Intelligence in the ESL Classroom: English teacher perspectives on the Effectiveness of ChatGPT for Personalized Language LearningEn.

Belmekki, M. (2023). Grammar competence and writing performance: A correlational study in higher education. TESOL and Technology Studies, 4(1), 37–44. https://doi.org/10.48185/tts.v4i1.594.

Bitchener, J., & Ferris, D. R. (2012). Written Corrective Feedback in Second Language Acquisition and Writing. Routledge.

Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1), 1-8. https://doi.org/10.59429/ajde.v18i1.621

Brown, H. D. (2014). Principles of language learning and teaching (6th ed.). Pearson Education, Inc.

Chen, X., Xie, H., & Hwang, G. J. (2020). A multi-perspective review of artificial intelligence in education. Education and Information Technologies, 25(4), 3361–3393.

Creswell, J. W., & Creswell, J. D. (2023). Research design: Qualitative, quantitative, and mixed methods approaches (6th ed.). SAGE Publications.

Chhabriya, M., Babu, N. N., & Kumari, A. S. P. (2024, November). AI’s Corrective Feedback vs. Traditional Recasts: which fosters better grammar accuracy? In 2024 2nd DMIHER International Conference on Artificial Intelligence in Healthcare, Education and Industry (IDICAIEI) (pp. 1-6). IEEE.

Ellis, R. (1989). Understanding second language acquisition (Vol. 31). Oxford: Oxford university press.

Ellis, R. (2006). The Study of Second Language Acquisition. Oxford University Press.

Følstad, A., & Brandtzæg, P. B. (2017). Chatbots and the new world of HCI. Interactions, 24(4), 38–42.

Holmes, N. G., et al. (2023). AI in education: The promise and the practice. Nature, 620(7976), 951-952.

Huang, J., Chen, L., & Zhao, M. (2020). Personalized feedback in digital learning: Enhancing student engagement and learning outcomes. Computers & Education, 149, 103834.

Huang, X., Zou, D., Cheng, G., Chen, X., & Xie, H. (2023). Trends, research issues and applications of artificial intelligence in language education. Educational Technology & Society, 26(1), 112-125. https://doi.org/10.30191/ETS.202301_26(1).0009

Kasneci, E., Sessler, K., Kühl, N., et al. (2023). ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education. Learning and Individual Differences, 101, 102274.

Krashen, S. D. (1982). Principles and Practice in Second Language Acquisition. Pergamon.

Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence Unleashed: An Argument for AI in Education. Pearson Education.

Mills, G. E., & Gay, L. R. (2019). Educational research: Competencies for analysis and applications (12th ed.). Pearson.

Muhayyang, M., Asriati, & Sakkir, G. (2023). Pengantar dasar-dasar penelitian pendidikan. CV Cendekiawan Indonesia Timur.

Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041

OpenAI. (2023). ChatGPT: Optimizing language models for dialogue.

OpenAI. (2023). ChatGPT Technical Overview. https://openai.com

Perkins, M., Roe, J., Postma, L., McGaughran, J., & Hickey, D. (2023). AI and the future of academic integrity: A crisis of assessment. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2023.2262911

Piaget, J. (1954). The Construction of Reality in the Child. Basic Books.

Rahayu, N., & Hidayat, R. (2023). Integrating Chatbots in Junior High School English Lessons: A Case from Indonesia. Journal of Technology in Language Education, 4(2), 55–64.

Rahmawati, D., & Ertin. (2023). The Effectiveness of Contextual Grammar Teaching in Indonesian EFL Classrooms. ELT-Edu Journal, 6(1), 1-15

Sahito, Z. H., Sahito, F. Z., & Imran, M. (2024). The Role of Artificial Intelligence (AI) in Personalized Learning: A Case Study in K-12 Education. Global Educational Studies Review, IX(III), 153-163. https://doi.org/10.31703/gesr.2024(IX-III).15

Tomlinson, C. A. (2001). How to Differentiate Instruction in Mixed-Ability Classrooms. ASCD.

Wang, X., Young, G. W., Plechatá, A., Mc Guckin, C., & Makransky, G. (2023). Utilizing virtual reality to assist social competence education and social support for children from under-represented backgrounds. Computers & Education, 201, 104815. https://doi.org/10.1016/j.compedu.2023.104815

Yu, S., Wong, B. E., & Poulova, P. (2025). Artificial Intelligence in Second Language Acquisition: Enhancing Teacher Self-Efficacy Through Tacit and Explicit Knowledge Sharing. Journal of Artificial Intelligence and Technology.

Zhang, Q., Nie, H., Fan, J., & Liu, H. (2025). Exploring the dynamics of artificial intelligence literacy on english as a foreign language learners’ willingness to communicate: The critical mediating roles of artificial intelligence learning self-efficacy and classroom anxiety. Behavioral Sciences, 15(4), 523. https://doi.org/10.3390/bs15040523