A BIBLIOMETRIC ANALYSIS OF AWE IN ACADEMIC WRITING: INSIGHT FROM THE SCOPUS DATABASE
DOI:
https://doi.org/10.51878/learning.v6i3.11692Keywords:
Evaluasi Tulisan Otomatis, Penulisan Akademik, Analisis BibliometrikAbstract
This study investigates the development and research trends of Automated Writing Evaluation (AWE) in academic writing through a bibliometric approach. A total of 75 documents were retrieved from the Scopus database, covering publications between 2015 and 2025, and were analyzed using VOSviewer. The analysis focused on three dimensions: (1) publication trend analysis to chart the growth of AWE scholarship over time; (2) keyword co-occurrence analysis to identify dominant concepts and thematic clusters; and (3) thematic mapping to reveal emerging research directions and future opportunities. The findings reveal a marked upward trend in AWE research output, with publication volume remaining modest between 2015 and 2021 before rising sharply from 2022 onward, culminating in 22 documents in 2025 alone. Keyword analysis identified "automated writing evaluation" as the central concept (occurrences = 34; Total Link Strength = 96), closely networked with "feedback" (TLS = 38), "artificial intelligence" (TLS = 36), "ChatGPT" (TLS = 35), and "natural language processing" (TLS = 30). Three major thematic clusters were identified: a technology-oriented cluster encompassing AI and NLP tools; a feedback and assessment cluster centered on the pedagogical functions of AWE; and a language learning cluster foregrounding applications in second language (L2) and EFL academic writing contexts. The results further reveal a fundamental transition shifting from conventional automated scoring systems toward more sophisticated, AI-powered formative feedback mechanisms. These findings advance a more structured understanding of the intellectual landscape of AWE research in academic writing and identify concrete directions for future inquiry.
ABSTRAK
Penelitian ini mengkaji perkembangan dan tren penelitian Evaluasi Tulisan Otomatis (Automated Writing Evaluation/AWE) dalam penulisan akademik melalui pendekatan bibliometrik. Sebanyak 75 dokumen diperoleh dari basis data Scopus, mencakup publikasi antara tahun 2015 dan 2025, dan dianalisis menggunakan VOSviewer. Analisis difokuskan pada tiga dimensi: (1) analisis tren publikasi untuk memetakan pertumbuhan penelitian AWE dari waktu ke waktu; (2) analisis ko-kemunculan kata kunci untuk mengidentifikasi konsep dominan dan kluster tematik; dan (3) pemetaan tematik untuk mengungkap arah penelitian yang sedang berkembang dan peluang di masa mendatang. Temuan menunjukkan tren peningkatan signifikan dalam output penelitian AWE, dengan volume publikasi yang tetap moderat antara 2015 dan 2021 sebelum meningkat tajam sejak 2022, dengan puncaknya 22 dokumen pada tahun 2025. Analisis kata kunci mengidentifikasi "automated writing evaluation" sebagai konsep sentral (kemunculan = 34; Total Link Strength = 96), yang terhubung erat dengan "feedback" (TLS = 38), "artificial intelligence" (TLS = 36), "ChatGPT" (TLS = 35), dan "natural language processing" (TLS = 30). Tiga kluster tematik utama teridentifikasi: kluster berorientasi teknologi mencakup alat AI dan NLP; kluster umpan balik dan penilaian berpusat pada fungsi pedagogis AWE; dan kluster pembelajaran bahasa menonjolkan penerapan dalam konteks penulisan akademik bahasa kedua (L2) dan EFL. Hasil penelitian mengungkap transisi mendasar dalam bidang ini, dari sistem penilaian otomatis konvensional menuju mekanisme umpan balik formatif berbasis AI yang lebih canggih. Temuan ini berkontribusi pada pemahaman yang lebih terstruktur tentang lanskap intelektual penelitian AWE dalam penulisan akademik.
Downloads
References
Ahlström, R., & Aldenkvist, A. (2026). Generativ artificiell intelligens: “En mer kunnig annan”? KTH Publication Database DiVA (KTH Royal Institute of Technology). http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-251645
Alghamdy, R. Z. (2023). Pedagogical and ethical implications of artificial intelligence in EFL context: A review study. English Language Teaching, 16(10), 87–87. https://doi.org/10.5539/elt.v16n10p87
Ardiana, A., Rum, E. P., Syatriana, E., & Khodjieva, Z. (2025). From feedback to artificial intelligence: A bibliometric mapping analysis of the thematic evolution of EFL writing assessment research trends (2014–2024). International Journal of Learning, Teaching and Educational Research, 24(9). https://doi.org/10.26803/ijlter.24.9
Baas, J., Schotten, M., Plume, A., Côté, G., & Karimi, R. (2020). Scopus as a curated, high-quality bibliometric data source for academic research in quantitative science studies. Quantitative Science Studies, 1(1), 377–386. https://doi.org/10.1162/qss_a_00019
Barrett, A., & Pack, A. (2023). Not quite eye to A.I.: Student and teacher perspectives on the use of generative artificial intelligence in the writing process. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00427-0
Barrot, J. S. (2023). Trends in automated writing evaluation systems research for teaching, learning, and assessment: A bibliometric analysis. Education and Information Technologies, 29(1), 261–285. https://doi.org/10.1007/s10639-023-12083-y
Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285–296. https://doi.org/10.1016/j.jbusres.2021.04.070
Eaton, C., Belmonte, I., Enaya, T., Flood, S., Khalil, Z., Makwanda, A., Shah, M. M. A., Toma, A., Wang, T., & Yu, C. (2026). Where we’re at, what we must know, and where we can go: A systematic review of research about writing and artificial intelligence. Discourse and Writing/Rédactologie, 35, 89–113. https://doi.org/10.31468/dwr.1167
Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: Insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00425-2
Fu, Q. K., Zou, D., Xie, H., & Cheng, G. (2024). A review of AWE feedback: Types, implementation, and effects. Computer Assisted Language Learning, 37(1–2), 179–221. https://doi.org/10.1080/09588221.2022.2033787
Hyland, K. (2019). Second language writing (2nd ed.). Cambridge University Press.
Jalambo, M. O., Wahab, M. O. A., Ismail, S. M., & Belton, B. (2025). “Assess, adapt, achieve”: Investigating the role of mobile-mediated dynamic assessment in cultivating L2 learners’ writing resilience, strategic revision, and linguistic accuracy. Language Testing in Asia, 15(1). https://doi.org/10.1186/s40468-025-00407-3
Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44, Article 100450. https://doi.org/10.1016/j.asw.2020.100450
Lin, L. (2025). Effectiveness analysis of a personalized English writing feedback system based on advanced language models. Educational Innovation Research, 3(6), 26–32. https://doi.org/10.18063/eir.v3i6.665
Litvinova, T., Panicheva, P., Sboev, A., Rybka, R., & Gudovskikh, D. (2024). Writing in the era of large language models: A bibliometric study. Education and Information Technologies, 29(12), 15283–15318. https://doi.org/10.1007/s10639-024-12590-6
McPhee, S., & Jerowsky, M. (2025). Beyond technical skills: A pedagogical perspective on fostering critical engagement with generative AI in university classrooms. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1593278
Mohammed, S. J., & Khalid, M. (2025). Under the world of AI-generated feedback on writing: Mirroring motivation, foreign language peace of mind, trait emotional intelligence, and writing development. Language Testing in Asia, 15(1). https://doi.org/10.1186/s40468-025-00343-2
Paul, J., & Criado, A. R. (2020). The art of writing literature review: What do we know and what do we need to know? International Business Review, 29(4), Article 101717. https://doi.org/10.1016/j.ibusrev.2020.101717
Perkins, M., Roe, J., & Furze, L. (2024). The AI Assessment Scale revisited: A framework for educational assessment. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2412.09029
Raitskaya, L., & Tikhonova, E. (2020). Pressure to publish internationally: Scholarly writing coming to the fore. Journal of Language and Education, 6(1), 4–7. https://doi.org/10.17323/jle.2020.10631
Salu, M. L., Emilia, E., & Gustine, G. G. (2025). Navigating obstacles in writing scholarly articles for reputable international journal publication. English Review: Journal of English Education, 13(2), 583–592. https://doi.org/10.25134/jfqhv682
Stojanov, A. (2023). Learning with ChatGPT 3.5 as a more knowledgeable other: An autoethnographic study. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00404-7
Tikhonova, E., & Mezentseva, D. A. (2024). Wordiness in academic writing: A systematic scoping review. Research Result: Theoretical and Applied Linguistics, 10(1), 133–157. https://doi.org/10.18413/2313-8912-2024-10-1-0-8
Tran, M., Balasooriya, C., Semmler, C., & Rhee, J. (2025, July 11). Generative artificial intelligence: The ‘more knowledgeable other’ in a social constructivist framework of medical education. npj Digital Medicine, 8(1), Article 430. https://doi.org/10.1038/s41746-025-01823-8
van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-0146-3
Wang, C. (2024). Exploring students’ generative AI-assisted writing processes: Perceptions and experiences from native and nonnative English speakers. Technology, Knowledge and Learning, 30(3), 1825–1846. https://doi.org/10.1007/s10758-024-09744-3
Xiao, Y., Cao, Y., & Zhao, Y. (2024). Automated writing evaluation in Chinese EFL contexts: A study of Pigai.org use and student perceptions. Language Learning & Technology, 28(1), 1–22.
Xu, X., Liu, J., Zhang, Y., & Zhang, H. (2024). Development and implementation of personal learning environment-based writing for publication scaffolding platform for Ph.D. students. Humanities and Social Sciences Communications, 11(1). https://doi.org/10.1057/s41599-024-03094-2
Xue, Y. (2024). Towards automated writing evaluation: A comprehensive review with bibliometric, scientometric, and meta-analytic approaches. Education and Information Technologies, 29(15), 19553–19594. https://doi.org/10.1007/s10639-024-12596-0
Zhang, S. (2025). Automated writing evaluation feedback in English learning and teaching. Antistasis, 13(1), 1–14.
Zhang, Y., Eto, H., & Cui, J. (2025). Linguistic challenges of writing papers in English for scholarly publication: Perceptions of Chinese academics in science and engineering. PLoS ONE, 20(5). https://doi.org/10.1371/journal.pone.0324760
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 LEARNING : Jurnal Inovasi Penelitian Pendidikan dan Pembelajaran
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
