Utilizing Retrieval Augmented Generation (RAG)-Based Chatbots as an Innovative Learning Tool in Higher Education: A Case Study on the Use of Digital Learning Resources
Article Sidebar
Large Language Model,
Learning Systems,
Retrieval Augmented Generation,
Technology Acceptance
Main Article Content
Yusza Murti
Dian Puteri Ramadhani
Herry Irawan
Abstract
Background: Higher education institutions face information fragmentation and cognitive overload as students struggle to access learning materials scattered across multiple digital platforms. Large Language Models encounter hallucination issues and static knowledge bases, limiting their educational application.
Aims: This research aims to design and evaluate a Retrieval Augmented Generation (RAG)-based chatbot system integrated with Learning Management Systems to address information fragmentation in educational environments
Methods: We employed Design Science Research Methodology (DSRM) to develop the RAG-based chatbot. Technology Acceptance Model (TAM) assessment with 267 undergraduate students and semi-structured interviews with five faculty members evaluate user acceptance and pedagogical perspectives.
Results: The RAG chatbot achieved strong initial user acceptance (mean score 4.097). Students valued perceived usefulness over ease of use, with high usage intentions and recommendation willingness. Faculty recognized pedagogical value while emphasizing quality assurance needs.
Conclusion: This exploratory study demonstrates technical feasibility and baseline user acceptance for RAG-based chatbots in education, showing promise for addressing information accessibility challenges. Demonstration-based evaluation requires validation through longitudinal field studies.
Downloads
Article Details
Copyright (c) 2025 Yusza Murti, Dian Puteri Ramadhani, Herry Irawan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
Aleven, V., Baraniuk, R., Brunskill, E., Crossley, S., Demszky, D., Fancsali, S., Gupta, S., Koedinger, K., Piech, C., Ritter, S., Thomas, D. R., Woodhead, S., & Xing, W. (2023). Towards the Future of AI-Augmented Human Tutoring in Math Learning (N. Wang, G. Rebolledo-Mendez, V. Dimitrova, N. Matsuda, & O. C. Santos, Eds.). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-36336-8
Alnagrat, A. J. A., Ahmed, K. M. S., Alkhallas, M. I., Almakhzoom, O. A. I., Syed Idrus, S. Z., & Che Ismail, R. (2023). Virtual Laboratory Learning Experience in Engineering: An Extended Technology Acceptance Model (TAM). Proceeding - 2023 IEEE 3rd International Maghreb Meeting of the Conference on Sciences and Techniques of Automatic Control and Computer Engineering, MI-STA 2023, 474–479. https://doi.org/10.1109/MI-STA57575.2023.10169123
Al-Hail, M., Zguir, M. F., & Koç, M. (2024). Exploring Digital Learning Opportunities and Challenges in Higher Education Institutes: Stakeholder Analysis on the Use of Social Media for Effective Sustainability of Learning–Teaching–Assessment in a University Setting in Qatar. Sustainability 2024, Vol. 16, Page 6413, 16(15), 6413. https://doi.org/10.3390/SU16156413
Bygstad, B., Øvrelid, E., Ludvigsen, S., & Dæhlen, M. (2022). From dual digitalization to digital learning space: Exploring the digital transformation of higher education. Computers and Education, 182. https://doi.org/10.1016/j.compedu.2022.104463
Chen, K. F., Hwang, G. J., & Chen, M. R. A. (2024). Effects of a concept mapping-guided virtual laboratory learning approach on students’ science process skills and behavioral patterns. Educational Technology Research and Development, 72(3), 1623–1651. https://doi.org/10.1007/s11423-024-10348-y
Chen, X., Xie, H., Zou, D., & Hwang, G. J. (2020). Application and theory gaps during the rise of Artificial Intelligence in Education. In Computers and Education: Artificial Intelligence (Vol. 1). Elsevier B.V. https://doi.org/10.1016/j.caeai.2020.100002
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T. S., & Li, Q. (2024). A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 24, 6491–6501. https://doi.org/10.1145/3637528.3671470
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. Proceedings - 2024 Conference on AI, Science, Engineering, and Technology, AIxSET 2024, 166–169. https://doi.org/10.1109/AIxSET62544.2024.00030
Goyibova, N., Muslimov, N., Sabirova, G., Kadirova, N., & Samatova, B. (2025). Differentiation approach in education: Tailoring instruction for diverse learner needs. In MethodsX (Vol. 14). Elsevier B.V. https://doi.org/10.1016/j.mex.2025.103163
Hwang, G. J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence, 1, 100001. https://doi.org/10.1016/J.CAEAI.2020.100001
Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28(12), 15873–15892. https://doi.org/10.1007/s10639-023-11834-1
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of Hallucination in Natural Language Generation. In ACM Computing Surveys (Vol. 55, Issue 12). Association for Computing Machinery. https://doi.org/10.1145/3571730
Kaklij, V. A., Kunal, M., Shah, V., & Umakant Mandawkar, M. (2019). Microlearning based content-curation using Artificial Intelligence for Learning Experience Platform: A Survey. International Journal of Research and Analytical Reviews, 6 (4), 580-584. https://ssrn.com/abstract=3676951
Klesel, M., & Wittmann, H. F. (2025). Retrieval-Augmented Generation (RAG). Business and Information Systems Engineering, 1-11. https://doi.org/10.1007/s12599-025-00945-3
Lauri, L., Virkus, S., & Heidmets, M. (2020). Information cultures and strategies for coping with information overload: case of Estonian higher education institutions. Journal of Documentation, 77(2), 518–541. https://doi.org/10.1108/JD-08-2020-0143
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2021). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, 33, 9459-9474. http://arxiv.org/abs/2005.11401
Li, Z., Wang, Z., Wang, W., Hung, K., Xie, H., & Wang, F. L. (2025). Retrieval-augmented generation for educational application: A systematic survey. Computers and Education: Artificial Intelligence, 8, 100417. https://doi.org/10.1016/J.CAEAI.2025.100417
Perković, G., Drobnjak, A., & Botički, I. (2024). Hallucinations in LLMs: Understanding and Addressing Challenges. 2024 47th ICT and Electronics Convention, MIPRO 2024 - Proceedings, 2084–2088. https://doi.org/10.1109/MIPRO60963.2024.10569238
Rahmawati, R. N., & Narsa, I. M. (2019). Intention to Use e-Learning: Aplikasi Technology Acceptance Model (TAM). Owner, 3(2), 260. https://doi.org/10.33395/owner.v3i2.151
Ramayani, C., Zainuddin, S. A. B., Said, N. B. M., Samudra, A. A., Areva, D., Harini, G., Ronald, J., & Selvia, N. (2023). Application of Technology Acceptance Model (TAM) in the Adoption of Accounting Information System (AIS) Among Indonesia Private Universities. In Contributions to Management Science: Vol. Part F1060 (pp. 419–428). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-27296-7_38
Shahrzadi, L., Mansouri, A., Alavi, M., & Shabani, A. (2024). Causes, consequences, and strategies to deal with information overload: A scoping review. International Journal of Information Management Data Insights, 4(2). https://doi.org/10.1016/j.jjimei.2024.100261
Swacha, J., & Gracel, M. (2025). Retrieval-Augmented Generation (RAG) Chatbots for Education: A Survey of Applications. In Applied Sciences (Switzerland) (Vol. 15, Issue 8). Multidisciplinary Digital Publishing Institute (MDPI). https://doi.org/10.3390/app15084234
Upadhyaya, P., & Vrinda. (2021). Impact of technostress on academic productivity of university students. Education and Information Technologies, 26(2), 1647–1664. https://doi.org/10.1007/s10639-020-10319-9
Wan, Y., Chen, Z., Liu, Y., Chen, C., & Packianather, M. (2025). Empowering LLMs by hybrid retrieval-augmented generation for domain-centric Q&A in smart manufacturing. Advanced Engineering Informatics, 65, 103212. https://doi.org/10.1016/J.AEI.2025.103212
Weng, F., Yang, R. J., Ho, H. J., & Su, H. M. (2018). A tam-based study of the attitude towards use intention of multimedia among school teachers. Applied System Innovation, 1(3), 1–9. https://doi.org/10.3390/asi1030036
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024). Explainability for Large Language Models: A Survey. ACM Transactions on Intelligent Systems and Technology, 15(2). https://doi.org/10.1145/3639372