AUTOMATED ASSESSMENT OF COMPUTER NETWORK KNOWLEDGE: SHORT ANSWER GRADING THROUGH DEEP LEARNING AND CONTEXT META-LEARNING

Abstract

Abstract. The field of automatic short-answer grading (ASAG) is essential for intelligent tutoring systems and has the potential to revolutionize education by providing quick and unbiased results, saving teachers time and effort. However, current techniques using pre-existing language models that have been trained on general language data may not be appropriate for educational subject domains and may not be easily scalable for large-scale use. To address this issue, this paper proposes a framework that fine-tunes pre-trained BERT, Roberta, Distil, and Deberta models specifically for question answering with meta-learning. The input is carefully designed for an in-context learning approach, leading to scalable solutions for computer network questions in the computer engineering domain. In order to assess the effectiveness of the proposed framework for ASAG, we had created a dataset of student’s 1800 responses for 300 questions by 6 students obtained from an online exam. The dataset was then graded by subject matter experts. This allowed for the evaluation of the performance of the framework in comparison to human grading, which is considered the gold standard in ASAG. The framework was evaluated using specified dataset comprising actual responses from student responses and achieved good results, with the QA-Distil method performing well in terms of AUC=0.892, RMSE=0.507, and Kappa=0.636. ¹¹