To What Extent Can Text Classification Help with Making Inferences About Students' Understanding

Anthony J Beaumont, Twba Al-Shaghdari

Research output: Contribution to journalConference article

Abstract

In this paper we apply supervised machine learning algorithms to
automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning.

Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.

The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training.

We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.
Original languageEnglish
Number of pages12
JournalLecture Notes in Computer Science
Publication statusAccepted/In press - 30 Jun 2019

Fingerprint

Text Classification
Classifier
Students
Machine Learning
Classifiers
Costs
Supervised Learning
Learning systems
Java
Learning Algorithm
Programming
Choose
Maximise
Classify
Module
Testing
Learning algorithms
Teaching
Learning
Training

Cite this

@article{46689a2b514943008dd57e754805027d,
title = "To What Extent Can Text Classification Help with Making Inferences About Students' Understanding",
abstract = "In this paper we apply supervised machine learning algorithms to automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning. Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training. We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.",
author = "Beaumont, {Anthony J} and Twba Al-Shaghdari",
year = "2019",
month = "6",
day = "30",
language = "English",
journal = "Lecture Notes in Computer Science",
issn = "0302-9743",
publisher = "Springer",

}

To What Extent Can Text Classification Help with Making Inferences About Students' Understanding. / Beaumont, Anthony J; Al-Shaghdari, Twba.

In: Lecture Notes in Computer Science, 30.06.2019.

Research output: Contribution to journalConference article

TY - JOUR

T1 - To What Extent Can Text Classification Help with Making Inferences About Students' Understanding

AU - Beaumont, Anthony J

AU - Al-Shaghdari, Twba

PY - 2019/6/30

Y1 - 2019/6/30

N2 - In this paper we apply supervised machine learning algorithms to automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning. Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training. We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.

AB - In this paper we apply supervised machine learning algorithms to automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning. Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training. We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.

M3 - Conference article

JO - Lecture Notes in Computer Science

JF - Lecture Notes in Computer Science

SN - 0302-9743

ER -