To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding

A. J. Beaumont; T. Al-Shaghdari

doi:10.1007/978-3-030-37599-7_31

To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding

A. J. Beaumont^*, T. Al-Shaghdari

^*Corresponding author for this work

Research output: Chapter in Book/Published conference output › Conference publication

Abstract

In this paper we apply supervised machine learning algorithms to
automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning.

Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.

The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training.

We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.

Original language	English
Title of host publication	Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings
Editors	Giuseppe Nicosia, Panos Pardalos, Renato Umeton, Giovanni Giuffrida, Vincenzo Sciacca
Publisher	Springer
Pages	372-383
Number of pages	12
ISBN (Electronic)	978-3-030-37599-7
ISBN (Print)	9783030375980
DOIs	https://doi.org/10.1007/978-3-030-37599-7_31
Publication status	Published - 3 Jan 2020
Event	5th International Conference on Machine Learning, Optimization, and Data Science, LOD 2019 - Siena, Italy Duration: 10 Sept 2019 → 13 Sept 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11943 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	5th International Conference on Machine Learning, Optimization, and Data Science, LOD 2019
Country/Territory	Italy
City	Siena
Period	10/09/19 → 13/09/19

Access to Document

10.1007/978-3-030-37599-7_31

Cite this

Beaumont, A. J., & Al-Shaghdari, T. (2020). To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding. In G. Nicosia, P. Pardalos, R. Umeton, G. Giuffrida, & V. Sciacca (Eds.), Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings (pp. 372-383). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11943 LNCS). Springer. https://doi.org/10.1007/978-3-030-37599-7_31

Beaumont, A. J. ; Al-Shaghdari, T. / To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding. Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings. editor / Giuseppe Nicosia ; Panos Pardalos ; Renato Umeton ; Giovanni Giuffrida ; Vincenzo Sciacca. Springer, 2020. pp. 372-383 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{46689a2b514943008dd57e754805027d,

title = "To What Extent Can Text Classification Help with Making Inferences About Students{\textquoteright} Understanding",

abstract = "In this paper we apply supervised machine learning algorithms to automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning. Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training. We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.",

author = "Beaumont, {A. J.} and T. Al-Shaghdari",

year = "2020",

month = jan,

day = "3",

doi = "10.1007/978-3-030-37599-7_31",

language = "English",

isbn = "9783030375980",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "372--383",

editor = "Giuseppe Nicosia and Panos Pardalos and Renato Umeton and Giovanni Giuffrida and Vincenzo Sciacca",

booktitle = "Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings",

address = "Germany",

note = "5th International Conference on Machine Learning, Optimization, and Data Science, LOD 2019 ; Conference date: 10-09-2019 Through 13-09-2019",

}

Beaumont, AJ & Al-Shaghdari, T 2020, To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding. in G Nicosia, P Pardalos, R Umeton, G Giuffrida & V Sciacca (eds), Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11943 LNCS, Springer, pp. 372-383, 5th International Conference on Machine Learning, Optimization, and Data Science, LOD 2019, Siena, Italy, 10/09/19. https://doi.org/10.1007/978-3-030-37599-7_31

To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding. / Beaumont, A. J.; Al-Shaghdari, T.
Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings. ed. / Giuseppe Nicosia; Panos Pardalos; Renato Umeton; Giovanni Giuffrida; Vincenzo Sciacca. Springer, 2020. p. 372-383 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11943 LNCS).

Research output: Chapter in Book/Published conference output › Conference publication

TY - GEN

T1 - To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding

AU - Beaumont, A. J.

AU - Al-Shaghdari, T.

PY - 2020/1/3

Y1 - 2020/1/3

N2 - In this paper we apply supervised machine learning algorithms to automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning. Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training. We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.

AB - In this paper we apply supervised machine learning algorithms to automatically classify the text of students' reflective learning journals from an introductory Java programming module with the aim of identifying students who need help with their understanding of the topic they are reflecting on. Such a system could alert teaching staff to students who may need an intervention to support their learning. Several different classifier algorithms have been validated on the training data set to find the best model in two situations; with equal cost for a positive or negative classification and with cost sensitive classification. Methods were used to identify those individual parameters which maximise the performance of each algorithm. Precision, recall and F1-score, as well as confusion matrices were used to understand the behaviour of each classifier and choose the one with the best performance.The classifiers that obtained the best results from the validation were then evaluated on a testing data set containing different data to that used for training. We believe that although the results could be improved with further work, our initial results show that machine learning could be applied to students' reflective writing to assist staff in identifying those students who are struggling to understand the topic.

UR - http://www.scopus.com/inward/record.url?scp=85078463688&partnerID=8YFLogxK

UR - https://link.springer.com/chapter/10.1007%2F978-3-030-37599-7_31

U2 - 10.1007/978-3-030-37599-7_31

DO - 10.1007/978-3-030-37599-7_31

M3 - Conference publication

AN - SCOPUS:85078463688

SN - 9783030375980

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 372

EP - 383

BT - Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings

A2 - Nicosia, Giuseppe

A2 - Pardalos, Panos

A2 - Umeton, Renato

A2 - Giuffrida, Giovanni

A2 - Sciacca, Vincenzo

PB - Springer

T2 - 5th International Conference on Machine Learning, Optimization, and Data Science, LOD 2019

Y2 - 10 September 2019 through 13 September 2019

ER -

Beaumont AJ, Al-Shaghdari T. To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding. In Nicosia G, Pardalos P, Umeton R, Giuffrida G, Sciacca V, editors, Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Proceedings. Springer. 2020. p. 372-383. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-37599-7_31

To What Extent Can Text Classification Help with Making Inferences About Students’ Understanding

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Reflective journals: can they be used to identify struggling students?

Cite this