Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework

Roger Howell, Shun H Wong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In the acquisition of software-development skills, feedback that pinpoints errors and explains means of improvement is important in achieving a good student learning experience. However, it is not feasible to manually provide timely, consistent, and helpful feedback for large or complex coursework tasks, and/or to large cohorts of students. While tools exist to provide feedback to student submissions, their automation is typically limited to reporting either test pass or failure or generating feedback to very simple programming tasks. Anecdotal experience indicates that clusters of students tend to make similar mistakes and/or successes within their coursework. Do feedback comments applied to students' work support this claim and, if so, to what extent is this the case? How might this be exploited to improve the assessment process and the quality of feedback given to students? To help answer these questions, we have examined feedback given to coursework submissions to a UK level 5, university-level, data structures and algorithms course to determine heuristics used to trigger particular feedback comments that are common between submissions and cohorts. This paper reports our results and discusses how the identified heuristics may be used to promote timeliness and consistency of feedback without jeopardising the quality.
Original languageEnglish
Title of host publicationProceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018
EditorsMark J.W. Lee, Sasha Nikolic, Gary K.W. Wong, Jun Shen, Montserrat Ros, Leon C. U. Lei, Neelakantam Venkatarayalu
PublisherIEEE
Pages286-293
Number of pages8
ISBN (Electronic)978-1-5386-6522-0
ISBN (Print)978-1-5386-6523-7
DOIs
Publication statusPublished - 16 Jan 2019
Event2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) - Wollongong, Australia
Duration: 4 Dec 20187 Dec 2018

Publication series

Name2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)
PublisherIEEE
ISSN (Print)2374-0191
ISSN (Electronic)2470-6698

Conference

Conference2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)
CountryAustralia
CityWollongong
Period4/12/187/12/18

Fingerprint

Computer programming
heuristics
programming
Feedback
Students
student
software development
automation
Data structures
Software engineering
experience
Automation
university

Bibliographical note

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Keywords

  • computer aided feedback
  • coursework assessment
  • static analysis
  • technology-enhanced learning

Cite this

Howell, R., & Wong, S. H. (2019). Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework. In M. J. W. Lee, S. Nikolic, G. K. W. Wong, J. Shen, M. Ros, L. C. U. Lei, & N. Venkatarayalu (Eds.), Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018 (pp. 286-293). [8615128] (2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)). IEEE. https://doi.org/10.1109/TALE.2018.8615128
Howell, Roger ; Wong, Shun H. / Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework. Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018. editor / Mark J.W. Lee ; Sasha Nikolic ; Gary K.W. Wong ; Jun Shen ; Montserrat Ros ; Leon C. U. Lei ; Neelakantam Venkatarayalu. IEEE, 2019. pp. 286-293 (2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)).
@inproceedings{c899be8abb4d41fd88660e08a75037f1,
title = "Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework",
abstract = "In the acquisition of software-development skills, feedback that pinpoints errors and explains means of improvement is important in achieving a good student learning experience. However, it is not feasible to manually provide timely, consistent, and helpful feedback for large or complex coursework tasks, and/or to large cohorts of students. While tools exist to provide feedback to student submissions, their automation is typically limited to reporting either test pass or failure or generating feedback to very simple programming tasks. Anecdotal experience indicates that clusters of students tend to make similar mistakes and/or successes within their coursework. Do feedback comments applied to students' work support this claim and, if so, to what extent is this the case? How might this be exploited to improve the assessment process and the quality of feedback given to students? To help answer these questions, we have examined feedback given to coursework submissions to a UK level 5, university-level, data structures and algorithms course to determine heuristics used to trigger particular feedback comments that are common between submissions and cohorts. This paper reports our results and discusses how the identified heuristics may be used to promote timeliness and consistency of feedback without jeopardising the quality.",
keywords = "computer aided feedback, coursework assessment, static analysis, technology-enhanced learning",
author = "Roger Howell and Wong, {Shun H}",
note = "{\circledC} 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.",
year = "2019",
month = "1",
day = "16",
doi = "10.1109/TALE.2018.8615128",
language = "English",
isbn = "978-1-5386-6523-7",
series = "2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)",
publisher = "IEEE",
pages = "286--293",
editor = "Lee, {Mark J.W.} and Sasha Nikolic and Wong, {Gary K.W.} and Jun Shen and Montserrat Ros and Lei, {Leon C. U.} and Neelakantam Venkatarayalu",
booktitle = "Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018",
address = "United States",

}

Howell, R & Wong, SH 2019, Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework. in MJW Lee, S Nikolic, GKW Wong, J Shen, M Ros, LCU Lei & N Venkatarayalu (eds), Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018., 8615128, 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), IEEE, pp. 286-293, 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), Wollongong, Australia, 4/12/18. https://doi.org/10.1109/TALE.2018.8615128

Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework. / Howell, Roger; Wong, Shun H.

Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018. ed. / Mark J.W. Lee; Sasha Nikolic; Gary K.W. Wong; Jun Shen; Montserrat Ros; Leon C. U. Lei; Neelakantam Venkatarayalu. IEEE, 2019. p. 286-293 8615128 (2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework

AU - Howell, Roger

AU - Wong, Shun H

N1 - © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2019/1/16

Y1 - 2019/1/16

N2 - In the acquisition of software-development skills, feedback that pinpoints errors and explains means of improvement is important in achieving a good student learning experience. However, it is not feasible to manually provide timely, consistent, and helpful feedback for large or complex coursework tasks, and/or to large cohorts of students. While tools exist to provide feedback to student submissions, their automation is typically limited to reporting either test pass or failure or generating feedback to very simple programming tasks. Anecdotal experience indicates that clusters of students tend to make similar mistakes and/or successes within their coursework. Do feedback comments applied to students' work support this claim and, if so, to what extent is this the case? How might this be exploited to improve the assessment process and the quality of feedback given to students? To help answer these questions, we have examined feedback given to coursework submissions to a UK level 5, university-level, data structures and algorithms course to determine heuristics used to trigger particular feedback comments that are common between submissions and cohorts. This paper reports our results and discusses how the identified heuristics may be used to promote timeliness and consistency of feedback without jeopardising the quality.

AB - In the acquisition of software-development skills, feedback that pinpoints errors and explains means of improvement is important in achieving a good student learning experience. However, it is not feasible to manually provide timely, consistent, and helpful feedback for large or complex coursework tasks, and/or to large cohorts of students. While tools exist to provide feedback to student submissions, their automation is typically limited to reporting either test pass or failure or generating feedback to very simple programming tasks. Anecdotal experience indicates that clusters of students tend to make similar mistakes and/or successes within their coursework. Do feedback comments applied to students' work support this claim and, if so, to what extent is this the case? How might this be exploited to improve the assessment process and the quality of feedback given to students? To help answer these questions, we have examined feedback given to coursework submissions to a UK level 5, university-level, data structures and algorithms course to determine heuristics used to trigger particular feedback comments that are common between submissions and cohorts. This paper reports our results and discusses how the identified heuristics may be used to promote timeliness and consistency of feedback without jeopardising the quality.

KW - computer aided feedback

KW - coursework assessment

KW - static analysis

KW - technology-enhanced learning

UR - https://ieeexplore.ieee.org/document/8615128

UR - http://www.scopus.com/inward/record.url?scp=85062059520&partnerID=8YFLogxK

U2 - 10.1109/TALE.2018.8615128

DO - 10.1109/TALE.2018.8615128

M3 - Conference contribution

SN - 978-1-5386-6523-7

T3 - 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)

SP - 286

EP - 293

BT - Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018

A2 - Lee, Mark J.W.

A2 - Nikolic, Sasha

A2 - Wong, Gary K.W.

A2 - Shen, Jun

A2 - Ros, Montserrat

A2 - Lei, Leon C. U.

A2 - Venkatarayalu, Neelakantam

PB - IEEE

ER -

Howell R, Wong SH. Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework. In Lee MJW, Nikolic S, Wong GKW, Shen J, Ros M, Lei LCU, Venkatarayalu N, editors, Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018. IEEE. 2019. p. 286-293. 8615128. (2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)). https://doi.org/10.1109/TALE.2018.8615128