Automatic summary assessment for intelligent tutoring systems

Yulan He, Siu C. Hui, Tho T. Quan

Research output: Contribution to journalArticle

Abstract

Summary writing is an important part of many English Language Examinations. As grading students' summary writings is a very time-consuming task, computer-assisted assessment will help teachers carry out the grading more effectively. Several techniques such as latent semantic analysis (LSA), n-gram co-occurrence and BLEU have been proposed to support automatic evaluation of summaries. However, their performance is not satisfactory for assessing summary writings. To improve the performance, this paper proposes an ensemble approach that integrates LSA and n-gram co-occurrence. As a result, the proposed ensemble approach is able to achieve high accuracy and improve the performance quite substantially compared with current techniques. A summary assessment system based on the proposed approach has also been developed.
Original languageEnglish
Pages (from-to)890-899
Number of pages10
JournalComputers and Education
Volume53
Issue number3
DOIs
Publication statusPublished - Nov 2009

Fingerprint

Intelligent systems
Semantics
grading
semantics
performance
Students
English language
examination
teacher
evaluation
student

Bibliographical note

NOTICE: this is the author’s version of a work that was accepted for publication in Computers and education. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in He, Y, Hui, SC & Quan, TT, 'Automatic summary assessment for intelligent tutoring systems' Computers and education, vol. 53, no. 3 (2009) DOI http://dx.doi.org/10.1016/j.compedu.2009.05.008.

Keywords

  • summary writing assessment
  • intelligent tutoring systems
  • latent semantic analysis
  • N-gram co-occurrence
  • adaptive feedback

Cite this

He, Yulan ; Hui, Siu C. ; Quan, Tho T. / Automatic summary assessment for intelligent tutoring systems. In: Computers and Education. 2009 ; Vol. 53, No. 3. pp. 890-899.
@article{10bac6e5cd384b9a84bb9873204eb698,
title = "Automatic summary assessment for intelligent tutoring systems",
abstract = "Summary writing is an important part of many English Language Examinations. As grading students' summary writings is a very time-consuming task, computer-assisted assessment will help teachers carry out the grading more effectively. Several techniques such as latent semantic analysis (LSA), n-gram co-occurrence and BLEU have been proposed to support automatic evaluation of summaries. However, their performance is not satisfactory for assessing summary writings. To improve the performance, this paper proposes an ensemble approach that integrates LSA and n-gram co-occurrence. As a result, the proposed ensemble approach is able to achieve high accuracy and improve the performance quite substantially compared with current techniques. A summary assessment system based on the proposed approach has also been developed.",
keywords = "summary writing assessment, intelligent tutoring systems, latent semantic analysis, N-gram co-occurrence, adaptive feedback",
author = "Yulan He and Hui, {Siu C.} and Quan, {Tho T.}",
note = "NOTICE: this is the author’s version of a work that was accepted for publication in Computers and education. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in He, Y, Hui, SC & Quan, TT, 'Automatic summary assessment for intelligent tutoring systems' Computers and education, vol. 53, no. 3 (2009) DOI http://dx.doi.org/10.1016/j.compedu.2009.05.008.",
year = "2009",
month = "11",
doi = "10.1016/j.compedu.2009.05.008",
language = "English",
volume = "53",
pages = "890--899",
journal = "Computers and Education",
issn = "0360-1315",
publisher = "Elsevier",
number = "3",

}

Automatic summary assessment for intelligent tutoring systems. / He, Yulan; Hui, Siu C.; Quan, Tho T.

In: Computers and Education, Vol. 53, No. 3, 11.2009, p. 890-899.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Automatic summary assessment for intelligent tutoring systems

AU - He, Yulan

AU - Hui, Siu C.

AU - Quan, Tho T.

N1 - NOTICE: this is the author’s version of a work that was accepted for publication in Computers and education. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in He, Y, Hui, SC & Quan, TT, 'Automatic summary assessment for intelligent tutoring systems' Computers and education, vol. 53, no. 3 (2009) DOI http://dx.doi.org/10.1016/j.compedu.2009.05.008.

PY - 2009/11

Y1 - 2009/11

N2 - Summary writing is an important part of many English Language Examinations. As grading students' summary writings is a very time-consuming task, computer-assisted assessment will help teachers carry out the grading more effectively. Several techniques such as latent semantic analysis (LSA), n-gram co-occurrence and BLEU have been proposed to support automatic evaluation of summaries. However, their performance is not satisfactory for assessing summary writings. To improve the performance, this paper proposes an ensemble approach that integrates LSA and n-gram co-occurrence. As a result, the proposed ensemble approach is able to achieve high accuracy and improve the performance quite substantially compared with current techniques. A summary assessment system based on the proposed approach has also been developed.

AB - Summary writing is an important part of many English Language Examinations. As grading students' summary writings is a very time-consuming task, computer-assisted assessment will help teachers carry out the grading more effectively. Several techniques such as latent semantic analysis (LSA), n-gram co-occurrence and BLEU have been proposed to support automatic evaluation of summaries. However, their performance is not satisfactory for assessing summary writings. To improve the performance, this paper proposes an ensemble approach that integrates LSA and n-gram co-occurrence. As a result, the proposed ensemble approach is able to achieve high accuracy and improve the performance quite substantially compared with current techniques. A summary assessment system based on the proposed approach has also been developed.

KW - summary writing assessment

KW - intelligent tutoring systems

KW - latent semantic analysis

KW - N-gram co-occurrence

KW - adaptive feedback

UR - http://www.scopus.com/inward/record.url?scp=67650232982&partnerID=8YFLogxK

U2 - 10.1016/j.compedu.2009.05.008

DO - 10.1016/j.compedu.2009.05.008

M3 - Article

AN - SCOPUS:67650232982

VL - 53

SP - 890

EP - 899

JO - Computers and Education

JF - Computers and Education

SN - 0360-1315

IS - 3

ER -