Length and readability of structured software engineering abstracts

Attempts to perform systematic literature reviews have identified a problem with the quality of software engineering abstracts for papers describing empirical studies. Structured abstracts have been found useful for improving the quality of abstracts in many other disciplines. However, there have been no studies of the value of structured abstracts in software engineering. Therefore this paper aims to assess the comparative length and readability of unstructured abstracts and structured versions of the same abstract. Abstracts were obtained from all empirical conference papers from the Evaluation and Assessment in Software Engineering Conference (EASE04 and EASE06) that did not have a structured abstract (23 in total). Two novice researchers created structured versions of the abstracts, which were checked by the papers' authors (or a surrogate). Web tools were used to extract the length in words and readability in terms of the Flesch reading ease index and automated readability index (ARI) for the structured and unstructured abstracts. The structured abstracts were on average 142.5 words longer than the unstructured abstracts (p<0.001). The readability of the structured abstracts was better by 8.5 points on the Flesch index (p<0.001) and 1.8 points on the ARI (p<0.003). The results are consistent with previous studies, although the increase in length and the increase in readability are both greater than the previous studies. Future work will consider whether structured abstracts increase the content and quality of abstracts.