Reliability testing the appraisal framework for analysing encouraged suicide data

This paper explores the suitability and replicability of the appraisal framework in the forensic context of analysing encouraged suicide evidence. Owing to critiques of its subjectivity as an analytical method, appraisal may be met with similar hesitation if used to present linguistic evidence in forensic analyses. While Calloway (2023) demonstrates how appraisal may be used to elicit a persuasion response in the context of encouraging suicide, this paper suggests that the appraisal framework in its current form is not reliable enough as a tool to categorise how influential language manifests in the conditions used here. In this preliminary study using 10 independent coders, appraisal analysis returns an unsatisfactory inter-rater reliability score in classifying encouraged suicide data (K-Alpha 0.29), and adds to the limited attention provided to reliability testing the appraisal framework. The findings suggest that the phrasing of the coding scheme, the way data is presented to participants, and participant familiarity with the appraisal system may impact reliability. Furthermore, the criteria of the appraisal framework are not specific to the identification of the presence or absence of encouragement, thereby indicating the need for an alternative framework for this task.