Test-retest reliability for common tasks in vision science

Kait Clark*, Kayley Birch-Hurst, Charlotte Rebecca Pennington, Austin C. P. Petrie, Joshua T. Lee, Craig Hedge

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Research in perception and attention has typically sought to evaluate cognitive mechanisms according to the average response to a manipulation. Recently, there has been a shift toward appreciating the value of individual differences and the insight gained by exploring the impacts of between-participant variation on human cognition. However, a recent study suggests that many robust, well-established cognitive control tasks suffer from surprisingly low levels of test-retest reliability (Hedge et al., 2018b). We tested a large sample of undergraduate students (n = 160) in two sessions (separated by 1–3 weeks) on four commonly used tasks in vision science. We implemented measures that spanned a range of perceptual and attentional processes, including motion coherence (MoCo), useful field of view (UFOV), multiple-object tracking (MOT), and visual working memory (VWM). Intraclass correlations ranged from good to poor suggesting that some task measures are more suitable for assessing individual differences than others. VWM capacity (ICC = 0.77), MoCo threshold (ICC = 0.60), UFOV middle accuracy (ICC = 0.60) and UFOV outer accuracy (ICC = 0.74) showed good-to-excellent reliability. Other measures, namely the maximum number of items tracked in MOT (ICC = 0.41) and UFOV number accuracy (ICC = 0.48), showed moderate reliability; the MOT threshold (ICC = 0.36) and UFOV inner accuracy (ICC = 0.30) showed poor reliability. In this paper, we present these results alongside a summary of reliabilities estimated previously for other vision science tasks. We then offer useful recommendations for evaluating test-retest reliability when considering a task for use in evaluating individual differences.
Original languageEnglish
JournalJournal of Vision
DOIs
Publication statusAccepted/In press - 24 May 2022

Keywords

  • individual differences
  • perception
  • attention
  • visual cognition

Fingerprint

Dive into the research topics of 'Test-retest reliability for common tasks in vision science'. Together they form a unique fingerprint.

Cite this