Smart data-driven medical decisions through collective and individual anomaly detection in healthcare time series

Farbod Khanizadeh, Alireza Ettefaghian, George Wilson, Amirali Shirazibeheshti, Tarek Radwan, Cristina Luca

Research output: Contribution to journalArticlepeer-review

Abstract

Background: Anomalies in healthcare refer to deviation from the norm of unusual or unexpected patterns or activities related to patients, diseases or medical centres. Detecting these anomalies is crucial for timely interventions and efficient decision-making, helping to identify issues like operational inefficiencies, fraud and emerging health complications. Objectives: This study presents a novel method for detecting both collective and individual anomalies in healthcare data through time series analysis using unsupervised machine learning. The dual-strategy approach leverages two methodologies: a ’practice centre-based approach’ which monitors changes across different practice centres and a ’process-based approach’ which focuses on identifying anomalies within individual centres. The former allows for early detection of systemic issues, while the latter highlights specific irregularities within a centre's operations. Methods: The study utilised a dataset over 500,000 medical records from multiple GP practice centres in the UK collected between 2018–2023. Data are clustered using DBSCAN to identify collective anomalies from deviations from linear trends in consecutive two-month scatterplots. Individual anomalies are identified by examining the SOM-clustered time series of various medical processes within a specific practice centre, where graphs show deviation from the typical pattern. Findings: Our approach addresses some challenges posed by the complexity and sensitivity of healthcare data by not requiring personal information. The method offers accurate visual representations making the data accessible and interpretable for non-technical users. Unlike traditional methods focusing solely on subsequence anomalies, our technique analyses the collective behaviour across multiple time series providing a more comprehensive perspective. Conclusion: This study underscores the importance of integrating unsupervised anomaly detection with clinical expertise to ensure that statistically anomalous patterns align with clinical relevance. The dual-strategy clustering method holds significant potential for enabling timely interventions, proactively identifying potential crises, and ultimately contributing to better decision-making and operational efficiency within the healthcare sector.

Original languageEnglish
Article number105696
Number of pages12
JournalInternational Journal of Medical Informatics
Volume194
Early online date17 Nov 2024
DOIs
Publication statusPublished - Feb 2025

Bibliographical note

Copyright © 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (https://creativecommons.org/licenses/by/4.0/).

Keywords

  • Anomaly Detection
  • Unsupervised learning
  • Outliers
  • Health forecasting
  • Time series
  • Decision making

Fingerprint

Dive into the research topics of 'Smart data-driven medical decisions through collective and individual anomaly detection in healthcare time series'. Together they form a unique fingerprint.

Cite this