Temporal and external validation of a prediction model for adverse outcomes among inpatients with diabetes

N. J. Adderley; S. Mallett; T. Marshall; S. Ghosh; G. Rayman; S. Bellary; J. Coleman; F. Akiboye; K. A. Toulis; K. Nirantharakumar

doi:10.1111/dme.13612

Temporal and external validation of a prediction model for adverse outcomes among inpatients with diabetes

N. J. Adderley
, S. Mallett
, T. Marshall
, S. Ghosh
, G. Rayman
, S. Bellary
, J. Coleman
, F. Akiboye
, K. A. Toulis
, K. Nirantharakumar^*

^*Corresponding author for this work

University College Birmingham
University Hospitals Birmingham NHS Foundation Trust
Ipswich Hospital NHS Trust
424 General Military Hospital

Research output: Contribution to journal › Article › peer-review

7 Citations (Scopus)

Abstract

Aim: To temporally and externally validate our previously developed prediction model, which used data from University Hospitals Birmingham to identify inpatients with diabetes at high risk of adverse outcome (mortality or excessive length of stay), in order to demonstrate its applicability to other hospital populations within the UK. Methods: Temporal validation was performed using data from University Hospitals Birmingham and external validation was performed using data from both the Heart of England NHS Foundation Trust and Ipswich Hospital. All adult inpatients with diabetes were included. Variables included in the model were age, gender, ethnicity, admission type, intensive therapy unit admission, insulin therapy, albumin, sodium, potassium, haemoglobin, C-reactive protein, estimated GFR and neutrophil count. Adverse outcome was defined as excessive length of stay or death. Results: Model discrimination in the temporal and external validation datasets was good. In temporal validation using data from University Hospitals Birmingham, the area under the curve was 0.797 (95% CI 0.785–0.810), sensitivity was 70% (95% CI 67–72) and specificity was 75% (95% CI 74–76). In external validation using data from Heart of England NHS Foundation Trust, the area under the curve was 0.758 (95% CI 0.747–0.768), sensitivity was 73% (95% CI 71-74) and specificity was 66% (95% CI 65–67). In external validation using data from Ipswich, the area under the curve was 0.736 (95% CI 0.711–0.761), sensitivity was 63% (95% CI 59–68) and specificity was 69% (95% CI 67–72). These results were similar to those for the internally validated model derived from University Hospitals Birmingham. Conclusions: The prediction model to identify patients with diabetes at high risk of developing an adverse event while in hospital performed well in temporal and external validation. The externally validated prediction model is a novel tool that can be used to improve care pathways for inpatients with diabetes. Further research to assess clinical utility is needed.

Original language	English
Pages (from-to)	798-806
Number of pages	9
Journal	Diabetic medicine
Volume	35
Issue number	6
Early online date	27 Feb 2018
DOIs	https://doi.org/10.1111/dme.13612
Publication status	Published - Jun 2018

Funding

N.J.A. was funded by Diabetes UK (grant number 15/ 0005281). N.J.A. and T.M. are supported by the National Institute for Health Research Collaborations for Leadership in Applied Health Research and Care for West Midlands initiative. This paper presents independent research and the views expressed are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health. N.A. and K.N. report a grant from Diabetes UK during the conduct of the study. N.A. and T.M. were supported by the National Institute for Health Research Collaborations for Leadership in Applied Health Research and Care for West Midlands during the conduct of the study. K.N. has received personal fees from Sanofi and a grant from AstraZeneca outside the submitted work. S.B. reports grants and personal fees from Novo Nordisk and Boehringer Ingelheim, and personal fees from Janssen, Takeda and AstraZeneca outside the submitted work. All other authors declare no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Access to Document

10.1111/dme.13612

https://research.birmingham.ac.uk/portal/en/publications/temporal-and-external-validation-of-a-prediction-model-for-adverse-outcomes-among-inpatients-with-diabetes(ce120579-9c70-4aab-9468-d35d0e687caa).html

Cite this

@article{3849f301c1254d0c9d43495bf85be5ef,

title = "Temporal and external validation of a prediction model for adverse outcomes among inpatients with diabetes",

abstract = "Aim: To temporally and externally validate our previously developed prediction model, which used data from University Hospitals Birmingham to identify inpatients with diabetes at high risk of adverse outcome (mortality or excessive length of stay), in order to demonstrate its applicability to other hospital populations within the UK. Methods: Temporal validation was performed using data from University Hospitals Birmingham and external validation was performed using data from both the Heart of England NHS Foundation Trust and Ipswich Hospital. All adult inpatients with diabetes were included. Variables included in the model were age, gender, ethnicity, admission type, intensive therapy unit admission, insulin therapy, albumin, sodium, potassium, haemoglobin, C-reactive protein, estimated GFR and neutrophil count. Adverse outcome was defined as excessive length of stay or death. Results: Model discrimination in the temporal and external validation datasets was good. In temporal validation using data from University Hospitals Birmingham, the area under the curve was 0.797 (95\% CI 0.785–0.810), sensitivity was 70\% (95\% CI 67–72) and specificity was 75\% (95\% CI 74–76). In external validation using data from Heart of England NHS Foundation Trust, the area under the curve was 0.758 (95\% CI 0.747–0.768), sensitivity was 73\% (95\% CI 71-74) and specificity was 66\% (95\% CI 65–67). In external validation using data from Ipswich, the area under the curve was 0.736 (95\% CI 0.711–0.761), sensitivity was 63\% (95\% CI 59–68) and specificity was 69\% (95\% CI 67–72). These results were similar to those for the internally validated model derived from University Hospitals Birmingham. Conclusions: The prediction model to identify patients with diabetes at high risk of developing an adverse event while in hospital performed well in temporal and external validation. The externally validated prediction model is a novel tool that can be used to improve care pathways for inpatients with diabetes. Further research to assess clinical utility is needed.",

author = "Adderley, \{N. J.\} and S. Mallett and T. Marshall and S. Ghosh and G. Rayman and S. Bellary and J. Coleman and F. Akiboye and Toulis, \{K. A.\} and K. Nirantharakumar",

year = "2018",

month = jun,

doi = "10.1111/dme.13612",

language = "English",

volume = "35",

pages = "798--806",

journal = "Diabetic medicine",

issn = "0742-3071",

publisher = "Wiley-Blackwell",

number = "6",

}

TY - JOUR

T1 - Temporal and external validation of a prediction model for adverse outcomes among inpatients with diabetes

AU - Adderley, N. J.

AU - Mallett, S.

AU - Marshall, T.

AU - Ghosh, S.

AU - Rayman, G.

AU - Bellary, S.

AU - Coleman, J.

AU - Akiboye, F.

AU - Toulis, K. A.

AU - Nirantharakumar, K.

PY - 2018/6

Y1 - 2018/6

N2 - Aim: To temporally and externally validate our previously developed prediction model, which used data from University Hospitals Birmingham to identify inpatients with diabetes at high risk of adverse outcome (mortality or excessive length of stay), in order to demonstrate its applicability to other hospital populations within the UK. Methods: Temporal validation was performed using data from University Hospitals Birmingham and external validation was performed using data from both the Heart of England NHS Foundation Trust and Ipswich Hospital. All adult inpatients with diabetes were included. Variables included in the model were age, gender, ethnicity, admission type, intensive therapy unit admission, insulin therapy, albumin, sodium, potassium, haemoglobin, C-reactive protein, estimated GFR and neutrophil count. Adverse outcome was defined as excessive length of stay or death. Results: Model discrimination in the temporal and external validation datasets was good. In temporal validation using data from University Hospitals Birmingham, the area under the curve was 0.797 (95% CI 0.785–0.810), sensitivity was 70% (95% CI 67–72) and specificity was 75% (95% CI 74–76). In external validation using data from Heart of England NHS Foundation Trust, the area under the curve was 0.758 (95% CI 0.747–0.768), sensitivity was 73% (95% CI 71-74) and specificity was 66% (95% CI 65–67). In external validation using data from Ipswich, the area under the curve was 0.736 (95% CI 0.711–0.761), sensitivity was 63% (95% CI 59–68) and specificity was 69% (95% CI 67–72). These results were similar to those for the internally validated model derived from University Hospitals Birmingham. Conclusions: The prediction model to identify patients with diabetes at high risk of developing an adverse event while in hospital performed well in temporal and external validation. The externally validated prediction model is a novel tool that can be used to improve care pathways for inpatients with diabetes. Further research to assess clinical utility is needed.

AB - Aim: To temporally and externally validate our previously developed prediction model, which used data from University Hospitals Birmingham to identify inpatients with diabetes at high risk of adverse outcome (mortality or excessive length of stay), in order to demonstrate its applicability to other hospital populations within the UK. Methods: Temporal validation was performed using data from University Hospitals Birmingham and external validation was performed using data from both the Heart of England NHS Foundation Trust and Ipswich Hospital. All adult inpatients with diabetes were included. Variables included in the model were age, gender, ethnicity, admission type, intensive therapy unit admission, insulin therapy, albumin, sodium, potassium, haemoglobin, C-reactive protein, estimated GFR and neutrophil count. Adverse outcome was defined as excessive length of stay or death. Results: Model discrimination in the temporal and external validation datasets was good. In temporal validation using data from University Hospitals Birmingham, the area under the curve was 0.797 (95% CI 0.785–0.810), sensitivity was 70% (95% CI 67–72) and specificity was 75% (95% CI 74–76). In external validation using data from Heart of England NHS Foundation Trust, the area under the curve was 0.758 (95% CI 0.747–0.768), sensitivity was 73% (95% CI 71-74) and specificity was 66% (95% CI 65–67). In external validation using data from Ipswich, the area under the curve was 0.736 (95% CI 0.711–0.761), sensitivity was 63% (95% CI 59–68) and specificity was 69% (95% CI 67–72). These results were similar to those for the internally validated model derived from University Hospitals Birmingham. Conclusions: The prediction model to identify patients with diabetes at high risk of developing an adverse event while in hospital performed well in temporal and external validation. The externally validated prediction model is a novel tool that can be used to improve care pathways for inpatients with diabetes. Further research to assess clinical utility is needed.

UR - https://www.scopus.com/pages/publications/85044210218

UR - https://onlinelibrary.wiley.com/doi/abs/10.1111/dme.13612

U2 - 10.1111/dme.13612

DO - 10.1111/dme.13612

M3 - Article

C2 - 29485723

AN - SCOPUS:85044210218

SN - 0742-3071

VL - 35

SP - 798

EP - 806

JO - Diabetic medicine

JF - Diabetic medicine

IS - 6

ER -

Temporal and external validation of a prediction model for adverse outcomes among inpatients with diabetes

Abstract

Funding

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this