Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation

Amirta Benning, Maisoon Ghaleb, Anu Suokas, Mary Dixon-Woods, Jeremy Dawson, Nick Barber, Bryony D. Franklin, Alan Girling, Karla Hemming, Martin Carmalt, Gavin Rudge, Thirumalai Naicker, Ugochi Nwulu, Sopna Choudhury, Richard Lilford*

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

Objectives: To conduct an independent evaluation of the first phase of the Health Foundation's Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design: Mixed method evaluation involving five substudies, before and after design. Setting: NHS hospitals in United Kingdom. Participants: Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1); 18 control hospitals. Intervention: The SPI1 was a compound (multicomponent) organisational intervention delivered over 18 months that focused on improving the reliability of specific frontline care processes in designated clinical specialties and promoting organisational and cultural change. Results: Senior staff members were knowledgeable and enthusiastic about SPI1. There was a small (0.08 points on a 5 point scale) but significant (P<0.01) effect in favour of the SPI1 hospitals in one of 11 dimensions of the staff questionnaire (organisational climate). Qualitative evidence showed only modest penetration of SPI1 at medical ward level. Although SPI1 was designed to engage staff from the bottom up, it did not usually feel like this to those working on the wards, and questions about legitimacy of some aspects of SPI1 were raised. Of the five components to identify patients at risk of deterioration - monitoring of vital signs (14 items); routine tests (three items); evidence based standards specific to certain diseases (three items); prescribing errors (multiple items from the British National Formulary); and medical history taking (11 items) - there was little net difference between control and SPI1 hospitals, except in relation to quality of monitoring of acute medical patients, which improved on average over time across all hospitals. Recording of respiratory rate increased to a greater degree in SPI1 than in control hospitals; in the second six hours after admission recording increased from 40% (93) to 69% (165) in control hospitals and from 37% (141) to 78% (296) in SPI1 hospitals (odds ratio for "difference in difference" 2.1, 99% confidence interval 1.0 to 4.3; P=0.008). Use of a formal scoring system for patients with pneumonia also increased over time (from 2% (102) to 23% (111) in control hospitals and from 2% (170) to 9% (189) in SPI1 hospitals), which favoured controls and was not significant (0.3, 0.02 to 3.4; P=0.173). There were no improvements in the proportion of prescription errors and no effects that could be attributed to SPI1 in non-targeted generic areas (such as enhanced safety culture). On some measures, the lack of effect could be because compliance was already high at baseline (such as use of steroids in over 85% of cases where indicated), but even when there was more room for improvement (such as in quality of medical history taking), there was no significant additional net effect of SPI1. There were no changes over time or between control and SPI1 hospitals in errors or rates of adverse events in patients in medical wards. Mortality increased from 11% (27) to 16% (39) among controls and decreased from17%(63) to13%(49) among SPI1 hospitals, but the risk adjusted difference was not significant (0.5, 0.2 to 1.4; P=0.085). Poor care was a contributing factor in four of the 178 deaths identified by review of case notes. The survey of patients showed no significant differences apart from an increase in perception of cleanliness in favour of SPI1 hospitals. Conclusions The introduction of SPI1 was associated with improvements in one of the types of clinical process studied (monitoring of vital signs) and one measure of staff perceptions of organisational climate. There was no additional effect of SPI1 on other targeted issues nor on other measures of generic organisational strengthening.

Original languageEnglish
Article numberd195
Number of pages14
JournalBMJ
Volume342
Issue number7793
DOIs
Publication statusPublished - 12 Feb 2011

Fingerprint

Patient Safety
Medical History Taking
Vital Signs
Safety Management
Organizational Innovation
Illegitimacy
Pharmacopoeias
Respiratory Rate
Compliance
Prescriptions
Pneumonia

Bibliographical note

​C​r​e​a​t​i​v​e ​C​o​m​m​o​n​s​: ​A​t​t​r​i​b​u​t​i​o​n​-​N​o​n​C​o​m​m​e​r​c​i​a​l

Cite this

Benning, A., Ghaleb, M., Suokas, A., Dixon-Woods, M., Dawson, J., Barber, N., ... Lilford, R. (2011). Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ, 342(7793), [d195]. https://doi.org/10.1136/bmj.d195
Benning, Amirta ; Ghaleb, Maisoon ; Suokas, Anu ; Dixon-Woods, Mary ; Dawson, Jeremy ; Barber, Nick ; Franklin, Bryony D. ; Girling, Alan ; Hemming, Karla ; Carmalt, Martin ; Rudge, Gavin ; Naicker, Thirumalai ; Nwulu, Ugochi ; Choudhury, Sopna ; Lilford, Richard. / Large scale organisational intervention to improve patient safety in four UK hospitals : mixed method evaluation. In: BMJ. 2011 ; Vol. 342, No. 7793.
@article{b12670373aa847679ddb4a6049524d22,
title = "Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation",
abstract = "Objectives: To conduct an independent evaluation of the first phase of the Health Foundation's Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design: Mixed method evaluation involving five substudies, before and after design. Setting: NHS hospitals in United Kingdom. Participants: Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1); 18 control hospitals. Intervention: The SPI1 was a compound (multicomponent) organisational intervention delivered over 18 months that focused on improving the reliability of specific frontline care processes in designated clinical specialties and promoting organisational and cultural change. Results: Senior staff members were knowledgeable and enthusiastic about SPI1. There was a small (0.08 points on a 5 point scale) but significant (P<0.01) effect in favour of the SPI1 hospitals in one of 11 dimensions of the staff questionnaire (organisational climate). Qualitative evidence showed only modest penetration of SPI1 at medical ward level. Although SPI1 was designed to engage staff from the bottom up, it did not usually feel like this to those working on the wards, and questions about legitimacy of some aspects of SPI1 were raised. Of the five components to identify patients at risk of deterioration - monitoring of vital signs (14 items); routine tests (three items); evidence based standards specific to certain diseases (three items); prescribing errors (multiple items from the British National Formulary); and medical history taking (11 items) - there was little net difference between control and SPI1 hospitals, except in relation to quality of monitoring of acute medical patients, which improved on average over time across all hospitals. Recording of respiratory rate increased to a greater degree in SPI1 than in control hospitals; in the second six hours after admission recording increased from 40{\%} (93) to 69{\%} (165) in control hospitals and from 37{\%} (141) to 78{\%} (296) in SPI1 hospitals (odds ratio for {"}difference in difference{"} 2.1, 99{\%} confidence interval 1.0 to 4.3; P=0.008). Use of a formal scoring system for patients with pneumonia also increased over time (from 2{\%} (102) to 23{\%} (111) in control hospitals and from 2{\%} (170) to 9{\%} (189) in SPI1 hospitals), which favoured controls and was not significant (0.3, 0.02 to 3.4; P=0.173). There were no improvements in the proportion of prescription errors and no effects that could be attributed to SPI1 in non-targeted generic areas (such as enhanced safety culture). On some measures, the lack of effect could be because compliance was already high at baseline (such as use of steroids in over 85{\%} of cases where indicated), but even when there was more room for improvement (such as in quality of medical history taking), there was no significant additional net effect of SPI1. There were no changes over time or between control and SPI1 hospitals in errors or rates of adverse events in patients in medical wards. Mortality increased from 11{\%} (27) to 16{\%} (39) among controls and decreased from17{\%}(63) to13{\%}(49) among SPI1 hospitals, but the risk adjusted difference was not significant (0.5, 0.2 to 1.4; P=0.085). Poor care was a contributing factor in four of the 178 deaths identified by review of case notes. The survey of patients showed no significant differences apart from an increase in perception of cleanliness in favour of SPI1 hospitals. Conclusions The introduction of SPI1 was associated with improvements in one of the types of clinical process studied (monitoring of vital signs) and one measure of staff perceptions of organisational climate. There was no additional effect of SPI1 on other targeted issues nor on other measures of generic organisational strengthening.",
author = "Amirta Benning and Maisoon Ghaleb and Anu Suokas and Mary Dixon-Woods and Jeremy Dawson and Nick Barber and Franklin, {Bryony D.} and Alan Girling and Karla Hemming and Martin Carmalt and Gavin Rudge and Thirumalai Naicker and Ugochi Nwulu and Sopna Choudhury and Richard Lilford",
note = "​C​r​e​a​t​i​v​e ​C​o​m​m​o​n​s​: ​A​t​t​r​i​b​u​t​i​o​n​-​N​o​n​C​o​m​m​e​r​c​i​a​l",
year = "2011",
month = "2",
day = "12",
doi = "10.1136/bmj.d195",
language = "English",
volume = "342",
journal = "BMJ",
issn = "0959-8138",
publisher = "BMJ Publishing Group",
number = "7793",

}

Benning, A, Ghaleb, M, Suokas, A, Dixon-Woods, M, Dawson, J, Barber, N, Franklin, BD, Girling, A, Hemming, K, Carmalt, M, Rudge, G, Naicker, T, Nwulu, U, Choudhury, S & Lilford, R 2011, 'Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation', BMJ, vol. 342, no. 7793, d195. https://doi.org/10.1136/bmj.d195

Large scale organisational intervention to improve patient safety in four UK hospitals : mixed method evaluation. / Benning, Amirta; Ghaleb, Maisoon; Suokas, Anu; Dixon-Woods, Mary; Dawson, Jeremy; Barber, Nick; Franklin, Bryony D.; Girling, Alan; Hemming, Karla; Carmalt, Martin; Rudge, Gavin; Naicker, Thirumalai; Nwulu, Ugochi; Choudhury, Sopna; Lilford, Richard.

In: BMJ, Vol. 342, No. 7793, d195, 12.02.2011.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Large scale organisational intervention to improve patient safety in four UK hospitals

T2 - mixed method evaluation

AU - Benning, Amirta

AU - Ghaleb, Maisoon

AU - Suokas, Anu

AU - Dixon-Woods, Mary

AU - Dawson, Jeremy

AU - Barber, Nick

AU - Franklin, Bryony D.

AU - Girling, Alan

AU - Hemming, Karla

AU - Carmalt, Martin

AU - Rudge, Gavin

AU - Naicker, Thirumalai

AU - Nwulu, Ugochi

AU - Choudhury, Sopna

AU - Lilford, Richard

N1 - ​C​r​e​a​t​i​v​e ​C​o​m​m​o​n​s​: ​A​t​t​r​i​b​u​t​i​o​n​-​N​o​n​C​o​m​m​e​r​c​i​a​l

PY - 2011/2/12

Y1 - 2011/2/12

N2 - Objectives: To conduct an independent evaluation of the first phase of the Health Foundation's Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design: Mixed method evaluation involving five substudies, before and after design. Setting: NHS hospitals in United Kingdom. Participants: Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1); 18 control hospitals. Intervention: The SPI1 was a compound (multicomponent) organisational intervention delivered over 18 months that focused on improving the reliability of specific frontline care processes in designated clinical specialties and promoting organisational and cultural change. Results: Senior staff members were knowledgeable and enthusiastic about SPI1. There was a small (0.08 points on a 5 point scale) but significant (P<0.01) effect in favour of the SPI1 hospitals in one of 11 dimensions of the staff questionnaire (organisational climate). Qualitative evidence showed only modest penetration of SPI1 at medical ward level. Although SPI1 was designed to engage staff from the bottom up, it did not usually feel like this to those working on the wards, and questions about legitimacy of some aspects of SPI1 were raised. Of the five components to identify patients at risk of deterioration - monitoring of vital signs (14 items); routine tests (three items); evidence based standards specific to certain diseases (three items); prescribing errors (multiple items from the British National Formulary); and medical history taking (11 items) - there was little net difference between control and SPI1 hospitals, except in relation to quality of monitoring of acute medical patients, which improved on average over time across all hospitals. Recording of respiratory rate increased to a greater degree in SPI1 than in control hospitals; in the second six hours after admission recording increased from 40% (93) to 69% (165) in control hospitals and from 37% (141) to 78% (296) in SPI1 hospitals (odds ratio for "difference in difference" 2.1, 99% confidence interval 1.0 to 4.3; P=0.008). Use of a formal scoring system for patients with pneumonia also increased over time (from 2% (102) to 23% (111) in control hospitals and from 2% (170) to 9% (189) in SPI1 hospitals), which favoured controls and was not significant (0.3, 0.02 to 3.4; P=0.173). There were no improvements in the proportion of prescription errors and no effects that could be attributed to SPI1 in non-targeted generic areas (such as enhanced safety culture). On some measures, the lack of effect could be because compliance was already high at baseline (such as use of steroids in over 85% of cases where indicated), but even when there was more room for improvement (such as in quality of medical history taking), there was no significant additional net effect of SPI1. There were no changes over time or between control and SPI1 hospitals in errors or rates of adverse events in patients in medical wards. Mortality increased from 11% (27) to 16% (39) among controls and decreased from17%(63) to13%(49) among SPI1 hospitals, but the risk adjusted difference was not significant (0.5, 0.2 to 1.4; P=0.085). Poor care was a contributing factor in four of the 178 deaths identified by review of case notes. The survey of patients showed no significant differences apart from an increase in perception of cleanliness in favour of SPI1 hospitals. Conclusions The introduction of SPI1 was associated with improvements in one of the types of clinical process studied (monitoring of vital signs) and one measure of staff perceptions of organisational climate. There was no additional effect of SPI1 on other targeted issues nor on other measures of generic organisational strengthening.

AB - Objectives: To conduct an independent evaluation of the first phase of the Health Foundation's Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design: Mixed method evaluation involving five substudies, before and after design. Setting: NHS hospitals in United Kingdom. Participants: Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1); 18 control hospitals. Intervention: The SPI1 was a compound (multicomponent) organisational intervention delivered over 18 months that focused on improving the reliability of specific frontline care processes in designated clinical specialties and promoting organisational and cultural change. Results: Senior staff members were knowledgeable and enthusiastic about SPI1. There was a small (0.08 points on a 5 point scale) but significant (P<0.01) effect in favour of the SPI1 hospitals in one of 11 dimensions of the staff questionnaire (organisational climate). Qualitative evidence showed only modest penetration of SPI1 at medical ward level. Although SPI1 was designed to engage staff from the bottom up, it did not usually feel like this to those working on the wards, and questions about legitimacy of some aspects of SPI1 were raised. Of the five components to identify patients at risk of deterioration - monitoring of vital signs (14 items); routine tests (three items); evidence based standards specific to certain diseases (three items); prescribing errors (multiple items from the British National Formulary); and medical history taking (11 items) - there was little net difference between control and SPI1 hospitals, except in relation to quality of monitoring of acute medical patients, which improved on average over time across all hospitals. Recording of respiratory rate increased to a greater degree in SPI1 than in control hospitals; in the second six hours after admission recording increased from 40% (93) to 69% (165) in control hospitals and from 37% (141) to 78% (296) in SPI1 hospitals (odds ratio for "difference in difference" 2.1, 99% confidence interval 1.0 to 4.3; P=0.008). Use of a formal scoring system for patients with pneumonia also increased over time (from 2% (102) to 23% (111) in control hospitals and from 2% (170) to 9% (189) in SPI1 hospitals), which favoured controls and was not significant (0.3, 0.02 to 3.4; P=0.173). There were no improvements in the proportion of prescription errors and no effects that could be attributed to SPI1 in non-targeted generic areas (such as enhanced safety culture). On some measures, the lack of effect could be because compliance was already high at baseline (such as use of steroids in over 85% of cases where indicated), but even when there was more room for improvement (such as in quality of medical history taking), there was no significant additional net effect of SPI1. There were no changes over time or between control and SPI1 hospitals in errors or rates of adverse events in patients in medical wards. Mortality increased from 11% (27) to 16% (39) among controls and decreased from17%(63) to13%(49) among SPI1 hospitals, but the risk adjusted difference was not significant (0.5, 0.2 to 1.4; P=0.085). Poor care was a contributing factor in four of the 178 deaths identified by review of case notes. The survey of patients showed no significant differences apart from an increase in perception of cleanliness in favour of SPI1 hospitals. Conclusions The introduction of SPI1 was associated with improvements in one of the types of clinical process studied (monitoring of vital signs) and one measure of staff perceptions of organisational climate. There was no additional effect of SPI1 on other targeted issues nor on other measures of generic organisational strengthening.

UR - http://www.scopus.com/inward/record.url?scp=79951652685&partnerID=8YFLogxK

U2 - 10.1136/bmj.d195

DO - 10.1136/bmj.d195

M3 - Article

C2 - 21292719

VL - 342

JO - BMJ

JF - BMJ

SN - 0959-8138

IS - 7793

M1 - d195

ER -

Benning A, Ghaleb M, Suokas A, Dixon-Woods M, Dawson J, Barber N et al. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ. 2011 Feb 12;342(7793). d195. https://doi.org/10.1136/bmj.d195