TY - JOUR
T1 - Applying machine learning classifiers to automate quality assessment of paediatric dynamic susceptibility contrast (DSC-) MRI data
AU - Powell, Stephen J
AU - Withey, Stephanie B
AU - Sun, Yu
AU - Grist, James T
AU - Novak, Jan
AU - MacPherson, Lesley
AU - Abernethy, Laurence
AU - Pizer, Barry
AU - Grundy, Richard
AU - Morgan, Paul S
AU - Jaspan, Tim
AU - Bailey, Simon
AU - Mitra, Dipayan
AU - Auer, Dorothee P
AU - Avula, Shivaram
AU - Arvanitis, Theodoros N
AU - Peet, Andrew
N1 - Copyright © 2023 The Authors. Published by the British Institute of Radiology. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 Unported License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited.
PY - 2023/4/1
Y1 - 2023/4/1
N2 - OBJECTIVE: Investigate the performance of qualitative review (QR) for assessing dynamic susceptibility contrast (DSC-) MRI data quality in paediatric normal brain and develop an automated alternative to QR.METHODS: 1027 signal-time courses were assessed by Reviewer 1 using QR. 243 were additionally assessed by Reviewer 2 and % disagreements and Cohen's κ (κ) were calculated. The signal drop-to-noise ratio (SDNR), root mean square error (RMSE), full width half maximum (FWHM) and percentage signal recovery (PSR) were calculated for the 1027 signal-time courses. Data quality thresholds for each measure were determined using QR results. The measures and QR results trained machine learning classifiers. Sensitivity, specificity, precision, classification error and area under the curve from a receiver operating characteristic curve were calculated for each threshold and classifier.RESULTS: Comparing reviewers gave 7% disagreements and κ = 0.83. Data quality thresholds of: 7.6 for SDNR; 0.019 for RMSE; 3 s and 19 s for FWHM; and 42.9 and 130.4% for PSR were produced. SDNR gave the best sensitivity, specificity, precision, classification error and area under the curve values of 0.86, 0.86, 0.93, 14.2% and 0.83. Random forest was the best machine learning classifier, giving sensitivity, specificity, precision, classification error and area under the curve of 0.94, 0.83, 0.93, 9.3% and 0.89.CONCLUSION: The reviewers showed good agreement. Machine learning classifiers trained on signal-time course measures and QR can assess quality. Combining multiple measures reduces misclassification.ADVANCES IN KNOWLEDGE: A new automated quality control method was developed, which trained machine learning classifiers using QR results.
AB - OBJECTIVE: Investigate the performance of qualitative review (QR) for assessing dynamic susceptibility contrast (DSC-) MRI data quality in paediatric normal brain and develop an automated alternative to QR.METHODS: 1027 signal-time courses were assessed by Reviewer 1 using QR. 243 were additionally assessed by Reviewer 2 and % disagreements and Cohen's κ (κ) were calculated. The signal drop-to-noise ratio (SDNR), root mean square error (RMSE), full width half maximum (FWHM) and percentage signal recovery (PSR) were calculated for the 1027 signal-time courses. Data quality thresholds for each measure were determined using QR results. The measures and QR results trained machine learning classifiers. Sensitivity, specificity, precision, classification error and area under the curve from a receiver operating characteristic curve were calculated for each threshold and classifier.RESULTS: Comparing reviewers gave 7% disagreements and κ = 0.83. Data quality thresholds of: 7.6 for SDNR; 0.019 for RMSE; 3 s and 19 s for FWHM; and 42.9 and 130.4% for PSR were produced. SDNR gave the best sensitivity, specificity, precision, classification error and area under the curve values of 0.86, 0.86, 0.93, 14.2% and 0.83. Random forest was the best machine learning classifier, giving sensitivity, specificity, precision, classification error and area under the curve of 0.94, 0.83, 0.93, 9.3% and 0.89.CONCLUSION: The reviewers showed good agreement. Machine learning classifiers trained on signal-time course measures and QR can assess quality. Combining multiple measures reduces misclassification.ADVANCES IN KNOWLEDGE: A new automated quality control method was developed, which trained machine learning classifiers using QR results.
UR - https://www.birpublications.org/doi/10.1259/bjr.20201465
U2 - 10.1259/bjr.20201465
DO - 10.1259/bjr.20201465
M3 - Article
C2 - 36802769
SN - 0007-1285
VL - 96
JO - British Journal of Radiology
JF - British Journal of Radiology
IS - 1145
ER -