Assessing an experimental approach to industrial policy evaluation: Applying RCT+ to the case of Creative Credits

Hasan Bakhshi, John S. Edwards, Stephen Roper*, Judy Scully, Duncan Shaw, Lorraine Morley, Nicola Rathbone

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.
Original languageEnglish
Pages (from-to)1462-1472
Number of pages11
JournalResearch policy
Volume44
Issue number8
Early online date8 Jun 2015
DOIs
Publication statusPublished - Oct 2015

Fingerprint

Innovation
Industry
Industrial policy
Credit
Policy evaluation
Randomized controlled trial
Experimental method
Mixed methods
Data collection
Longitudinal data
Evaluation

Bibliographical note

© 2015, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/

Funding: Nesta, the Economic and Social Research Council, the Arts and Humanities Research Council and the North West Development Agency.

Keywords

  • evaluation
  • experimental
  • industrial policy
  • innovation
  • creative
  • qualitative research

Cite this

Bakhshi, Hasan ; Edwards, John S. ; Roper, Stephen ; Scully, Judy ; Shaw, Duncan ; Morley, Lorraine ; Rathbone, Nicola. / Assessing an experimental approach to industrial policy evaluation : Applying RCT+ to the case of Creative Credits. In: Research policy. 2015 ; Vol. 44, No. 8. pp. 1462-1472.
@article{46adf4bb4ab045dfb31dfca2b64f2085,
title = "Assessing an experimental approach to industrial policy evaluation: Applying RCT+ to the case of Creative Credits",
abstract = "Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.",
keywords = "evaluation, experimental, industrial policy, innovation, creative, qualitative research",
author = "Hasan Bakhshi and Edwards, {John S.} and Stephen Roper and Judy Scully and Duncan Shaw and Lorraine Morley and Nicola Rathbone",
note = "{\circledC} 2015, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/ Funding: Nesta, the Economic and Social Research Council, the Arts and Humanities Research Council and the North West Development Agency.",
year = "2015",
month = "10",
doi = "10.1016/j.respol.2015.04.004",
language = "English",
volume = "44",
pages = "1462--1472",
journal = "Research policy",
issn = "0048-7333",
publisher = "Elsevier",
number = "8",

}

Assessing an experimental approach to industrial policy evaluation : Applying RCT+ to the case of Creative Credits. / Bakhshi, Hasan; Edwards, John S.; Roper, Stephen; Scully, Judy; Shaw, Duncan; Morley, Lorraine; Rathbone, Nicola.

In: Research policy, Vol. 44, No. 8, 10.2015, p. 1462-1472.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Assessing an experimental approach to industrial policy evaluation

T2 - Applying RCT+ to the case of Creative Credits

AU - Bakhshi, Hasan

AU - Edwards, John S.

AU - Roper, Stephen

AU - Scully, Judy

AU - Shaw, Duncan

AU - Morley, Lorraine

AU - Rathbone, Nicola

N1 - © 2015, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/ Funding: Nesta, the Economic and Social Research Council, the Arts and Humanities Research Council and the North West Development Agency.

PY - 2015/10

Y1 - 2015/10

N2 - Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.

AB - Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.

KW - evaluation

KW - experimental

KW - industrial policy

KW - innovation

KW - creative

KW - qualitative research

UR - http://www.scopus.com/inward/record.url?scp=84937833417&partnerID=8YFLogxK

U2 - 10.1016/j.respol.2015.04.004

DO - 10.1016/j.respol.2015.04.004

M3 - Article

AN - SCOPUS:84937833417

VL - 44

SP - 1462

EP - 1472

JO - Research policy

JF - Research policy

SN - 0048-7333

IS - 8

ER -