Ryan J. A. Murphy, Lewis Muirhead, and Louise Gallagher
An analysis of acceptance and use of systemic practices in program design and evaluation
This paper explores the adoption of systemic practices in the evaluation of complex change initiatives through the lens of the Unified Theory of Acceptance and Use of Technology (UTAUT2) model. Despite the potential of systemic practices to enhance program evaluation in systemic change initiatives — and the celebration of systemic principles in program evaluation in general — the adoption of systemic practices in many initiatives remains incomplete. This study seeks to understand the barriers to adoption and proposes a framework to facilitate the complete adoption of systemic practices in complex changemaking initiatives. Our analysis reveals that many of the UTAUT2 factors may be critical barriers to the adoption of systemic practices; in turn, practitioners should anticipate these specific barriers when developing projects in systemic design. To address these barriers, we propose the SHIFT framework, a set of five questions designed to scaffold the adoption of systemic practices in complex change initiatives. The SHIFT framework emphasises the importance of setting up facilitating conditions, identifying the habitual practices that need to change, adopting an incremental approach to introducing systemic practices, focusing on the ultimate goal of systemic change when evaluating the utility of the practices, and testing performance by triangulating progress on sensitive signals to systemic change. This paper contributes to the field of systemic evaluation by providing a structured approach to overcoming the barriers to the adoption of systemic practices. By leveraging the UTAUT2 model and proposing the SHIFT framework, we offer practical strategies for practitioners and organizations seeking to enhance their systemic change initiatives through the adoption of systemic practices. Our findings suggest that by addressing facilitating conditions and adopting a modular approach to systemic practices, the field can move closer to the full integration of systemic evaluation methods, ultimately leading to more effective and sustainable systemic change.
KEYWORDS: evaluation, theories of systemic change, framework, practices, adoption