How will we know that change is an improvement?
When trying to improve health systems we often focus on isolated component parts – a hospital, a department, a service, a condition – losing sight of the interdependencies between them. Yet if we truly want to improve outcomes for people and patients who will need support from many different parts of the system, we must view care systems as a whole, beyond their component parts.
Improving the whole system requires change. And with change, we must ask ourselves:
- What are we trying to achieve?
- How will we know that “change” has made an improvement?
- What changes can we make that will lead to improvement?1
Health system designers should answer the first question by defining the aim of the healthcare system at all levels, including:
- The whole system (the macrosystem)
- Integrated patient pathways within the whole system (the mesosystems)
- The role of different clinical delivery units in delivering those pathways (the microsystems)
To achieve the best possible outcomes for people and patients within resource constraints, any health
system will need to aim for value by balancing the Triple Aim framework identified by the Institute for Health Improvements:
- Improving the experience of care
- Improving the health of populations
- Reducing per capita costs of health care2
This overarching aim should trickle down to the mesosystem and microsystem design and improvement work.
Measuring change at the mesosystem level
In cases of whole system transformation, implementing change often requires running small scale pilots. Without clear measurement, we can never know if a pilot is actually an improvement that is ready to be scaled up.
That said, measuring the impact of pilots in whole system transformation is challenging: measuring at the microsystem level will miss out on the impact of pathway integration, while measuring at the macrosystem level will dilute their impact within the whole system.
When piloting small scale patient pathways during the process of whole system transformation, measures should be developed to focus on the mesosystem level and to “provide a view of performance that reflects care provided in different sites and across the continuum of care.”3
Pathway measures in relation to the three levels of a healthcare system
Mesosystem measures feed into PDSA cycles to drive the improvement of pathways
As new patient pathways are piloted, a core question comes to mind: “How do we know that the pilot was successfully complete, and are we ready to scale up?”
“A pilot for a pathway is complete when it achieves “target goals” for key measures.”
Inspired by Deming’s Methodology for Improvement, implementation teams can use mesosystem measures to study the impact of implementing small scale pilots, and use findings to feed into Plan - Do- Study- Act (PDSA) cycles of improvement. PDSA cycles should be embedded into refining system design before rolling out to the whole system, building trust in the required change, and creating a culture of system thinking and continuous improvement.4
Mesosystem measures framework
Inspired by the Institute for Health Improvement’s Triple Aim and the Institute of Medicine’s domains of healthcare quality and whole system measures, we developed a measures framework that facilitates assessing the impact of piloting small scale patient pathways as part of a specific mesosystem. The framework is used to develop measures that assess implementation against two dimensions:
- Patient experience
- Workforce experience
Patient pathways under four care categories that cover the various health needs of people: planned care, chronic care, urgent/emergency care, and last phase of life. We then identified core mesosystem measures that are few enough to be frequently and manually collected but broad enough to allow for providing a big picture view on the success of a pathway pilot in improving the experience of care in each category.5
Dimension |
Sub-dimension |
Measure |
Care Categories |
|||
Planned |
Chronic |
Urgent |
Last Phase |
|||
Care Experience |
Safe |
Standardised rate of adverse events |
✔ | ✔ | ✔ | ✔ |
Effective |
Standardised mortality ratio |
✔ | ✔ | ✔ | ||
Percentage of hospital readmissions |
✔ | ✔ | ✔ | |||
Rate of ambulatory sensitive hospitalisations and ED visits |
✔ | |||||
Percentage of ED patients that are CTAS 4-5 |
✔ | |||||
Equitable |
Average number of hospital days per decedent in the last three months of life |
✔ | ||||
Efficient |
Average length of stay |
✔ | ||||
Average number of patient visits prior to scheduling a procedure |
✔ | |||||
Patient-centered |
Patient satisfaction and experience score |
✔ | ✔ | ✔ | ✔ | |
Equitable |
Variability in care experience and satisfaction rate based on demographics |
✔ | ✔ | ✔ | ✔ | |
Timely |
Average number of days to third next available appointment for doctor visits |
✔ | ✔ | ✔ | ✔ | |
Workforce Experience |
Occupational health |
Workforce awareness of guidelines |
✔ | ✔ | ✔ | ✔ |
Incidence of nonfatal occupational injuries and illnesses |
✔ | ✔ | ✔ | ✔ | ||
Joy at work |
Job satisfaction and experience score |
✔ | ✔ | ✔ | ✔ | |
Variability in workforce experience and satisfaction rate based on demographics |
✔ | ✔ | ✔ | ✔ | ||
Engagement |
Care delivery engagement score |
✔ | ✔ | ✔ | ✔ | |
These mesosystem measures were tailored to feed into whole system measures that support promoting improvements in capacity and capability for frontline staff and leadership. While these measures currently only focus on the “experience of care” aspect of the Institute for Health Improvement’s Triple Aim framework (population health, experience of care, and costs), they may be expanded to cover all dimensions and provide an overview of the value the whole system is achieving. (Berwick, Nolan, & Whittington, 2008)
Applying mesosystem measures inpractice
To study change at the mesosystem level, we propose the following eight step approach:
How will this methodology support the implementation of new models of care in the GCC region?
Many of the GCC countries have ambitious visions and are embarking on national healthcare transformation journeys, with new models of care and patient pathways at the core. To do so, they need to mobilise great numbers of healthcare professionals to change the way they work together. GCC countries are guided by the goodwill, the experience of hundreds of clinicians, and somewhat similar experiences of international health systems. A good start, but not a science.
As we begin to implement new models of care, we want to back our art with science. We therefore recommend applying the methodology detailed in this paper to study the impact of new pathways and ensure they are achieving their aim in the piloting stage, prior to fully rolling them out.
[1] Langley, G., Nolan, K., Nolan, T., Norman, C., & Provost, L. (2009). The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco: Jossey-Bass Publishers.
[2] Berwick, D. M., Nolan, T. W., & Whittington, J. (2008). The Triple Aim: Care, Health, and Cost. Health Affairs, 759-769.
[3] Martin, L. A., Nelson, E., Lloyd, R., & Nolan, T. W. (2007). Whole System Measures. IHI Innovation Series White Paper. Cambridge, Massachusetts: Institute for Health Improvement. Retrieved from Institute for Health Improvement Innovation Series.
[4] Langley, G., Nolan, K., Nolan, T., Norman, C., & Provost, L. (2009).
[5] Martin, L. A., Nelson, E., Lloyd, R., & Nolan, T. W. (2007)
