How to measure improvement in new models of care

From pilot to rollout

How will we know that change is an improvement?

When trying to improve health systems we often focus on isolated component parts – a hospital, a department, a service, a condition – losing sight of the interdependencies between them. Yet if we truly want to improve outcomes for people and patients who will need support from many different parts of the system, we must view care systems as a whole, beyond their component parts.

Improving the whole system requires change. And with change, we must ask ourselves:

  1. What are we trying to achieve?
  2. How will we know that “change” has made an improvement?
  3. What changes can we make that will lead to improvement?1

Health system designers should answer the first question by defining the aim of the healthcare system at all levels, including:

  • The whole system (the macrosystem)
  • Integrated patient pathways within the whole system (the mesosystems)
  • The role of different clinical delivery units in delivering those pathways (the microsystems)

To achieve the best possible outcomes for people and patients within resource constraints, any health
system will need to aim for value by balancing the Triple Aim framework identified by the Institute for Health Improvements:

  • Improving the experience of care
  • Improving the health of populations
  • Reducing per capita costs of health care2

This overarching aim should trickle down to the mesosystem and microsystem design and improvement work.

Measuring change at the mesosystem level

In cases of whole system transformation, implementing change often requires running small scale pilots. Without clear measurement, we can never know if a pilot is actually an improvement that is ready to be scaled up.

That said, measuring the impact of pilots in whole system transformation is challenging: measuring at the microsystem level will miss out on the impact of pathway integration, while measuring at the macrosystem level will dilute their impact within the whole system.

When piloting small scale patient pathways during the process of whole system transformation, measures should be developed to focus on the mesosystem level and to “provide a view of performance that reflects care provided in different sites and across the continuum of care.”3

Effectively assessing patient pathway pilots requires measures at the mesosystem level

Pathway measures in relation to the three levels of a healthcare system

Mesosystem measures feed into PDSA cycles to drive the  improvement of pathways

Mesosystem measures feed into PDSA cycles to drive the improvement of pathways

As new patient pathways are piloted, a core question comes to mind: “How do we know that the pilot was successfully complete, and are we ready to scale up?”

“A pilot for a pathway is complete when it achieves “target goals” for key measures.”

Inspired by Deming’s Methodology for Improvement, implementation teams can use mesosystem measures to study the impact of implementing small scale pilots, and use findings to feed into Plan - Do- Study- Act (PDSA) cycles of improvement. PDSA cycles should be embedded into refining system design before rolling out to the whole system, building trust in the required change, and creating a culture of system thinking and continuous improvement.4

Mesosystem measures framework

Inspired by the Institute for Health Improvement’s Triple Aim and the Institute of Medicine’s domains of healthcare quality and whole system measures, we developed a measures framework that facilitates assessing the impact of piloting small scale patient pathways as part of a specific mesosystem. The framework is used to develop measures that assess implementation against two dimensions:

  • Patient experience
  • Workforce experience

Patient pathways under four care categories that cover the various health needs of people: planned care, chronic care, urgent/emergency care, and last phase of life. We then identified core mesosystem measures that are few enough to be frequently and manually collected but broad enough to allow for providing a big picture view on the success of a pathway pilot in improving the experience of care in each category.5

Dimension

Sub-dimension

Measure

Care Categories

Planned

Chronic

Urgent

Last  Phase

Care  Experience

Safe

Standardised rate of adverse events

Effective

Standardised mortality ratio

 

Percentage of hospital readmissions

 

Rate of ambulatory sensitive  hospitalisations and ED visits

     

Percentage of ED patients that are  CTAS 4-5

     

Equitable

Average number of hospital days per  decedent in the last three months of  life

     

Efficient

Average length of stay

     

Average number of patient visits prior  to scheduling a procedure

     

Patient-centered

Patient satisfaction and experience  score

Equitable

Variability in care experience  and satisfaction rate based on  demographics

Timely

Average number of days to third next  available appointment for doctor visits

Workforce  Experience

Occupational  health

Workforce awareness of guidelines

Incidence of nonfatal occupational  injuries and illnesses

Joy at work

Job satisfaction and experience score

Variability in workforce experience  and satisfaction rate based on  demographics

Engagement

Care delivery engagement score

These mesosystem measures were tailored to feed into whole system measures that support promoting improvements in capacity and capability for frontline staff and leadership. While these measures currently only focus on the “experience of care” aspect of the Institute for Health Improvement’s Triple Aim framework (population health, experience of care, and costs), they may be expanded to cover all dimensions and provide an overview of the value the whole system is achieving. (Berwick, Nolan, & Whittington, 2008)

How will this methodology support the implementation of new models of care in the GCC region?

Many of the GCC countries have ambitious visions and are embarking on national healthcare transformation journeys, with new models of care and patient pathways at the core. To do so, they need to mobilise great numbers of healthcare professionals to change the way they work together. GCC countries are guided by the goodwill, the experience of hundreds of clinicians, and somewhat similar experiences of international health systems. A good start, but not a science.

As we begin to implement new models of care, we want to back our art with science. We therefore recommend applying the methodology detailed in this paper to study the impact of new pathways and ensure they are achieving their aim in the piloting stage, prior to fully rolling them out.


[1] Langley, G., Nolan, K., Nolan, T., Norman, C., & Provost, L. (2009). The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco: Jossey-Bass Publishers.
[2] Berwick, D. M., Nolan, T. W., & Whittington, J. (2008). The Triple Aim: Care, Health, and Cost. Health Affairs, 759-769.
[3] Martin, L. A., Nelson, E., Lloyd, R., & Nolan, T. W. (2007). Whole System Measures. IHI Innovation Series White Paper. Cambridge, Massachusetts: Institute for Health Improvement. Retrieved from Institute for Health Improvement Innovation Series.
[4] Langley, G., Nolan, K., Nolan, T., Norman, C., & Provost, L. (2009).

[5] Martin, L. A., Nelson, E., Lloyd, R., & Nolan, T. W. (2007)

Contact us

Oussama Nicolas

Oussama Nicolas

Health Industries Partner, Health System Transformation, PwC Middle East

Tel: +974 (0) 44 192 777

Tara Makarem

Tara Makarem

Director, Health Industries, PwC Middle East

Follow us