Regular reviews of clinical variation data help health services identify areas of practice that need improvement. They are also mandated by Action 1.28 of the National Safety and Quality Health Service (NSQHS) Standards. This User Guide presents a six-step approach to the review of clinical variation data, and case studies that put those steps into action.
Why measure clinical variation?
Substantial variation in healthcare outcomes or processes is an alarm bell that should make us stop and investigate whether appropriate care is being delivered.
Variation in itself is not necessarily bad. When it reflects differences in patients’ needs, it can be an indicator of good quality healthcare. When it doesn’t reflect patients’ needs, it is ‘unwarranted’ and represents an opportunity for a health service to improve its performance.
Action 1.28 of the NSQHS Standards requires health service organisations to identify potentially unwarranted variation, and regularly review and improve the appropriateness of clinical care.
How will this guide help?
This guide explains how health service organisations can implement Action 1.28 of the NSQHS Standards.
In order to implement Action 1.28, a health service organisation needs to collect data on its own clinical care processes and outcomes, and review their performance in comparison to:
- Other health service organisations, and/or
- Evidence-based guidelines or clinical care standards.
These two comparisons can also be framed as questions:
- How does care delivered in this organisation compare with care in similar organisations?
- How does care delivered in this organisation compare with best practice care?
Health service organisations can choose to examine clinical variation at the level of clinical teams within the organisation and/or at an organisational level. Examining both levels is recommended. Clinical leaders and clinical teams should be involved throughout the entire process of examining variation in clinical care.
How can small services use the guide?
Projects conducted by small health service organisations can be simpler and targeted on one specific area of safety and quality. For example, they may involve auditing one clinical area against one aspect of an evidence-based guideline or a clinical care standard that is regularly used by the health service. Smaller, simpler projects are appropriate for day procedure services and small hospitals (generally under 50 beds).
Step 1. Select priority areas
Select areas of clinical care to investigate based on:
- High volume of patients
- High risk to patients, regardless of volume
- High morbidity, mortality or patient dissatisfaction
- Clinical areas identified in the health service’s risk register as high or potentially high risk
- Existence of an established evidence base for best practice
- Evidence that overuse or underuse of the intervention increases risk to patient health
- Advice from clinical leaders in the health service organisation
- Availability of external data or standards/guidelines for comparison with the health service organisation’s practice
- Clinical areas where new evidence or technology has substantially changed the standard of care
- Interventions that have been identified as low-value care
- Choosing Wisely Australia recommendations.
Decision making about selecting the priority areas to investigate can be aligned with the
organisation’s risk management approach (Action 1.10 of Standard 1, Clinical Governance).
Step 2. Plan the project
Once you have chosen the clinical area, you will need to decide whether you will compare your data against data from other health service organisations, and/or evidence-based guideline recommendations or a clinical care standard. This decision will be influenced by the availability of these comparators.
You will also need to decide whether you will use clinical team data, departmental-level data and/or health service organisation-level data, for comparison with external data, guideline recommendations or standards.
In some cases there may be substantial variation within the health service organisation even if the data does not show marked variation at a whole-of-organisation level compared with external data or with guidelines. For this reason, exploring the data at the level of clinical teams as well as whole-of-organisation level is very useful for ensuring appropriate care.
Step 3. Measure and review
This step requires health service organisations to have processes that:
- Measure clinical care delivered and outcomes achieved in the health service organisation
- Compare the care delivered or outcomes achieved within the health service organisation to those of other organisations, using external reports, clinical quality registries and audits, and/or
- Assess the extent to which care delivery and outcomes align with evidence-based guideline recommendations and clinical care standards
- Assess the clinical importance of any noted variation and take action if required.
Determine the measures or indicators
Use a clear, consistent definition of the measures or indicators to be assessed to allow meaningful comparison with other health service organisations’ data and with guidelines or standards. In many instances there are existing high-quality indicators that can be used to consistently measure data over periods of time, and that allow comparison with results from similar health service organisations. For example, Clinical Care Standards include indicators that can be used.
Define the target population
Other key requirements include a definition of the target population – the group of patients who should be offered the specific type of care outlined in the clinical care standards or guideline recommendations – and identification of the benchmarks that should be achieved within the health service organisation.
Audit the clinical records
Clinical records can then be audited using the systems established for this purpose. Action 1.16 of the NSQHS Standards requires the health service organisation to have healthcare record systems that support systematic audit of clinical information. If electronic systems are used to extract information it should be possible to determine whether care was provided to all eligible patients within a certain time period. However, if manual collection of information from medical records is required, it may only be feasible to collect information from a sample of patients. Once the information has been collected it can be summarised, and the extent of any identified gaps between best evidence and current practice can be assessed. See Using data for quality improvement.
Step 4. Explore reasons
Auditing the relevant aspects of patient records is an effective way to gain insights into how and why clinical care is deviating from best practice. Ensuring that relevant staff discuss the findings as a group will draw out different perspectives on processes that affect clinical care, and can help foster a team approach to improving practice.
Reasons for clinical variation
There are many reasons for variation in rates of care processes and in outcomes of care. Variation can be due to differences in the needs of patients, in which case it is warranted and desirable. Other reasons for clinical variation include:
- Clinical care not changing in line with updated evidence
- Differences in clinicians’ knowledge of the latest evidence or skills related to new diagnostic or interventional procedures
- Clinical uncertainty about an intervention’s place in therapy, and the need for better data on its benefits and harms
- Inequity of access to care
- Effects of financial incentives or disincentives
- Inadequate system supports for appropriate care
- Inadequate information sharing and discussion with consumers
- Barriers experienced by Aboriginal and Torres Strait Islander peoples in accessing appropriate, culturally safe care.
It is important to give patients clear information about options, and to provide adequate opportunity to discuss this information. The Commission has a range of resources to assist with shared decision making so patients can make properly informed choices. For some interventions there is a lack of data on benefits and risks; in this situation it is important to discuss what is known and what is unknown, and what the other options are.
Relatively low or high rates of use of an intervention may arise for reasons other than decisions made by individual clinicians. For example, low use of some interventions may signal a problem with access to clinical care for people who need certain tests, treatments or procedures. In this instance the health service organisation may need to examine whether services, and the workforce and resources to deliver them, are appropriately allocated given the needs of people within their catchment area. However, unwarranted variation in use may also indicate ongoing use of an out-dated method of treatment, and signal the need for clinical training in new procedures or treatments. Unwarranted high rates of interventions may also occur because of an oversupply of workforce or technology leading to lowering of clinical thresholds for undertaking these interventions.
Occasionally the investigation will highlight a pattern of problems with individual or team decision-making or skills. In this situation: consider what extra support is needed to deliver the best possible care; determine whether an external review is needed with the relevant manager; and follow the performance review processes in accordance with Action 1.22 of the NSQHS Standards. Also, implement the incident management systems (Action 1.11 of the NSQHS Standards) and open disclosure (Action 1.12 of the NSQHS Standards) when relevant.
For detailed advice on auditing patient records and other methods of investigating reasons for clinical variation, see the resources listed under Using data for quality improvement in Step 3.
Step 5. Act to improve
Actions to improve appropriateness of care that are prompted by your variation findings should be incorporated into the organisation’s overall approach to safety and quality improvement.
Continuous quality improvement occurs through a cyclical approach:
- Specify the desired goal for improvement
- Explore the reasons for current practice
- Identify the barriers or enablers for any desired change in practice
- Make changes to health care processes
- Monitor progress and make further changes as necessary.
Actions to improve appropriateness of care will need to be embedded in the health service organisation for improvements to care to be sustained. See Using data for quality improvement for resources offering detailed advice on data-driven quality improvement methods.
‘Case for improvement’ documents
‘Case for improvement’ documents, which are available for some of the clinical care standards, provide further useful information for quality improvement. For each quality statement in a clinical care standard, the case for improvement document asks the following questions:
- Why is it important?
- What is known about current practice?
- What could be achieved with more consistent application of the aspects of care described?
When possible, examples are provided showing how specific approaches or systems for implementing best practice have demonstrated measurable change.
Do not delay investigating patient care to check data
If you have found that practice within the health service varies substantially from other health service organisations or from evidence-based guideline recommendations or clinical care standards, the priority – and prime responsibility – is to ensure there is no problem with patient care. Do not delay investigating potentially suboptimal care in order to recheck data.
Step 6. Monitor and report
The health service organisation should maintain records of reviews and actions taken as a result of examining variation and appropriateness of care. Processes should be in place to report actions and outcomes, consistent with the requirements of Action 1.09 of the NSQHS Standards.
Clinicians have a professional responsibility to review the care they provide and to participate in efforts to improve the quality of care. Organisations employing clinicians need to ensure that there is clinical participation in internal reviews of clinical care performance, and should encourage and provide practical support for clinical participation in relevant external clinical quality registries and audits.
Health service organisations should keep records of clinician participation in reviews of clinical variation.
Clinicians can use these records to demonstrate compliance with continuing professional development requirements; for example, the continuing professional requirements for medical practitioners. This aligns with Action 1.22 of the NSQHS Standards.
Requirements for continuing professional development
Stronger continuing professional development is one of the core features of the Medical Board of Australia’s new Professional Performance Framework for medical practitioners which comes into effect from January 2023. As well as participating in educational activities to build knowledge and skills, medical practitioners will be required to review their performance and measure the outcomes of care they provide.
The expectation that health service organisations review variation and appropriateness of clinical care is consistent with the expectation of the registering body that individual medical practitioners should regularly review their performance and outcomes.
Case studies on clinical variation
These case studies showcase best practice and innovation in the review of clinical variation, in a variety of different healthcare settings.
Do you have a case study to contribute?
We will work with you to write the case study. Please contact us on Atlas@safetyandquality.gov.au
Browse latest case studies
Potential clinical topics for investigation and sources of evidence-based recommendations
The list below highlights a range of potential clinical topics for investigation, and related evidence-based recommendations on which to base a review of clinical variation.
Clinical care standards
A complete list of national clinical care standards can be accessed on the website of the Australian Commission on Safety and Quality in Health Care.
- Acute Coronary Syndromes Clinical Care Standard
- Acute Stroke Clinical Care Standard
- Antimicrobial Stewardship Clinical Care Standard
- Cataract Clinical Care Standard
- Colonoscopy Clinical Care Standard
- Delirium Clinical Care Standard
- Heavy Menstrual Bleeding Clinical Care Standard
- Hip Fracture Care Clinical Care Standard
- Management of Peripheral Intravenous Catheters Clinical Care Standard
- Opioid Analgesic Stewardship in Acute Pain Clinical Care Standard
- Osteoarthritis of the Knee Clinical Care Standard
- Third and Fourth Degree Perineal Tears Clinical Care Standard
- Venous Thromboembolism Prevention Clinical Care Standard
Evidence of implementing Action 1.28
See the list below for examples of evidence of implementing Action 1.28 of the NSQHS Standards.
About the NSQHS Standards
The National Safety and Quality Health Service (NSQHS) Standards were developed by the Australian Commission on Safety and Quality in Health Care (the Commission) in collaboration with the Australian Government, states and territories, the private sector, clinical experts, patients and carers. The primary aims of the NSQHS Standards are to protect the public from harm and to improve the quality of health service provision. They provide a quality assurance mechanism, which tests whether relevant systems are in place to ensure that expected standards of safety and quality are met.
There are a number of related actions within the Clinical Governance Standard which can be used alongside Action 1.28 to address clinical variation. These are summarised in the items below.