Advisory D23/01: Sampling for assessments to the Digital Mental Health Standards

Advisory details

Purpose

To describe the methodology to be used by accrediting agencies when sampling multi-site digital mental health service providers at assessment to the National Safety and Quality Digital Mental Health Standards (Digital Mental Health Standards) or the Digital Mental Health Modules.

Issue

The Australian Health Service Safety and Quality Accreditation Scheme requires digital mental health service providers to be assessed either onsite, virtually or using a hybrid method (e.g. a combination of both onsite and virtual assessment). 

To ensure consistency, the Australian Commission on Safety and Quality in Health Care (the Commission) has developed a methodology to apply when assessing digital mental health service providers. Sampling is only permitted for providers where multiple digital mental health service offerings operate under a single governance structure – this is referred to as a multi-site service provider. 

Multi-site service providers will have sites in separate geographical locations which provide the same type of service offering being assessed to the Digital Mental Health Standards.

Multi-site service providers are to be assessed at the level of the site, regardless of whether they provide a number of different service streams (e.g., telephone, SMS-based or online chat services).

Where a digital mental health service provider is being assessed to a Digital Mental Health Module, this sampling advisory only applies to the in-scope digital mental health services.

Requirements

For digital mental health services providers with up to 30 sites that are to be assessed and where sampling is approved, the following calculation will be used:

• Five (5) service offerings, plus one third of all remaining services are to be assessed, (rounded up to the nearest whole number).

For digital mental health services providers with more than 30 sites that are to be assessed and where sampling is approved, the following calculation will be used:

• Ten (10) service offerings plus one quarter of all remaining health services are to be assessed, (rounded up to the nearest whole number).

For digital mental health services providers with greater than 100 sites an application to the Commission can be made to have the number of sites that undergo onsite assessment capped at an upper limit.

The following rules will apply when selecting the assessment sites:

1. The corporate or main entity is always to be assessed.
2. 75 percent of sites assessed are to be selected based on a risk assessment by the accrediting agency. This assessment will take into consideration factors such as (but not limited to) previous assessment results and the last time the site was assessed (where relevant), information provided in a gap analysis such as incidents and complaints, information from external reports, and information gained in discussions with the service provider.
3. 25 percent of sites assessed are to be selected at random, ensuring a representative sample of the service is assessed.
4. Accrediting agencies are to notify the service provider of sample sites just prior to or at the commencement of the assessment, and for rural or regional sites no more than six weeks in advance.
5. Sample sites are to be selected by the accrediting agency, without taking into consideration the views of the service provider.
6. Exemptions may apply to points 4 and 5 on a case-by-case basis following application of exemption to the Commission.

Exemptions

Exceptions to point 4 and 5 of these rules can be made for services where the availability of staff and patients is unpredictable and therefore requires some advance planning, or advanced notice is a requirement of entry to the service.

Where approval for exemption is granted, the approved accrediting agency can work in collaboration with the service provider implementing rules 1-3 to maximise the number of staff, service users (where relevant) and sites that can be interviewed and visited during an assessment. 

Organisations seeking exemption from this sampling methodology are to make a submission in writing (which can be in the form of an email), to the Commission via their approved accrediting agency. The Commission will, in collaboration with accrediting agencies make a determination on these applications.

Examples of sampling methodology calculation

Where ‘X’ represents the total number of service offerings and ‘Y’ represents the number of services offerings that are to be assessed.