Improve
Improve policies and procedures through continuous quality improvement.
Improve
Data is essential
Like the other Zero Suicide elements, it’s impossible to provide a recipe that works for every organization, but one very important ingredient is data. Measuring suicide death and suicide attempt rates are important, but another vital part of continuous quality improvement is determining if the interventions, workflows, and procedures are working the way they are intended to.
If data collection is not a regular occurrence it might feel daunting to use data to improve specific aspects of suicide care but once it is started the benefits become obvious quickly. Organizations should use data to focus improvements and also to celebrate successes.
Question
What is our plan for evaluating the interventions, workflows, and policies we have implemented? What information do we need to collect to answer our evaluation questions?
Start early
The Improve element is often last in the list of elements but that does not mean it should be thought about after everything else has been done. Data collection should start right when implementation starts. It can begin with the question: “How will we know if this new “thing” is working?”
When organizations implement a screening process, the care pathway, a new way to document risk, or another Zero Suicide component, it is crucial to understand if it is having the desired effect.
- Are more individuals being screened?
- Are appropriate individuals being added to the care pathway?
- Are individuals receiving caring contacts?
- What information is needed to be able to measure it?
Start small
It isn’t necessary to have a report full of data points with graphics and statistical analyses. It is okay to start small, with one or two data points. It can be helpful to start with a question (or questions).
- Are staff doing “the thing” they are trained to do (e.g., screening, assessing, safety planning, warm handoffs, providing a specific treatment)?
- Are staff doing “the thing” the way they were trained to do it (e.g., are staff asking all the questions on the screening tool)?
Also, questions like:
- Do clinical staff feel confident in their ability to provide collaborative and supportive suicide care for individuals at risk of suicide?
- Do the individuals served feel cared about by staff in the organization?
Next, decide what type of data is needed to answer those questions and how to collect it. Do you need numbers, words or do you need to watch someone do something? Do you need to build and run a report from the electronic health record system? Is an anonymous survey needed? Or a facilitated discussion within a team? Do staff members need to be observed as they provide an intervention?
What does data tell you?
Once data collection is complete leaders and supervisors/managers should review it and make meaning out of it. Without this step data is just numbers or words. Be curious about the story that the data is telling. Not all data is equal. Data, particularly numbers, are just one part of the story of the impact of workflow, procedures, or interventions on staff and individuals in care. Collecting data from observations, conversations, and anonymous feedback can help round out the story that the numbers might be illustrating.
The following is an example of how the review of data led to important changes in Centura Health's (a healthcare system in Colorado and Kansas that is now known as CommonSpirit and AdventHealth) screening processes which improved patient care:
Universal screening of all patients for suicide took place from October 2021 until June 2022. At that point, universal screening was discontinued and only patients who presented with primary behavioral health complaints were screened for suicide.
In October 2022, the Zero Suicide implementation team approached the Centura executive team to advocate for reinstatement of universal screening based on data that was collected from prior months. The data clearly demonstrated the limitations of screening only those with a primary behavioral health complaint.
Over an 18-month period of universal screening (Oct 2021 - June 2022), only 27% of patients who screened high on the C-SSRS presented to the ED with a primary behavioral health complaint. This suggests that 73% of high-risk patients may have been missed, if they did not seek care primarily for behavioral health issues. During this same time frame (Oct 2021 - June 2022), approximately 600 patients per month were referred for crisis assessments due to suicidality. However, when universal screening was suspended for just three months (July 2022 - October 2022), that number dropped significantly to an average of 300 referrals per month, highlighting the importance of universal screening to identifying at-risk individuals.
As a result, Executive Leadership approved the change and updated their policy to reinstate universal screening across all of Centura’s Colorado hospitals. Universal screening began again in November 2022.
Multiple sources of data will illustrate a holistic perspective and align with a Just Culture. After meaning is made from the data, aggregate data should be shared with staff. This promotes transparency, collaboration, and psychological safety.
After that, develop plans to celebrate successes and improve the processes, workflow, and/or interventions.
One way to structure your continuous quality improvement efforts is to utilize the Institute for Healthcare Improvement (IHI) Model for Improvement to start small using Plan-Do-Study-Act (PDSA) cycles and test small changes to your healthcare system and processes related to improving your organization’s suicide care.
In addition to suicide death and suicide attempt rates, Zero Suicide organizations should measure, evaluate, and improve outcomes, workflow, policies, and processes around things like:
- Training plans
- Screening, assessing, and risk determination
- Safety planning
- Means safety counseling
- Suicide care pathway
- Care transition processes and interventions
- Suicide death and suicide attempt review process
- Postvention activities
- Confidence and competence of staff
Data
Zero Suicide is a data-driven continuous quality improvement framework.
Data-driven refers to the use of quantitative and qualitative information to measure progress and success of the implemented policies, processes, and workflows of the organization’s suicide care improvements.
These can be divided into two types of outcomes: individual level and process level outcomes. Individual level outcomes include the rate of suicide deaths and suicide attempts. Process level data include things that can provide insight into the progress of implementation and whether the policies, processes, and workflows are being sustained to fidelity.
Information can be gathered in a multitude of ways include reports from the electronic health record (EHR), chart reviews, observation of staff, fidelity audits, data collected from external sources (e.g., public health department, coroner or medical examiner’s offices).
After collecting the information, it should be analyzed within a Just Culture. Organizations want to look for the gaps in care provision to understand what areas need to be addressed. An organization with a Just Culture doesn’t ignore blatant disregard of fraud, gross misconduct, etc. but recognizes that humans make mistakes, and that the system should seek better ways to support staff to reduce errors and increase the safety of those in care. Often staff need additional training, reminders, or other supports to address barriers like discomfort working with people who have suicidal thoughts that are causing the gaps in care.
Zero Suicide Data Dashboard
One way to collect, analyze, and report Zero Suicide data is to use the Zero Suicide Data Dashboard. This is an online data reporting system to assist health and behavioral health care organizations to monitor their data that is specific to the implementation and sustainment of the Zero Suicide framework. Data Dashboard users can tailor how they set up their Dashboard to ensure the data reflects how their organization’s suicide care improvements impact outcomes over time.
• Organizations input data on a quarterly time-frame to track the progress of implementation and sustainment of Zero Suicide policies and processes.
• The Dashboard includes key metrics that systems should use to monitor fidelity to the Zero Suicide framework.
• The Dashboard displays and summarizes data for multiple groups (i.e., different departments in the same organization or multiple clinics or hospitals within a healthcare system) over time.
For organizations that do not want to use the Zero Suicide Data Dashboard, the Zero Suicide Data Elements Worksheet provides guidance on appropriate data to collect.
Outcome Data
The main metric that is used to measure individual outcomes is suicide deaths. If an organization’s Zero Suicide efforts are creating change, the number of individuals dying by suicide should reduce. This is an important data point but also one that can take years to show a difference. In the beginning it is possible that there could be an increase in suicide deaths. The organization may be gathering more death data overall and therefore more aware of suicide deaths than before which can be the cause of an increase.
Obtaining Suicide Death Data
Many organizations rely on informal sources of death data like information from support people or published obituaries to collect suicide death data. An important step for a Zero Suicide organization is to develop formal relationships with their local vital statistics office, public health department, medical examiner or coroner’s office, and any applicable Tribal Government to obtain official death data.
Organizations would then cross-reference this data with individuals who have been in their care within a certain timeframe. For example, an organization’s policy might be to review any individual who has died by suicide within one year of separation from care. The organization should determine what the post-discharge time frame is appropriate for them, based on factors like accreditation and organizational policies.
A formalized relationship with a county, state or Tribal government ensures the most available death data. The State Suicide Prevention Coordinator often has access to information and agencies that can assist an organization collect appropriate suicide death data for review.
Beyond initial data collection
Data can be used to tell the story of how the organization is progressing in its Zero Suicide efforts and whether it is faithful to the model. One way to start that story is to ask questions. Once the questions are identified then the data points can be ascertained. Below are some sample evaluation questions.
Individual outcome questions:
- Are individuals who screen positive for suicide risk receiving a thorough risk assessment within the determined timeframe?
- Do individuals who are on the pathway or engaged in suicide specific treatment have lower no-show rates?
- Are we seeing progress for individuals who are in treatment? Reduction in ER visits, lower CSSRS or PhQ-9 scores?
Process measure questions:
- What is the comfort and confidence level of staff who are screening individuals?
- How often are standardized protocols overruled by clinical judgment?
- Is safety planning done collaboratively and is it seen as an intervention and not as a check-box thing?
These are just examples of potential evaluation questions that organizations can use to guide their data collection and analysis.
Continuous Quality Improvement (CQI)
Definition of CQI
Zero Suicide is a continuous quality improvement framework. An organization that with continuous quality improvement process regularly collects and reviews data to review the outcomes of processes and policies in addition to individual outcomes like the rate of suicide attempts and deaths. Quantitative and qualitative data is used to measure progress and drive process improvements to advance patient safety. The data is reviewed in a structured manner that contributes to the understanding of the gaps in care and guides improvements that will address those gaps.
Data can help save lives
If people are not asked if they are experiencing suicidal thoughts it is impossible to provide appropriate support and care. Data that illuminates the frequency of screening for suicide risk will help answer the question “are individuals at risk of suicide being identified?” and “are staff following the policy and/or procedure?” If it is found that the number of people being screened for suicide does not match the number that should be screened, then other data can fill in more information to answer the “why is this happening?” Collected and analyzed data should connect to an evaluation question. Don’t collect data just for the sake of having data.
TBD Solutions in Grand Rapids, MI along with Riverwood Center in Benton Harbor, MI worked in collaboration to develop a risk factor queue which utilizes data from their EHR. They identified multiple risk areas that includes missed suicide screenings. The risk factor queue informs whether the PHQ-9 and CSSRS were completed. The use of this tool allows them to catch gaps in care and address them quickly. They are also working to integrate the use of this tool into clinical supervision where supervisors can talk to staff and focus interventions on addressing identified risk factors and provide trainings on associated screening tools.
See the data tab in this selection (Improve) of the Zero Suicide Toolkit for more information about data.
Below is a diagram of a simple CQI process that could be used as a foundation to developing one that fits your organization.
Plan-Do-Study-Act cycles (PDSA)
One continuous quality improvement process organizations can use is the PDSA cycle. Following this process enables organizations to test small improvements in a way that allows for adaptations and adjustments as implementation occurs. The Institute for Healthcare Improvement (IHI) website offers more information on how to use this improvement process.
Fidelity and Sustainability
Fidelity and sustainability are important for any implementation. Once a workflow, procedure, policy, or intervention are implemented we want to make sure that they are continued (sustainability) and done the way they are supposed to be done (fidelity).
Fidelity: What is being measured?
First, assessing fidelity depends on what is being measured -- policy, procedure, workflow, or evidence-based intervention or treatment. Some interventions and treatments have been shown to work if they are provided in a specific way (e.g., screening tool, safety plan, therapeutic intervention). Many policies and protocols include specific steps to be followed. For those, fidelity is measured by how well staff follow the instructions. Workflows and procedures might have less structure and depend on the setting, staff member, or individual in care. Measurement of fidelity in this circumstance might be less structured.
There are two “buckets” to fidelity. One bucket is whether “the thing” is being done at all (the workflow, procedure, intervention/treatment). This information is often gathered from a report in the electronic health record (e.g., how many screenings have been done, how many safety plans have been done).
The other bucket is how “the thing” is being done. Are staff doing the safety plan collaboratively with the individual? Are staff using validation, empathy, and neutral verbal and body language while screening for suicide risk? These are often measured through self-report or observation.
Fidelity should also be used as part of the continuous quality improvement cycle. If staff members are not able to maintain fidelity to an intervention, policy, or workflow it should be evaluated to see if staff need additional support (e.g., refresher training, reminders) or if the intervention, policy, or workflow needs to be adapted to the population, setting, or circumstances (retaining the core components of the evidence-based intervention/treatment).
Once adaptations are made, fidelity should continue to be measured and evaluated. All should be done within a Just Culture, where the system takes responsibility for improving processes, workflows, and policies rather than blaming staff. Fidelity processes should include input from staff who are providing the intervention, doing the workflow, or fulfilling the policy.
Return to the Organizational Self-Study
The Organizational Self-Study is a tool that can be used to measure overall fidelity to the Zero Suicide Framework. It is recommended to do the OSS at the beginning of implementation and then annually.
Used alone, the results of the self-study will show where an organization’s suicide care practices are already effective and where they can be strengthened, which will inform the overall work plan. Completing the self-study every year will tell you and your organization how well you are adhering to the Zero Suicide model and point out the next areas that need strengthening.
Sustainability: Keep it Going
Once an organization invests in a large initiative like Zero Suicide it’s important to continue it. Organizations might focus on starting Zero Suicide and not focus on how to sustain it until after implementation resources (often grant funding) have ended.
Implementation scientists recommend teams plan for sustainability at the same time they plan for implementation. This can translate into “how do we get started and how do we keep it going?”
Planning for sustainability is important because often the benefits of an evidence-based intervention aren’t realized for many years after initial implementation.
Sustainability planning connects to continuous quality improvement because the cycle of measure, evaluate, and review helps maintain the Zero Suicide workflows, procedures, and interventions that were implemented.
Next Steps
Apply Data-Driven Quality Improvement
The basic next step to measure improvements in the quality of suicide care is to:
- Enlist the implementation team in developing an evaluation plan, including a plan to evaluate progress, using the Organizational Self-Study, and measure results, using the Zero Suicide Data Elements Worksheet.
Planning Next Steps
There are several additional items to help you plan these next actions:
Quick Guide to Getting Started with Zero Suicide
This one-page tool lists ten actions you can take to start implementation of Zero Suicide.
Getting Further with Zero Suicide
This tool lists several actions you can take if you have been implementing Zero Suicide for a while and are not sure what to do next or need help taking your Zero Suicide work a little further.
Zero Suicide Organizational Self-Study
Every organization should complete the self-study as one of the first steps in adopting a Zero Suicide approach. While the self-study is available in the Lead section of the Zero Suicide online toolkit, it’s provided again here for your convenience.
Zero Suicide Work Plan Template
This form contains an expanded list of action steps to guide your implementation team in creating a full work plan to improve care and service delivery in each of the seven core Zero Suicide components.
Recommendations for Improving Data Collection and Data Definitions
This resource highlights some common questions related to data collection and offers recommendations for improving data collection systems and data definitions. Use this tool to inform discussions about data priorities, data infrastructure needs, and defining quality improvement metrics.