Evaluation Report
November 25, 2020
Introduction
This evaluation report represents progress made by the Behavioral Health Administration’s Hawaiʻi Opioid Initiative (HOI) based at the Alcohol and Drug Abuse Division (ADAD) of the Department of Health and reflects the work of 193 volunteer members from 106 organizations located across the Hawaiian islands to mitigate the impact of opioid use disorder (OUD) on patients, communities and systems of care.
Epidemiologic Landscape
The context of OUD in Hawaiʻi is established with a series of data-informed inquiries about OUD, exploring factors associated with opioid overdose such as prescription pick up timing, comparison to other substance use disorders and barriers to treatment and primary care in rural areas. Following this, a Hawaiʻi-focused data dashboard from the National Survey of Drug Use and Health is introduced.
HOI Scorecards
Each year the HOI sets work objectives for the year by focus area. The scorecards track 2018’s 1.0, 2019’s 2.0 objectives, and 2020’s 3.0 objectives which are currently in progress. They include contact information for each focus area’s co-chairs. Anyone interested in supporting the HOI is encouraged to join a work group by contacting the co-chairs.
Evidence-Informed Strategies
This section presents current evidence and literature was reviewed for selected priority areas identified in the HOI Scorecards. The synthesis of evidence has been summarized in standalone infographics, which may be requested by the Work Groups as part of their objectives. Topic areas include alternative pain management therapies, care coordination, Medication-Assisted Treatment, prescriber education, and cultural competency.
Achievements
Presented as a series of infographics, this section inventories progress made on the HOI 1.0 and 2.0 objectives as documented by the evaluation team through survey, interview, and attendance of all HOI Work Group meetings. Highlights focus on naloxone training and distribution, the Prescription Drug Monitoring Program, “Mocha Minutes” prescriber education, Take Back Boxes, a pilot of Maui Police Department’s Overdose Detection Mapping Application Program (ODMAP), and the launch of the Hawaiʻi CARES (Coordinated Access Resource Entry System) line.
Overall Successes
Major successes of the HOI include education and training initiatives, interagency collaboration, and continuing a process of desiloizing our health system that may be replicated to address other complex public health problems.
Challenges
Two policies from the 2019 legislative session directly impact the HOI: Act 154 “Relating to Pharmacists Prescribing and Dispensing of Opioid Antagonist” and Senate Concurrent Resolution 103 “Urging the Inclusion of Native Hawaiian Cultural Intervention Treatment Programs, Wellness Plans, and Holistic Living Systems of Care in the State of Hawaiʻi’s Response to the Rise of Misuse and Abuse of Opioids or Illicit Substances in Hawaiʻi.” An overarching goal of this evaluation is to gather feedback for HOI capacity-building and policy enforcement.
Recommendations
In 2020, increasing member diversity, cultural competency, project alignment, evidence-informed strategy, current programming and additional policy-driven programming are recommended to sustain the success of the HOI.
Pūpūkahi I Holomua Unite to Move Forward
The opioid epidemic continues to be a major public health and safety challenge impacting thousands of people in the State of Hawaiʻi. Granted, the epidemic in Hawaiʻi continues to be less severe than the continental U.S. In 2018, the drug overdose death rate (age-adjusted) in Hawaiʻi was 14.3 deaths per 100,000 people, compared to a national age-adjusted rate of 20.7 deaths per 100,000 people. Nevertheless, opioid misuse continues to needlessly kill and harm people in Hawaiʻi. In 2018, 213 people died from a drug overdose in Hawaiʻi, killing more people than kidney disease (183 deaths), making OUD a significant public health challenge.
Opioid use disorder (OUD) also has significant economic consequences. In the U.S., the estimated overall economic burden of opioid use disorder was $78.5 billion in 2016. A significant contributor of these costs is the higher utilization of health care resources, with the national average cost to an employer for an OUD patient of $10,627 per year. OUD continues to disproportionately impact different groups such as older adults, men, certain ethnic groups, low-income individuals, and Medicaid beneficiaries.
The Hawaiʻi Opioid Initiative (HOI) was established through the joint effort of Governor David Ige and the Hawaiʻi State Department of Health in July 2017. This initiative aims to address the opioid epidemic with a coordinated and proactive response that incorporates different stakeholders, approaching the issue from several angles and sectors. The Hawaiʻi Opioid Initiative approaches the epidemic from seven Focus Areas and corresponding Work Groups:
Work Groups are organized for each Focus Area and convene members voluntarily from different sectors and agencies with differing expertise and interdisciplinary perspectives to address the opioid epidemic in Hawaiʻi.
Since 2017, the University of Hawaiʻi at Mānoa Myron B. Thompson School of Social Work has served as the Evaluation Team of the HOI. This UH HOI Evaluation Team supports the coordination of activities and technical assistance to the HOI Work Groups. This team released its first Evaluation Report in 2019 initially funded by the Centers for Disease Control and Disease Prevention Data-Driven Prevention Initiative (DDPI).
In 2020, this Team conducted an evaluation with an emphasis on the HOI Objectives through a mixed methods approach including document review, surveys, and key informant interviews. Funding for the evaluation of the HOI came from the U.S. Substance Abuse and Mental Health Services Administration (SAMHSA) State Opioid Response (SOR) grant.
ADAD requested this Evaluation Report to assess the successes and challenges and to explore recommendations for the HOI. The purpose of the evaluation is to understand how to better serve the people of Hawaiʻi by mitigating the impact of opioid misuse on our communities with a specific focus on breaking down silos in our health system to improve health care delivery.
Evaluation seeks to assess the effectiveness of public health programs, by identifying what is working well, opportunities for improvement, and potential for innovation. Evaluation can help to identify barriers and areas for improvement. To sustain the successes of the HOI, this evaluation will be a tool to better understand how the Work Groups organize to achieve outcomes. As a result, other statewide initiatives may learn and replicate this methodology in addressing other multi-sectoral public health and public safety issues.
Since the Initiative’s implementation in July 2017, the UH HOI Evaluation Team has monitored the progress and completion of the objectives outlined in the HOI Action Plan, a living document that is updated and changed each year with the progress of the Initiative. This evaluation report identifies areas of achievement and challenges in implementing the Action Plan.
A complex issue such as opioids requires a complex system of multi-sectoral agencies that engages various community members and organizations. This evaluation report is a tool of accountability for the many stakeholders involved in this initiative. The 2020 Evaluation Report allows Work Group Co-chairs to voice recommendations and articulate the successes and challenges they have faced between January and December 2019. Furthermore, this evaluation intends to provide insights on how focus areas continue to evolve and how Work Groups enable collaboration across sectors and agencies.
The UH HOI Evaluation Team is led by Principal Investigator Dr. Victoria Fan (Center on Aging) and Co-Principal Investigator Dr. Seunghye Hong (Department of Social Work) with Katherine Burke as lead evaluator and junior epidemiologist, Dr. Andrew Abe as junior epidemiologist, and a group of undergraduate and graduate students.
Like the 2019 Evaluation Report, the 2020 Evaluation Report has four key domains: (1) Epidemiologic Landscape, (2) Document Review, (3) Data Collection, and (4) Synthesis.
The Epidemiologic Landscape is aggregated as an interactive data dashboard based on the National Survey on Drug Use and Health (NSDUH) to enhance our data-driven understanding of statewide opioid use. The Evaluation Team also obtained epidemiologic analyses from the Emergency Medical Services and Injury Prevention System Branch (EMSIPSB) epidemiologist Dr. Daniel Galanis as part of its assessment, contextualized with the social determinants of health.
The Document Review is based on the consistent service of participant observation provided by the UH HOI Evaluation Team. Coordination activities for the HOI included preparing meeting agendas, meeting minutes, and other logistics convening stakeholders spanning various sectors, disciplines, expertise, and backgrounds of the seven Work Groups, and the Operational Work Group. Remarkably, the activities and meetings of the HOI are not required by statute but rather are organized voluntarily through pertinent agencies. As part of the documentation review for this evaluation report, the UH HOI Evaluation Team requested additional documentation from ADAD on its activities and contracts as part of the HOI.
The 2020 Evaluation Report has an expanded Data Collection methodology including: (1) the Pharmacist Survey on Naloxone, (2) an HOI Work Group Member Survey, and (3) Key Informant Interviews.
The Pharmacist Naloxone Survey intended to assess the impact of Act 154 “Relating to Pharmacists Prescribing and Dispensing of Opioid Antagonist,” one of three opioids-related bills passed during the 2019 legislative session, that focused on naloxone distribution. With the support of Work Group 5, this survey was disseminated by the Hawaiʻi Pharmacists Association to assess the knowledge, awareness, and training needs associated with Act 154 as well as to assess successes, challenges and opportunities to ensure equitable naloxone access statewide. The performative scope of this evaluation strategy is to gather information from key stakeholders (pharmacists), while raising awareness of naloxone distribution (see Annex 2 for the Pharmacist Naloxone Survey). This online survey was conducted from June 8 to July 6, 2020, canvassed all 1113 active and non-active Hawaiʻi Pharmacists Association members, of which 58 are active pharmacists. In total, 18 respondents completed the survey.
The HOI Work Group Member Survey was distributed to invite all 186 Work Group members to provide feedback on HOI 2.0 Objectives progress, successes, challenges and recommendations (see Annex 2 for the questionnaire). The survey was distributed twice via email to all members during the month of June.
Key Informant Interviews were conducted with ADAD-contracted providers and staff as well as the Native Hawaiian Health Care System (NHHCS) on each of the neighbor islands (Hawaiʻi, Maui, Molokaʻi, Kauaʻi, Lanaʻi) and Oʻahu. ADAD providers were recruited from monthly meetings, and NHHCS providers were recruited by the executive director of Papa Ola Lōkahi. The UH HOI Evaluation Team began meeting with providers in July 2020. An interview guide was prepared and a team of five trained undergraduate students were trained by the lead evaluator to complete interviews in pairs via Zoom. The interview guide inquires about awareness of the HOI, the relevance of the 2019 epidemiologic data with their current patient population, the recruitment to and participation in the HOI, and finally, recommendations. See Annex 3 for the interview guide and background information sheet. A total of 14 providers have been interviewed. Interviews will be transcribed, coded for themes, and summarized.
All methods of data collection included two additional focus areas. Questions were included regarding (1) the COVID-19 pandemic that began during data collection in March 2020 in order to maximize the opportunity to learn from providers about impacts and challenges in ensuring continuity of care; and (2) inputs on the Senate Concurrent Resolution 103 (2019) “Urging the Inclusion of Native Hawaiian Cultural Intervention Treatment Programs, Wellness Plans, and Holistic Living Systems of Care in the State of Hawaiʻi’s Response to the Rise of Misuse and Abuse of Opioids or Illicit Substances in Hawaiʻi” to assess training needs, perceptions, and readiness.
Chapter 2 “Epidemiologic Landscape” presents NSDUH data and summarizes data presented by Dr. Daniel Galanis in an infographic. Chapter 3 “Scorecards 2.0” presents the Focus Area and Work Group objectives while tracking their overall progress. Chapter 4 “Evidence-Informed Strategies” provides current evidence and literature reviews on select HOI priority items. Chapter 5 summarizes the achievements of the Focus Areas and Work Groups. Based on an overall synthesis from the previous chapters as well as the different data collection sources, Chapter 6 discusses overall successes of the HOI, Chapter 7 discusses challenges of the HOI, and Chapter 8 makes recommendations for the future of the HOI.
CLICK TO ENLARGE
CLICK TO ENLARGE
CLICK TO ENLARGE
CLICK TO ENLARGE
The overall successes, challenges, and recommendations sections of this evaluation report are synthesized from the previous sections of the report including primary data collection (pharmacist naloxone survey, Work Group Member Survey, and the key informant interviews), with greater emphasis on the Work Group Member Survey.
The 2019 Evaluation Report identified several key themes of HOI activities including the importance of inclusive leadership, interagency collaboration, a platform for action through effective partnership, and stakeholder networking.
Of the 24 Work Group members who responded to the Work Group Member Survey, the majority of participants at 54.2% reported their Work Group 2.0 objectives as “mostly complete”, while an additional 20.8% reported “complete,” and 16.7% reported “partially complete.” Only 8.3% reported the objectives as “mostly incomplete.” No participants reported incompletion of the 2.0 objectives.
Based on the Document Review of minutes and discussion with Work Group Co-Chairs, 6 of the 7 Work Groups had at least one HOI 2.0 objective and 2 of the 7 Work Groups had at least one HOI 1.0 objective that is in progress, indicating the need for further work in subsequent years.
Work Group members generally perceived the purpose of their Group in four key areas:
Members identified the achievements of their specific Work Group as follows:
As an interagency collaboration, the HOI has a wide reach across 106 public and private agencies. Since launching in July 2017, the HOI has provided a platform for agencies to join forces to tackle complex public health problems such as opioid use disorder.
As noted in the 2019 HOI Evaluation Report, the HOI draws on a broad and active multisectoral membership across multidisciplinary Work Groups enabling agencies to “de-siloize” and “link and sync” by meeting regularly to discuss shared objectives, overcome barriers and maximize resources. For example, one Work Group member stated that her participation informs her work to increase access to treatment and prevention among rural health organizations. The concept of a community of practice rises to the forefront as members extend their reach beyond their own agencies, beyond their own islands, by collaborating on shared goals.
This community of practice is equipped with high quality education on OUD, pain management, SBIRT, Medication Assisted Treatment (MAT) and naloxone, disseminated through Project ECHO, Mocha Minutes and the Hawaiʻi Health and Harm Reduction Center to train providers as well as law enforcement and probation officers. Increasing awareness of evidence-based approaches to treatment and prevention is aimed at creating a unified response to OUD.
While evaluating the impact of HOI on lives saved remains a work in progress, impacts on the public’s sense of agency to take action to stop OUD or reverse an overdose is evident. As one provider noted, the most essential program of the HOI to their patient population is “the Take Back program,” adding, “through our prevention programs we promote for both youth and parents, members at school, school staff. We have folks associated with the opiate initiative that do talks with our staff about opioids and access to naloxone.” Another provider stated, “We know when people pass away, we make sure we get the word out for them to bring in their unused prescription drugs from their loved one to take it off the street. I also work closely with the care homes and hospitals to make sure they know about our Take Back day so they can bring in their expired drugs as well.” Since July 2019 – the state Medication Drop Box Program, which began in July 2018, has accumulated approximately 3,000 lbs of excess or unused prescription drugs from across all four counties (Kauaʻi, Honolulu, Maui, and Hawaiʻi). In 2019, a total of 448 individuals completed naloxone training and of those, 157 took home naloxone kits. In total, 1,172 kits were distributed by the HOI. This in addition to the prescribing authority now extended to pharmacists suggests that Hawaiʻi will become more capable of responding to opioid misuse in real-time and take the necessary actions to save lives.
Work Group activities are supported by supplemental financial support and technical assistance offered by interagency partnerships. The Overdose Data to Action grant (OD2A), which champions many Focus Area 3 objectives, continues to pursue collaborations that prioritize timely access to morbidity and mortality data from hospitals and medical examiners so that the impact of the HOI on fatal and non-fatal opioid overdose may be better understood. With access to data in real time or of sufficient geographic granularity emerging within the coming year, the use of evidence synthesis to demonstrate the importance of the chosen HOI interventions and strategies can soon be used to estimate and model plausible impacts.
Challenges identified in the 2019 Evaluation Report included: activity tracking; evolving scope; voluntary membership; state procurement and administrative processing; sustainability; inclusive voices; new models of care; best clinical practice; disproportionate participation; and insurance billing and reimbursement.
In the 2020 evaluation, the most common areas of challenges reported by members were in four areas:
Three key themes emerged from respondents regarding the challenges of the COVID-19 pandemic:
The UH HOI Evaluation Team recognized COVID-19 as a potential challenge to Work Group progress on HOI objectives and added a question on how the pandemic is affecting Work Group member’s practice and continuity of care and delivery to the survey and interview tools. One Work Group member noted that many objectives were “slowed or interrupted” because of the immediate need to divert resources to the pandemic response.
Work Group members acknowledged that face-to-face interactions and direct patient care had certain limitations and were especially difficult in the treatment of elderly patients and youth. One provider interviewee noted that some of their older clientele with other comorbid conditions, like HIV, that further complicate their treatment often require greater management of care. Using a web-based platform for community-facing services, while a viable solution to bridging the communication gap, presented challenges to those who struggle with learning new technology and those without access to internet or mobile devices, which is prevalent among vulnerable populations. Work Group members pointed out that they were either lukewarm about the quick shift to virtual meetings, or had strong aversion to it as it felt “forced” on them and “destroyed [their] practice.”
The pandemic brought along a sense of uncertainty for many Work Group projects, and even work beyond the HOI. The priorities pivoted to more urgent COVID-19 response work, leaving the opioids-related work with longer waiting times. As one Work Group member noted, they switched to “reactive mode” and had to put their own work on hold. Some of the causes for their standstill include “slow responses with other agencies, delivery of goods, [issues with] Zoom, 12 step programs, etc.” The programs that were directly involved with the Department of Health’s COVID-19 response had to limit their community outreach/education efforts and non-essential home visiting.
Half of the respondents felt that some challenges to procurement were out of their control. Two Work Group members noted that problems with funding caused significant roadblocks in their progress. One Work Group member mentioned that the amount available for the campaigns was reportedly changing, whereas another Work Group member noted that procurement delays for the Mocha Minutes “made it fail.” One Work Group member cited contract issues with Med-QUEST for SBIRT as a Work Group specific challenge.
An external circumstance cited was the difficulty with gathering accurate data on drug overdoses and deaths as well as with providing nonmedical overdose data to law enforcement officers in a timely fashion.
A third of surveyed Work Group members agreed that overall communication and logistics could be improved. Communication challenges stemmed from specific Work Group activities, such as miscommunication in Work Group 5 on the number of individuals each managed care organization (MCO) was expected to train, and the responsiveness of communication from DOH, ADAD, and UH. Because the HOI is system wide, the scope of audiences intended to access the website is broad which resulted in the need for in-depth feedback from multiple stakeholders, culminating in a delayed go-live date.
One Work Group member reported that sustained engagement in HOI is a challenge and recognized that membership was essential to “ensure that [Work Group 1 members] are all on the same page with MAT and treatment services.”
COVID-19 is a significant challenge for ensuring continuity of care and addressing and reducing the risk of opioid overdose among Hawaiʻi’s population.
Of the challenges identified in 2019, issues of state procurement and administrative processing continue to be a main priority.
The dynamics of coordinating and supporting the communication and logistics of 7 distinct, dynamic Work Groups is an area that the DOH and UH HOI Evaluation Team continue to work to improve, particularly in the face of evolving scopes and voluntary roles of Work Group members.
The Work Group meeting participation analysis by the UH HOI Evaluation Team indicates that disproportionate participation and ensuring inclusive voices also continues to be a challenge.
In addressing new models of care and best clinical practice, the UH HOI Evaluation Team has conducted focused literature reviews on topics relevant to the HOI or at the request of Work Groups including on new models of care, pain management approaches, etc. More work is needed to further translate knowledge to action and implementation.
CLICK TO ENLARGE
This chapter summarizes recommendations by Work Group members and interview participants for improving the HOI.
Increasing Work Group member diversity by invitation to community members is a recurring recommendation that carries over from the 2019 HOI Evaluation Report. New member recruitment could occur on a scheduled basis, and a system could be put in place to formally support the stepping-down of old members akin to a passing of the baton each year.
This sentiment was echoed by a current Work Group member, who suggested that an individual from the private sector “steer [the] project[s] in the direction the state wants it to be.” Targeted recruitment of members with backgrounds representing an array of disciplines including managed care groups and neighbor island providers has been a strategy utilized by the Treatment Access Work Group to diversify their member population.
Specific to cultural competency, including experts in Native Hawaiian health and healing is strongly recommended to achieve a culturally relevant, common language and understanding among HOI members, such as those pertaining to federal regulatory guidelines and the requests of Senate Concurrent Resolution (SCR) 103 (2019).
To meet HOI 3.0 objectives created in response to SCR 103, it is recommended that a consistent definition of cultural competency be used to ensure that all HOI members have received cultural competency training. Additional continuing education units may be offered to address gaps in knowledge identified through the surveys in the areas of “Awareness of Native Hawaiian cultural practices,” “Understanding of Hawaiian history” and “Linguistic Ability in ʻŌlelo Hawaiʻi (Hawaiian language).” There was a clear difference among survey and interview participants regarding completion of cultural competency training and proficiency in these three areas such that those that had completed training affirmed greater proficiency than those who did not. Specific to prescribers, “Consideration of Native Hawaiian herbs and therapy” is a recommended, targeted application of cultural competency to practice.
Because cultural competency training was not established until 1989, this may be of particular need and benefit to those who completed professional training prior to that year. The HOI is encouraged to seek partnerships and collaborate with Papa Ola Lōkahi and the Native Hawaiian Health Care System to meet these recommendations through member recruitment and application of the 2019 Task Force Recommendations for Impact presented in E Ola Mau a Mau: The Next Generation of Native Hawaiian Health.
Based on information identified through a systematic review of the literature, the objectives and programs of the HOI have been in line with best evidence-based practices.
Recommendations from Work Group members included the need for stronger adherence to the HOI objectives in the following ways:
Having a point person for meetings from the UH HOI Evaluation Team who not only understands the culture of the Work Groups but is also consistently responsive via email has been instrumental to their success. Moving forward, the UH HOI Evaluation Team acknowledges the value of this service to the overall cohesion of the HOI.
Work Group members made the following suggestions to increase impact of the HOI:
CLICK TO ENLARGE
CLICK TO ENLARGE
This document is the evaluation report of the Hawaiʻi Opioid Initiative, a partnership led by the Alcohol and Drug Abuse Division (ADAD) of the Hawaiʻi State Department of Health (DOH) with more than 100 different stakeholders and numerous public and private agencies across the state.
This report is authored by Katherine Burke, Andrew Abe, Charmaine Milla, Keani Valdez, Edra Ha, Samantha Lumbao, Cielo Subia, Shelby McKee, Rachel Untalan, Shelley Liu, Mary Guo, Amber Ichinose, Deveraux Talagi, Joel Nicolow, Sarah Yasuda, Jaclyn Topinio, Rojelle Bohol, Taylor Puʻuohau, Craig Yamaguchi, Daniel Galanis, Angela Bolan, Seunghye Hong, and Victoria Y. Fan.
The authors are grateful to Edward Mersereau, Ramon Ibarra, Katia Worley, Tammie Smith, John Valera, Amy Curtis, Joshua Holmes, Jared Redulla, Gary Yabuta, Valerie Mariano, Marc Alexander, Mike Kobayashi, and all the executive steering committee members and Work Group Co-chairs and members for all their contributions to the Hawaiʻi Opioid Initiative and their participation in this evaluation report. The authors are also grateful to the Alcohol and Drug Abuse Division, the Substance Abuse and Mental Health Services Agency (SAMHSA) State Opioid Response as well as the ADAD State Plan for Substance Use Treatment Project. All errors and omissions are our own. The University of Hawaiʻi Human Studies Program approved the evaluation of the Hawaiʻi Opioid Initiative as exempt on July 9, 2020 (2018-01094).
The PI of the UH Hawaiʻi Opioid Initiative Evaluation Team is Dr Victoria Fan and the Co-Principal Investigator is Dr Seunghye Hong of the University of Hawaiʻi at Mānoa, Myron B. Thompson School of Social Work, Center on Aging, Pacific Health Analytics Collaborative. Please send all comments on the report to the UH HOI Evaluation Team at phac+uheval@hawaii.edu.
Copyright © University of
Hawaiʻi at Mānoa. 2020. Some Rights Reserved.
Pacific Health Analytics Collaborative
Center on Aging
Myron B. Thompson School of Social Work
University of Hawaiʻi at Mānoa
1960 East-West Road
Biomed C105
Honolulu, HI 96822
Email: phac+uheval@hawaii.edu
Recommended citation:
University of Hawaiʻi at Mānoa.
Hawaiʻi Opioid Initiative Evaluation Report.
Report Submitted to the
Alcohol and Drug Abuse Division
Hawaiʻi State Department of Health.
November 25, 2020.
CLICK TO ENLARGE