RP307 - Implementation Science: A Process Evaluation to Guide the Design of Effective Interventions

Abstract:
We will introduce Implementation Science and describe a process evaluation conducted during an effective study of an internet-based intervention for adults with unaddressed hearing impairment. An effectiveness study evaluates interventions in ‘real-world’ settings to determine if clinicians and potential participants will take up and complete the intervention. Because of low uptake, a process evaluation was completed to determine if the intervention design was faulty. We describe interviews with providers, marketing experts, and participants, plus analysis of Google Analytics from study webpages and electronic medical record referral counts. Suggestions for the delivery of health interventions over the Internet will be discussed.

Rationale:
Sometimes researchers develop a treatment that is shown to be effective in the laboratory, but clinicians don’t use it because it doesn’t fit into their service delivery system. Alternatively, researchers may think they have developed a great new treatment but cannot determine efficacy because individuals (with the condition) choose not to participate in the research study. Implementation science is the study of procedures to determine the strategies that result in the uptake and use of health interventions in the community. Process evaluations are a vital component as they may determine why an intervention is not taken up and used by clinicians or potential participants. Process evaluations are used to evaluate public health interventions that are implemented in the community using existing systems to distinguish between inherently faulty interventions and those that are poorly delivered (implementation failure). We will describe a process evaluation that was conducted after an effectiveness study of an Internet-based Decision Coaching Guide (DCG) to aid older adults in deciding whether to visit an audiologist. Effectiveness studies are conducted in a ‘real-world’ setting with investigators having no contact with and no payment to potential participants.

Methods:
For the effectiveness study, we recruited participants from four different arms that varied in terms of trust, cues to action, and accessibility to determine how these factors affected rates of uptake and completion of the DCG. The four arms included an Optimal Aging Center (high trust, high cue to action), Primary Care Physicians (PCP) office (high trust, high cue to action), informational lectures in the community (moderate trust, limited cues to action) and Social Media (e.g., Google ads; low trust, low cue to action, high accessibility). In each setting, potential subjects were screened for hearing loss (with the single question: Do you think you have a hearing loss right now?) and given instructions to access the DCG on the internet. We hypothesized that locations with higher trust and cues to action would have a higher uptake of the DCG. Despite 18 months of recruitment and adjustments to the process, only 2 subjects enrolled in the study. Midway through the study, the research team began a process evaluation to understand the barriers and facilitators that led to these outcomes. We conducted interviews with providers and staff at study locations, with participants, and with social media marketing experts. Google Analytics was analyzed for virtual recruitment methods and tracking outcomes after making changes to study landing pages. Finally, electronic medical records (EMR) were analyzed to determine the impact recruitment efforts had on provider referrals to audiology before, during, and after the study period.

Results and

Conclusions:
Our findings suggest that the cue to action from some trusted providers, e.g., PCP, was strong enough for potential participants to skip the DCG and make an appointment with audiology. However, in other cases, the trusted provider did not recommend the use of the DCG. Study results and previously published research will be reviewed to guide the uptake of internet-based treatments in the future.

Learning Objectives:

• describe how an internet-based decision coaching guide could be delivered to adults with unaddressed hearing impairment.
• describe how process evaluations can be a valuable component to evaluate failed effectiveness intervention studies.
• describe a process evaluation conducted after an effectiveness study of an internet-based guide aimed at increasing visits to an audiologist after a failed hearing screening.