STAKEHOLDER ENGAGEMENT NAVIGATOR

DICEmethods.org | Dissemination, Implementation, Communication, and Engagement
A guide for health researchers

Home Education Hub Find Engagement Strategies Glossary Contact Us
Home fal fa-angle-right Stakeholder Engagement Education Hub fal fa-angle-right Background and development of the Stakeholder Engagement Selection Tool EDUCATION HUB FIND ENGAGEMENT STRATEGIES BACKGROUND

In April and May of 2017, researchers from the D2V Engagement Core held interviews about stakeholder engagement with D2V’s pilot project awardees. (D2V funds 1-year grants for researchers who have projects that contribute directly to our program’s mission to bridge the disciplines of data science and health outcomes research.) During these interviews with pilot project awardees, we found that stakeholder-engagement—as defined on our "Defining Stakeholder Engagement" page (link here)—is not something researchers are necessarily comfortable with, budget for, or know how to do.
Based on this information, our group set out to create an online toolkit to help healthcare researchers identify and select methods of engagement based on the constraints and/or criteria of their studies and their identified stakeholders to promote equity in research.


DEVELOPMENT

In order to approach this as a user-centered designed toolkit, the D2V Engagement Core interviewed principal investigators interested but inexperienced in stakeholder engagement research.

We then determined the following components important to choosing a method of engagement for a research topic:

  1. Does it fit within our definition of stakeholder engagement?
    • Stakeholder: An individual or group who is responsible for or affected by health-and healthcare-related decisions that can be informed by research evidence.
    • Engagement: A bi-or multi-directional relationship between the researcher and stakeholders that results in informed decision-making about the selection, conduct, and use of research.
    • Longitudinal: At least some of the same stakeholders are engaged more than once during the study/engagement period
  2. Is method specifically designed for one or more of the following purposes of engagement?
    1. Identify and explore new perspectives or understanding
    2. Identify which topics are most important to stakeholders
    3. Develop research questions relevant to stakeholders
    4. Select outcomes and measures that matter to stakeholders
    5. Refine and help implement effective recruitment strategies
    6. Expand and diversify stakeholder outreach
    7. Investigate best ways to successfully implement a study, based on stakeholder insights
    8. Describe findings in ways stakeholders can understand and use
    9. Disseminate findings to relevant audiences
    10. Design strategies for translating research into practice
    11. Other
  3. What is the resource intensity of using this method? More specifically:
    • To what extent does this method require substantial cost (e.g. personnel, space, equipment)?
    • What is the general duration of engagement using this method? How long does it take to conduct this method?
    • How much training/experience is needed to conduct this method appropriately?
  4. How strong is the evidence base supporting the use of this method?

We then adapted the NIH-style grant review method to rigorously compile, review, and categorize engagement methods and tools based on their strengths and weaknesses regarding consistency with the definition of engagement, generalizability for multiple purposes of engagement, resource intensity, and the strength of the evidence base.

  • First, the D2V Engagement Core reviewed extensive literature surrounding engagement methods in order to create a preliminary list of approximately 40 methods/tools. Then, the list was reviewed and refined to 25 methods that could be defined more narrowly as engagement. Approximately 12 healthcare researchers were then interviewed about their use of engagement to ascertain the best way to disseminate our toolkit. A methods review form was created, and each method was given to two engagement core members to review.
  • After individual review, the methods were presented, assessed, and voted on by eight members of DICE.

After completing the review sessions to categorize engagement methods, a distinction was made between engagement approaches, methods, and tools. More on the differences between approaches, methods, and tools can be found here

A sample of 21 researchers was surveyed regarding the sorts of features they would like to see in our toolkit. Respondents overwhelming reported that when determining what stakeholder engagement method to use for their research, the most important domain is purpose of engagement. Based on this insight, we decided that our selection tool would first ask users about the purpose of their engagement. We then developed an organizing framework for the website, which can be seen on the following page.

This organizing framework was used to create a prototype of the website, which was used in two rounds of user-testing:

  1. Contextual Inquiry:

    A user-testing method in which users engage with the interface in the context of their own environment. Users are first asked a set of semi-structured interview questions, and are then observed and questioned while they engage with the interface. This method is helpful for identifying what works and doesn’t work, improving processes, and learning about what is important to users.

    • Conducted with 4 user-testers
    • Audience: Junior to mid-level investigators who want to learn about an engagement method for a project and who are not familiar with engagement methods
    • Findings were used to inform the next iteration of the prototype, which was handed off to a team of web developers. The first version of the website was used in the next round of user-testing, using the ‘Think Aloud’ approach.
  2. Think-Aloud:

    A user-testing method in which users are asked to verbalize their thoughts, feelings, and opinions as they perform predetermined tasks. This method provides an understanding of how the user approaches the interface and what considerations the user keeps in mind when using the interface. Often, the terminology the user uses to express and idea or function can be incorporated into the product design. This method is helpful for understanding users' expectations and identifying what aspects of the interface are confusing.

    • Conducted with user-testers. Some user-testers were invited back from the first round, while some were new.
    • Audience: Junior to mid-level investigators who want to learn about an engagement method for a project and who are not familiar with engagement methods
    • Results from this final user testing session will be used to inform the final version of the website.

The Stakeholder Engagement Navigator is a service of the Data Science to Patient Value Initiative at the
University of Colorado Anschutz Medical Campus

About us | Find us on