What If?: Transforming Diagnostic Research by Leveraging a Diagnostic Process Map to Engage Patients in Learning from Errors

 

The Problem

Researchers estimate that up to 80,000 deaths per year in U.S. hospitals can be attributed to some form of diagnostic error [1]. Misdiagnosis affects 12 million Americans in ambulatory care settings annually. The National Academies of Sciences, Engineering, and Medicine’s (the National Academies) 2015 report Improving Diagnosis in Health Care [2] highlights the urgent need for a research agenda on the diagnostic process and diagnostic errors. The report concludes that “patients are central to the solution,” and highlights the critical need to establish partnerships with patients and families to improve diagnosis [2].

 

The Challenge

Patients and family members with lived experiences of diagnostic error have unique perspectives with which to view the process of diagnosis—a perspective that has historically not been considered during the conceptualization or development of diagnostic research projects. This failure can be attributed to two fundamental reasons: the first reason is that involving patients in research development requires a culture change that recognizes the benefits of including patients on the research team as subject-matter experts, rather than simply as subjects or “victims.” Traditionally, academic medical research has followed a model of researchers serving as subject-matter experts and patients as the objects of or beneficiaries of the research. In this paradigm, patients are not engaged as active participants and researchers conceive and prioritize projects based on their perceptions of research gaps, what intrigues them professionally, and what findings they think could advance the field. This model fails to consider asking patients what questions they want answered or what advances in medicine would matter most to patients and their families. Outputs from this research model may have significant limitations because they are not informed by the in-depth understanding that patients and families have from their lived experiences. The second reason why patient and family members have not been routinely integrated as research partners in the design, execution, and dissemination of research is related to the lack of organizational knowledge of ways to effectively implement methods that engage the patient community in diagnostic research.

The Society to Improve Diagnosis in Medicine (SIDM) has, from its founding more than a decade ago, sought to include patients as partners in understanding and preventing diagnostic errors. Recently, researchers joined forces with trained patients and family members with lived experience in diagnostic error to co-produce diagnostic safety research projects, positing that hypotheses and outcomes would be more relevant, effective, and patient-centered and that the methods to test these hypotheses would be broader than those postulated without the unique input of patients [3].

 

A Solution

With funding from the Patient-Centered Outcomes Research Institute (PCORI) and in collaboration with Project Patient Care and the Medstar Health Institute for Quality and Safety, SIDM created the Patients Improving Research in Diagnosis (PAIRED) project to co-produce an innovative curriculum to educate Patient Partners in research methodology, including how to participate in the design, execution, and dissemination of research to improve diagnosis [4,5]. The PAIRED project team recruited 20 patients and family members (Patient Partners) from key patient organizations and representatives from patient and family advisory councils (PFACs) at major health care institutions. The Patient Partners represented diverse geographic areas, age groups, genders, ethnicities/races, and different lived experiences with diagnostic error. Prominent diagnostic researchers from four academic medical centers also participated in the PAIRED project as Research Mentors. Together with the PAIRED project team, the Patient Partners and Research Mentors co-developed a curriculum to provide patients and family members with the knowledge, skills, and tools to effectively partner with researchers in the design, execution, and dissemination of diagnostic patient-centered outcomes research (PCOR) and comparative effectiveness research (CER). The objectives of this curriculum included:

1. Teaching the fundamentals of research methodology, including research question development and study design;
2. Describing the ways patients and family members can function as true partners in the research process;
3. Explaining the complexity of the diagnostic process, as well as how factors like implicit bias and uncertainty can contribute to the risk of diagnostic error; and
4. Illustrating how to translate personal lived experiences into diagnostic research topics and questions for future research opportunities.

 

Curriculum Development Process

The PAIRED curriculum was co-developed using Kolb’s theory of experiential learning theory and the ADDIE model principles for co-production in instructional design (analysis, design, development, implementation, and evaluation) [6,7,8]. Both of these approaches are proven and effective educational approaches to adult learning. The curriculum content was informed by findings from a systematic literature review and consultation with patients, families, researchers, and health professionals.

 

Deploying, Evaluating, and Redirecting

The PAIRED project team designed and deployed a 14-session curriculum to Patient Partners via 12 virtual learning webinars and 2 in-person meetings over a 12-month period. Evaluation methods including phenomenology, grounded theory, direct observation, and structured and unstructured interviews were used to understand the impact of the curriculum. After each session, the PAIRED project team queried participants about what was going well, what they were struggling with, what successes they experienced since the last session, and their plans for the next month. The evaluation uncovered a number of unanticipated challenges that surprised the PAIRED project team.

The project team discovered that patients had difficulty understanding their role as partners in research design, specifically in the planning phase and in identifying research questions. Patients also had difficulty concisely communicating their diagnostic experience and pinpointing pertinent areas for consideration as diagnostic research ideas. The Research Mentors had difficulty integrating the patients’ lived experiences into the development of research ideas, given that they had been trained within a traditional academic culture and in conventional research methods, which did not include patients and family members as team members or effective methods to translate patients’ lived experiences into research themes and topics.

To address these challenges, the PAIRED project team implemented a novel interactive method called diagnostic journey mapping that was previously developed by SIDM and used at the National Academy of Engineering Regional Meeting and Symposium held at the University of Wisconsin-Madison College of Engineering. Utilizing a visual of the Diagnostic Process Framework from the 2015 National Academies report on diagnosis in health care (see Figure 1), a member of the PAIRED project team “mapped” the diagnostic journey of the failure to communicate a malignant pathology that resulted in the death of the patient. The mapping considered each step in the diagnostic process to identify decision points in the diagnostic process and to highlight the points at which communication breakdowns occurred. A second innovative approach was to then prompt participants to ask “what if” a hypothetical idea had been considered at a specific moment in the process and to contemplate options for reengineering the patient pathway to eliminate or reduce risk.

 

 

The PAIRED project team introduced this approach to the Patient Partners and Research Mentors, and then they were broken into small groups consisting of Patient Partners, Research Mentors, and a project team facilitator. The facilitator guided the group through a structured reflection process by walking through the framework step by step and prompting questions while the Patient Partners told their stories of diagnostic error. This helped both the Patient Partners and Research Mentors identify the breakdowns in the diagnostic process and to formulate the “what if” questions for each of these steps. Participants from each small group then collectively “mapped” the diagnostic journeys, breakdowns, and “what ifs” on an enlarged replica of the diagnostic framework from the 2015 National Academies report, resulting in a visual representation of “groupings” of diagnostic breakdowns that corresponded to the various steps in the diagnostic process framework.

Through this exercise, the Patient Partners and Research Mentors discussed and mapped multifaceted diagnostic experiences without losing detail. They were better able to “see” the personal stories of diagnostic error and the breakdowns and to visualize contributing factors and mitigating actions that could have been taken, as well as potential future actions to reduce the risk of future patient safety events related to diagnosis. These discussions gave both the researchers and patients a greater appreciation for each other’s contributions, and potential patient-centered research topics and questions emerged that could inform future diagnostic research projects. See Figure 2 for an example of the mapped patient stories.

 

 

The PAIRED participants were asked to focus in on the one “what if” question that could have been the most pivotal or impactful in preventing the diagnostic errors they experienced. Figure 3 provides examples from the PAIRED project group discussions. They fall into three areas on the Diagnostic Process Map: Patient Engagement; Information Collection and Interpretation; and Communication, Treatment, and Outcomes.

 

 

Patient Engagement

• A 40-year old African American woman experienced migraines, vision loss, and high blood pressure and then suffered a stroke after several visits to her health care providers. She asked, “What if in the patient engagement step of the diagnostic process I had been better informed of the risk factors for and symptoms of stroke in African American women?”

 

Information Collection and Interpretation

• A 36-year-old man suffered from fatigue, low back pain, weight loss, and digestive abnormalities. He visited his primary care physician twice and went to Urgent Care once. He was told that it was probably either his gall bladder or work-related stress. He finally went to his mother for advice. After she brought him to the doctor, he was diagnosed with stage 4 colon cancer. He died 1 year later. The man’s mother asked, “What if my son had been listened to without me having to be there? What if the proper tests were done initially? What if my son’s young age had not been a factor (in failure to consider this diagnosis)?”
• A young man who was hospitalized under observational care over the weekend after having suffered from respiratory failure at home was deteriorating rapidly. He and his family were told that tests would be ordered for the following weekday because it was a weekend. He died over the weekend before the tests were completed and read. His sister and advocate asked in the information-gathering step, “What if the doctor would have acted with more urgency to test on a Sunday?” and “What if there had been a rapid response system that I could have initiated to escalate the urgency that my brother’s condition required?”

 

Communication, Treatment, and Outcomes

• A middle-aged technology executive had a tumor removed from his cervical spine and was told that the tumor was benign. Shortly after the patient was discharged, a final pathology revealed that the tumor was actually a synovial cell sarcoma. The final report was never reviewed by the surgeon or reported to the patient, who subsequently died from metastasis to his spinal cord, likely from the delay in diagnosis and treatment. His wife asked in the communication step of the diagnostic process, “What if the diagnostic loop had been closed and the final results were communicated to all members of the health care team, including the referring doctor and the patient?” and “What if I had access to the pathology in the patient portal to be an extra set of eyes to ensure accuracy?“
• A newborn born in a military hospital had a positive newborn screening test for galactosemia, a rare metabolic disorder. The newborn and the parents had already been discharged. The diagnosis was delivered over the phone to the parents. The physician failed to relay the risks and severity of the metabolic condition, leaving the new parents confused about the disorder and the impact of this diagnosis. The miscommunicated diagnosis ultimately delayed treatment and at 2 months old, the baby suffered liver failure, which required a liver transplant. The father, pointing to the communication step of the diagnostic process, asked, “What if the doctor confirmed my understanding of the diagnosis, next steps to take, and how urgent the situation was?”

 

Impact

The impact of integrating the patient diagnostic journey mapping exercise into the overall conversation around diagnostic error and future research was to give all of the participants a “common language” and a foundational structure on which to base the remaining curriculum. This was a pivotal and clarifying moment for the Patient Partners, Research Mentors, and the project team. The PAIRED project team learned that to optimize participant learning for both the Patient Partners and the Research Mentors, the curriculum needed to be experiential in nature rather than overly academic, conceptual, and theoretical. Specifically, the curriculum needed to adapt and be tailored to the varying states of Patient Partners’ knowledge and help Patient Partners highlight and communicate key aspects of their stories that were especially important to focus on to decrease the potential of serious harm.

 

Research Mentors

This project helped advance diagnostic researchers to understand the importance of engaging patients in research, as illustrated by their comments below:

• “As a diagnostic safety researcher, the PAIRED project provided important visibility into patient-participant issues that I had not fully considered…. Through participation in this effort, I am much more aware that there are research considerations from the patient perspective that are best identified through close partnership with patients throughout the entire research project planning and execution.” Rob El-Kareh, MD
• “This has given me a completely new perspective and has sensitized me and indeed underscored the real reason for all this work—the patient is at the center of it all….” Prashant Mahajan, MD
• “The project has opened my eyes and had a profound impact on me as a diagnostic safety researcher in a number of ways. One of the most important ways is gaining a renewed appreciation of the differing vantage points and lenses that patients view diagnostic errors and delays as they unfold, versus what we see and experience as clinicians and researchers. I was able to see and appreciate this more clearly during the several days working together with the patients in Chicago (at the workshop) and later in New Orleans (at the DEM [Diagnostic Error in Medicine] annual conference) … processing their stories and sorting out their narrative to identify ways diagnosis could be improved in the future.” Gordon Schiff, MD
• “Staying close to the lived experiences of patients and families has always been important to me. Yet, this project heightened my commitment. The patient and family members were clearly eager to learn new skills for partnering with researchers. I learned alongside them about how they could use their diagnostic journeys and often heart wrenching experiences to motivate new insights about what goes wrong. And how to think even more deeply about effective solutions, so that actions can be taken.” Kathy McDonald, PhD

 

Patients and Family Members

The PAIRED project had significant impact on the Patient Partners both personally and professionally and offered a path for them to leverage their lived experience of diagnostic error in future research efforts and beyond. Patient Partners reported feeling a profound sense of community, as well as an increased sense of confidence, purpose, and validation. They shared that they now understood how their lived experiences can catalyze new research and improved diagnosis. This experience also spurred several patients to pursue new professional directions and increased advocacy. Examples of Patient Partner reactions include:

• One patient shared, “I had never worked on a project like this, you know, because my experience is fairly still fresh and it’s been great to form friendships and connections in so many different places and hopefully continue to get to work on some different projects.” Another simply stated “I loved the way the whole story framework was laid out.”
• At SIDM’s annual DEM conference, the PAIRED participants gave a plenary session of “what ifs” to prompt researchers and others to integrate the “what ifs” into their diagnostic improvement efforts. They received a standing ovation following their presentation. One Patient Partner remarked “I felt powerful. I finally feel influential.”
• The Patient Partners are eager to continue collaborating with other PAIRED members, SIDM, and other organizations to engage in research and other diagnostic improvement efforts. Some who had lost family members or felt their misdiagnosis “didn’t matter” reported that participating in the PAIRED project had a significant “healing effect.”
• One Patient Partner reported that he is applying for a master’s degree in clinical quality, safety, and leadership at Georgetown University. Another Patient Partner reported that she became a proposal reviewer for the U.S. Department of Defense and other groups.

 

Outcomes

The PAIRED project team successfully created a cohesive, committed, and skilled community of highly diverse patients and family members with lived experiences of diagnostic error, and diagnostic safety researchers who actively collaborated for more than 1 year to co-produce a curriculum that promoted patient engagement in diagnostic research. This curriculum “met patients and family members where they were,” incorporated innovative patient-centered strategies that “personalized” the curriculum, and maximized engagement and learning. Patient Partners and Research Mentors pragmatically exchanged knowledge and collaborated in the development of relevant, patient-centered diagnostic research questions not previously recognized or considered. This PAIRED curriculum provided the necessary capacity building and skills for the Patient Partners and Research Mentors.

Since the completion of the PAIRED project, Patient Partners have served as subject-matter experts in funded diagnostic research. For example, a research question generated during the diagnostic journey mapping by Patient Partners and a Research Mentor about disparities in diagnosis related to visible factors of age, sex, and race/ethnicity resulted in a 2-year funded research project led by SIDM and Stanford University titled Exploring and Addressing Diagnostic Error Disparities Related to Cognitive Reasoning Pitfalls. Patient Partners have also been recruited to participate in a national Diagnostic Research Prioritization initiative as well as an Agency for Healthcare Research and Quality–funded project titled Answering the Call to Engage Patients and Families in the Diagnostic Process: A New Patient-Centered Approach Using Health Information Transparency to Identify Diagnostic Breakdowns in Ambulatory Care. Additionally, as a result of the PAIRED project, SIDM was the recipient of a second PCORI award to develop a curriculum and educate a community of diagnostic researchers in PCOR, CER, and how to effectively engage the patient and family community as research partners.

The diagnostic journey mapping approach is replicable and has provided a resource to others in the development of training materials to engage patients and other stakeholders as partners in diagnostic research and has been applied to other diagnostic improvement efforts more broadly. SIDM leadership, Patient Partners, and research members have been implementing the mapping of case studies on the 2015 National Academies report’s diagnostic framework coupled with the “what if” question prompt both nationally and internationally to educate others in medical education, policymaking, measure development, and quality improvement efforts on the importance of understanding diagnostic error, the breakdowns, and possible risk mitigation strategies. For example, SIDM collaborated with Johns Hopkins University to incorporate relevant PAIRED modules, including the diagnostic process mapping and the “what if” approach, into their curriculum for diagnostic research in urgent care settings. In addition, the diagnostic journey mapping and “what if” exercise was used at the PFAC Leadership Convening co-hosted by SIDM and the National Academy of Medicine to help mobilize PFACs as drivers of change within the health care system to improve diagnosis. The participants included national PFAC leaders, health care system leadership, government agencies, and health and research organizations to identify risk points in the diagnostic process during the delivery of care in hospitals and to identify interventions, tools, and strategies to help mitigate that risk.

 

Call to Action

During this collective pursuit of identifying and eradicating the causes of diagnostic errors, as well as ways to reduce the risk of and improve diagnostic safety and quality, it is incumbent upon the entire health care community to consider a few key learnings:

• Diagnostic errors are complex, “messy” events that often defy easy categorization and simple solutions.
• Patients and family members who have first-hand experiences with diagnostic error are eager to engage in diagnostic safety research and offer unique research questions and topic suggestions. Their stories raise multiple fertile issues and research questions that often go unrecognized or unconsidered. The health care community and health care researchers should create opportunities to enable them to share their lived experiences by partnering with patient advocacy organizations, researchers, and health care systems.
• Researchers should recognize the value of partnering with and welcome the opportunity to partner with patients and their families to develop, implement, and disseminate research on health care safety and quality.
• National disease groups and PFACs are valuable resources for future training and partnering in diagnostic research and should seek partnerships with researchers and academic institutions.
• Policy makers should require that patients and family members are engaged as partners in research funding proposals.

 

Co-producing research projects with patients yields new questions, new approaches, and findings that are more meaningful to patients and their families, diagnostic researchers, and clinicians. The diagnostic mapping journey and asking “what if” process described in this paper will help illuminate the way forward.

 

---

Join the conversation!

  Tweet this!  Patients and family members with lived experience of diagnostic error have unique perspectives that can be utilized to inform needed and impactful research on how to improve the diagnostic process. Read more in a new #NAMPerspectives Commentary: https://doi.org/10.31478/202002a

  Tweet this! Misdiagnosis affects 12 million Americans annually, and their experiences can and should be leveraged when considering how to improve the diagnostic process. Read about a method for including their voices: https://doi.org/10.31478/202002a #NAMPerspectives

  Tweet this! While developing a curriculum to teach patients how to participate in the design of research studies, @ImproveDX stumbled on a process that helped patients function as partners. Read about diagnostic process mapping in a new #NAMPerspectives: https://doi.org/10.31478/202002a

 

Download the graphics below and share them on social media!

 

 

References

1. Gabel, R. A. 2002. Counting deaths due to medical errors. JAMA 288(19):2404–2405. https://doi.org/10.1001/jama.288.19.2404-JLT1120-2-1
2. National Academies of Sciences, Engineering, and Medicine. 2015. Improving diagnosis in health care. Washington, DC: The National Academies Press. https://doi.org/10.17226/21794
3. Society to Improve Diagnosis in Medicine. 2019. Patients improving research in diagnosis. Available at: https://www.improvediagnosis.org/patientsimproving-research-in-diagnosis (accessed February 5, 2020).
4. Society to Improve Diagnosis in Medicine. 2019. Patient stories. Available at: https://www.improvediagnosis.org/stories (accessed February 5, 2020).
5. Society to Improve Diagnosis in Medicine. 2019. Patient partners. Available at: https://improvediagnosis.org/patient-partners (accessed February 5, 2020).
6. McLeod, S. A. 2017. Kolb’s learning styles and experiential learning cycle. Simply Psychology. Available at: https://www.simplypsychology.org/learning-kolb.html (accessed February 5, 2020).
7. McGriff, S. J. 2000. Instructional System Design (ISD): Using the ADDIE model. Available at: https://www.lib.purdue.edu/sites/default/files/directory/butler38/ADDIE.pdf (accessed February 5, 2020).
8. InstructionalDesign.org. 2019. ADDIE model. Available at: https://www.instructionaldesign.org/models/addie (accessed February 5, 2020).

Tweets by @theNAMedicine