A Pragmatic Approach for Organizations to Measure Health Care Professional Well-Being

 

Introduction

There is a high prevalence of burnout, depression, and suicide among health care professionals (HCPs) [1-5]. Compromised well-being among HCPs is associated with medical errors, medical malpractice suits, health care associated infections, patient mortality, lower interpersonal teamwork, lower patient satisfaction, job dissatisfaction, reduction in professional effort, and turnover of staff [2]. In addition, burnout among physicians is an independent predictor of suicidal ideation and substance abuse and dependence [6-9]. As burnout is adversely affecting quality, safety, and health care system performance, as well as the personal lives of HCPs, there is a need for organizations to add measures of HCP well-being to their routine institutional performance measures (e.g., patient volume, quality metrics, patient satisfaction, financial performance) [10, 11]. Institutional performance measures, including measurements of HCP well-being, hold the potential to substantially improve health care systems. However, putting measures in place without sufficient thought and care (e.g., insufficiently valid data) may result in the misdirection of resources, a false sense of the scope of the problem, and delay in improvement. The successful evaluation of HCP well-being depends on a series of strategic decisions, including who to survey (e.g., all employees or only a subset), how to survey (electronic or paper survey, local administration or external vendor), when to survey (timing and frequency), and what to include on the survey (i.e., items). The following paper will focus on which survey tools are available for institutions to measure burnout and composite well-being.

While several instruments measuring HCP well-being have well-established reliability and validity, they are not all equally pragmatic for use by organizations interested in local assessment and quality improvement efforts. In this paper, we provide a list of considerations for individuals charged with measuring HCP well-being at their institutions to guide them in selecting the most appropriate measurement instrument. Such a decision should be based on which dimensions of well-being are most important to stakeholders in addition to instrument characteristics (i.e., respondent and organizational burden, how actionable the provided data is, sensitivity to change, psychometric support, and applicability; see Table 1) [12, 13].

 

 

Instrument Characteristics to Consider

Dimensions of Well-Being Important to Stakeholders

Measuring HCP well-being involves determining which dimension(s) of well-being to assess (e.g., burnout, fatigue, engagement, emotional health, quality of life, and professional fulfillment/satisfaction). For each of these dimensions, multiple survey instruments exist. The chosen measure(s) should assess dimensions of well-being deemed important by HCPs, organizational decision makers (department chairs, chief medical officer, chief wellness officer, chief executive officer, or legal department), patient experience advocates, patient safety advocates, and other local stakeholders. Focus groups and surveys can be conducted with stakeholders to identify common dimensions of greatest interest to guide decision making.

Measuring Burnout and Composite Well-Being

Although the selection of which dimension of well-being to monitor locally should be determined locally, in this paper we discuss principles of instrument selection for burnout and composite well-being measures. Burnout is one construct that an organization may choose to measure, given its prevalence and associated serious personal, professional, and organizational consequences.

The most commonly used instrument to measure burnout in HCPs is the Maslach Burnout Inventory-Human Services Survey for Medical Personnel (MBI-HSS [MP]) [14]. The MBI-HSS [MP] has three domains: emotional exhaustion, depersonalization, and low sense of personal accomplishment. Other instruments available to measure burnout include the Oldenburg Burnout Inventory (OBI) [15] and the Copenhagen Burnout Inventory (CBI) [16]. The OBI has three domains: physical, cognitive, and affective exhaustion and disengagement from work. The CBI has three domains: personal (physical and psychological fatigue and exhaustion), work (physical and psychological fatigue and exhaustion related to work), and client-related (or a similar term such as patient, student, etc.) burnout [16]. Some health systems and investigators use the Physician Worklife Study (PWLS) single-item (“Overall, based on your definition of burnout, how would you rate your level of burnout?”) to measure burnout symptoms [17].

Focusing on the single construct of burnout can simplify the process of choosing, administering, and analyzing measurement results. However, as distress can manifest differently in individuals who have similar experiences [18], single construct measures may miss individuals who are not well, but may not be feeling burned out. As such, organizations may prefer to use brief composite measures that assess multiple dimensions of distress commonly experienced by HCPs, such as the Well-Being Index (WBI) [19-23] or the newly developed Stanford Professional Fulfillment Index (PFI) [24] that measure symptoms of burnout (work exhaustion and interpersonal disengagement) as well as professional fulfillment.

 

Low Respondent Burden

Given the busy work lives of HCPs, it is best to use brief surveys to limit responder burden. Measures used in research studies are often lengthy and can lead to substantial responder fatigue and poor response rates, especially if administered at regular intervals. With respect to burnout measures, the MBI-HSS [MP], OBI, and CBI are 22, 16, and 19 items long, respectively (see Table 2) [25-27]. To reduce burden, West and colleagues identified two single questions with the highest loading factor on the emotional exhaustion and depersonalization subscales of the MBI-HSS [MP], resulting in a two-item MBI [MP] [28, 29]. The two single MBI items have an area under the receiver operating characteristic curve (AUC) of 0.94 and 0.93 for the emotional exhaustion and depersonalization domains, respectively, relative to the full MBI-HSS [MP] [28, 29]. Thus, the two-item MBI [MP] can reasonably be used as a substitute for the full MBI-HSS [MP] to reduce responder burden. The PWLS single-item is the shortest measurement instrument [17]. For composite measures, the WBI and PFI are 9 and 16 items, respectively [19-24].

 

Low Organizational Burden

Organizational burden is dependent on the cost of the instrument and the cost and complexity of analysis. Measures that are difficult to analyze require sophisticated statistical resources and expertise that may not be available to individuals charged with assessing HCP well-being for an organization. The MBI-HSS [MP], OBI, CBI, and PFI are moderately complex to analyze. The PWLS, two single-item MBI-HSS [MP], and WBI are relatively simple to analyze. While the MBI-HSS [MP] is proprietary, the other measures are free for research purposes and for use in quality improvement efforts by nonprofit organizations. Currently, the license to reproduce the MBI-HSS [MP] for a paper and pencil survey or an online survey is $125 per 50 administrations ($2.50 each with a minimum purchase of 50). The MBI manual is an additional $50. There are discounts offered for large volume purchases. Volume discount rates vary bt the number of administrations purchased. For an organization with 450 physicians, the cost is $837.50, including the purchase of the manual. This is a trivial amount in the face of an estimated cost of $5,625,000 due to turnover from burnout at a similarly sized organization (assumes an annual turnover rate of 7.5 percent and a replacement cost of $500,000 per physician) [30].

 

Actionable Measures

Measures most likely to lead to action provide scores that are meaningful to stakeholders. Typically, scores have meaning if they can be compared to historical institutional data, have relevant national benchmark data (e.g., scores in national samples of physicians and other HCPs), and correlate with important outcomes, such as medical error, malpractice litigation suits, patient satisfaction, or HCP turnover [30]. National benchmark data provide context in interpreting survey results. For example, if the prevalence of burnout among an organization’s emergency physicians is higher than national norms, then there is likely to be locally active issues within the sphere of influence of an organization that could benefit from local action planning. To date only the MBI-HSS [MP] and WBI have national benchmark data for physicians, nurses, and advanced practice providers [1, 20-23].

Correlation with Other Important Outcomes

Actionable measures of well-being should also be linked with important outcomes that may have a secondary response to interventions that directly impact HCP well-being. The MBI-HSS [MP] is supported by the most extensive association with important outcomes. In large studies of U.S. physicians and trainees, MBI-HSS [MP] scores have been associated with medical error [31-33], malpractice [34], suboptimal patient care practices [35], physician turnover and early retirement [36, 37], lower medical knowledge [38], suboptimal professionalism [39, 40], alcohol abuse [8, 9, 41], suicidal ideation [6, 7], and motor vehicle incidents [56]. The two single-item measures from the MBI-HSS [MP] have been demonstrated in large samples of more than 10,000 physicians and trainees to also correlate with key outcomes such as major medical errors, suicidal ideation, serious thoughts of dropping out of medical school, and poor professionalism [28, 29].

Less is known about the meaning of OBI and CBI scores, as most studies have been small and conducted outside the United States. Existing data suggest correlations between OBI scores of Swedish nurses and public health professionals with the intent to change jobs [42-44], and between OBI scores of Swedish medical students and residents with self-rated health [45, 46]. High CBI scores have been associated with perceptions of lower quality of care in a study of German surgeons [47]; lower World Health Organization-Five Well-Being Index scores in a study of 317 Canadian residents [48]; turnover intention in a small study of Iranian intensive care unit (ICU) nurses [49]; job strain, overcommitment, and low social support in a study of Taiwanese HCPs [50]; and recent reorganization at work in a study of Norwegian midwives [27]. In non-HCPs, studies have found correlations between high CBI scores and absenteeism, sleep problems, use of pain killers, intention to quit work, and antidepressant treatment [16, 51, 52].

Studies of the PWLS single item have mostly been small and included almost exclusively primary care physicians [17, 53-56]. In these studies, the PWLS single-item burnout scores were associated with lower job satisfaction, greater intent to leave the medical practice (on univariate analysis) [55], self-reported medical error, and suboptimal patient care practices [56]. WBI scores have been shown to stratify career satisfaction, intent to leave current job, and self-reported medical error among physicians, residents, medical students, nurses, and other health care workers [19-23]. Data on the meaning of PFI scores are more limited, with one study of 250 resident and practicing physicians showing a relationship between PFI scores and self-reported medical errors [24].

 

Sensitivity to Change

Ideally, measures of HCP well-being should detect meaningful effect sizes from interventions. If initiatives intended to improve HCP well-being are relatively short in duration, measures that assess a short time frame would be best suited. However, as a multi-faceted approach is likely needed to implement systematic change to improve HCP well-being, a longer time frame (such as 1 year or more) may be more appropriate.

The MBI-HSS [MP] response categories include “never,” “a few times a year,” and “every day” [25]. Although the MBI-HSS [MP] may not be sensitive to change over a short time period, data from a systematic review and meta-analysis support that the MBI-HSS [MP]  can detect moderate effect size from a range of interventions intended to improve physician burnout [57, 58]. In contrast, the OBI, CBI, and PWLS single-item do not refer to a time frame and less is known about their ability to detect meaningful effect size from an intervention. The PFI asks about symptoms over the past 2 weeks while the WBI asks about symptoms over the past month. Preliminary data suggest that the PFI may be sensitive to change [24].

 

Psychometrically Strong

HCP well-being measures should have an acceptable degree of construct validity, but may not meet all of the requirements expected of measures used in intervention studies [12, 13]. Validity is to what extent results obtained from a measure should be trusted [59]. The degree of validity depends on the evidence collected in support of how well a measure’s scores reflect the intended construct (e.g., overarching dimension of well-being that the instrument intends to measure). Sources of evidence within the construct validity rubric include content evidence (e.g., steps taken to ensure that items represent the construct), response process evidence (e.g., thoughts respondents have while answering the questions, methods for scoring, and reporting), internal structure evidence (e.g., reliability and factor analysis), relations to other variables (e.g., correlation with scores on other measures or outcomes of interest), and consequence validity (e.g., improvement in HCP well-being) [59]. Single-item measures do not have internal structure evidence. Reviews detailing current concepts in validity for measures have been published [59].

Among the common burnout measures, the 22-item MBI-HSS [MP] has the strongest construct validity data for use for U.S. physicians and other HCPs (see Table 2). Similarly, the two items from the MBI-HSS [MP]  have strong validity evidence of relationship to other variables in large samples of U.S. physicians [28, 29]. The WBI has strong validity content evidence and evidence of relationships to other variables in U.S. physicians, residents, medical students, nurses, and other workers [19-23].

Less robust construct validity is available for the remaining measures, especially for use for U.S. physicians and other HCPs. The OBI has mixed evidence of internal structure as some studies suggest a two-factor structure [15, 61, 62] while another study confirmed a four-factor structure [63]. As mentioned previously, the OBI has limited evidence of relationship to other variables in studies of Swedish nurses, public health professionals, medical students, and residents [42-46]. In samples of non-HCPs, OBI scores have been shown to correlate modestly with the scores from the MBI-General Survey in a sample of U.S. workers [62], Chinese nurses [63], and other non-U.S. workers [26, 61].

The CBI has evidence of internal structure and limited evidence of relationship to other variables in studies of non-U.S. physicians and other HCPs (i.e., German surgeons [47], Canadian residents [48], Iran ICU nurses [49], Taiwanese HCPs [50], and Norwegian midwives [27]) and non-HCPs [16, 51]. In a study of 1,914 individuals from 7 different workplaces, CBI scores correlated with quality-of-life scores as measured by the 36-item Short Form Health Survey scale [64].

For the PWLS, stem and response items vary in publications, making it difficult to synthesize findings. Existing publications provide some evidence of relationship to other variables. For example, the PWLS single-item predicts a high level of emotional exhaustion. However, scores from the PWLS do not correlate as strongly with the full MBI-HSS [MP] emotional exhaustion scale in comparison to the original single-item MBI-Emotional Exhaustion item [17, 53, 54].

The PFI has some evidence of internal structure validity (i.e., three-factor structure, test-retest reliability) and relationship to other variables (PFI scores correlate moderately with MBI scores) in a sample of 250 U.S. physicians [24]. Regardless of the instrument chosen, validated and established approaches to scoring should be employed in order to meaningfully interpret the data collected. Best practices for how to analyze data from the aforementioned instruments as well as a summary of the data demonstrating relationships between instrument scores and important health care and personal outcomes (i.e., relations to other variables) have been published [60].

 

Broadly Applicable

Broadly applicable measures are those that are appropriate for a wide range of individuals, such as when a health care organization is trying to measure the well-being of all of its employees. Depending on the intended respondents, measures may need to be selected based on required reading level, applicability of questions to individuals in different job roles, cultural sensitivity, and other factors.

The MBI-HSS is the original and most widely used version of the MBI [25]. The only difference between the MBI-HSS and the MBI-HSS [MP] is the word “patients” used in place of “recipients.” The MBI-HSS [MP] is most applicable to HCPs who have direct patient care roles. Other versions of the MBI are available for other occupational groups, but less is known about how scores on these versions relate to health care outcomes.

The PFI refers to patients and to work in general. The OBI and CBI were developed for use in any occupational group. The OBI and PWLS single-item refer to “my work” whereas the CBI refers to “work” and “clients.” The WBI refers to “my work” or “medical school” (depending on the version) and has been validated in multiple independent samples that have included more than 25,800 health care professionals and other U.S. workers, suggesting that it is broadly applicable [19-22]. It is worth pointing out that not all instruments have data on validity in more than one occupational group.

 

Additional Considerations

Utilizing an external organization to administer and analyze the results may help provide extra confidence of confidentiality and boost willingness to honestly answer questions. Limiting demographic items that could allow for identification of individuals (e.g., the only Hispanic woman physician in a department) is another strategy to help ensure confidentiality. Measures that assess depressive symptoms or substance use or inquire about more sensitive topics may be less well-received by HCPs. Administering anonymous or confidential surveys, however, eliminates the ability to directly identify and help those in distress. Providing information about local employee assistance programs and other avenues for how to seek help if experiencing distress at the end of a survey can help mitigate these concerns. Measures and accompanying items should address only topics the organization is willing and able to address. Asking about problem areas and then not addressing them may cause staff to feel ill will toward organizational leadership.

 

 

Conclusion

The available tools to measure HCP burnout and overall well-being have a variety of strengths and limitations. Most health care systems will be able to find a validated instrument or instruments that meet their particular needs and situation. Table 3 summarizes seven common instruments in terms of the pragmatic dimensions discussed above and in terms of their overall strengths and limitations. The proprietary two-item MBI [MP] [28, 29] for burnout and the nine-item WBI [22] for overall well-being are pragmatic, broadly applicable tools with a good balance of strengths and limitations that health care organizations without other constraints may wish to use [12, 13]. Prior to administering a survey to assess HCP well-being, organizations should be ready to act on the results with planned human and financial resources allocated for such a response. Fortunately, meta-analysis and systemic reviews of intervention studies suggest that interventions can lead to improvements in HCP well-being [57, 58]. An annotated bibliography published by the Collaborative for Healing and Renewal in Medicine on evidence-based, well-being interventions can be found on the Alliance for Academic Internal Medicine website (http://www.im.org/page/annotatedbibliography). Knowing that improvements can be achieved can help make the case for organizations undertaking monitoring and improvement efforts for the first time. Sharing the results with responding HCPs, involving them in action planning and implementation, and discussing the impact of the response are important to achieving ongoing buy-in and participation by the targeted group and ultimately crucial in successfully addressing HCP well-being.

 

 

---

 

Join the conversation!

Tweet this!  New from @theNAMedicine: A Pragmatic Approach for Organizations to Measure Health Care Professional Well-Being: https://doi.org/10.31478/201810b #ClinicianWellBeing #NAMPerspectives

Tweet this!  Measuring #ClinicianWellBeing at your organization is critical to long-term, positive outcomes. New #NAMPerspectives discussion paper provides an overview of useful measurement tools: https://doi.org/10.31478/201810b

Tweet this!  To reverse clinician burnout, you first must be able to measure it. New #NAMPerspectives provides insight into tools that can help: https://doi.org/10.31478/201810b #ClinicianWellBeing

 

Download the graphics below and share them on social media!

 

References

1. Shanafelt, T. D., O. Hasan, L. N. Dyrbye, C. Sinsky, D. Satele, J. Sloan, and C. P. West. 2015. Changes in Burnout and Satisfaction with Work-Life Balance in Physicians and the General US Working Population Between 2011 and 2014. [Erratum appears in Mayo Clinic Proceedings 2016 Feb; 91(2):276]. Mayo Clinic Proceedings 90(12). https://doi.org/10.1016/j.mayocp.2015.08.023
2. Dyrbye, L. N., T. D. Shanafelt, C. A. Sinsky, P. F. Cipriano, J. Bhatt, A. Ommaya, C. P. West, and D. Meyers. 2017. Burnout among health care professionals: A call to explore and address this underrecognized threat to safe, high-quality care. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/201707b
3. Dyrbye, L. N., and T. D. Shanafelt. 2011. Physician Burnout. A Potential Threat to Successful Health Care Reform. JAMA 305(19). https://doi.org/10.1001/jama.2011.652
4. Schernhammer, E. 2005. Taking Their Own Lives—The High Rate of Physician Suicide. New England Journal of Medicine 352(24). https://doi.org/10.1056/NEJMp058014
5. Mata, D. A., M. A. Ramos, N. Bansal, R. Khan, C. Guille, E. DiAngelantonio, and S. Sen. 2015. Prevalence of Depression and Depressive Symptoms Among Resident Physicians: A Systematic Review and Meta-analysis. JAMA 314(22). https://doi.org/10.1001/jama.2015.15845
6. Dyrbye, L. N., M. R. Thomas, F. S. Massie, D. V. Power, A. Eacker, W. Harper, S. Durning, C. Moutier, D. W. Szydlo, P. J. Novotny, J. A. Sloan, and T. D. Shanafelt. 2008. Burnout and Suicidal Ideation Among US Medical Students. Annals of Internal Medicine 149. https://doi.org/10.7326/0003-4819-149-5-200809020-00008
7. Shanafelt, T. D., C. M. Balch, L. Dyrbye, G. Bechamps, T. Russell, D. Satele, T. Rummans, K. Swartz, P. J. Novotny, J. Sloan, and M. R. Oreskovich. 2011. Suicidal Ideation Among American Surgeons. The Archives of Surgery 146(1). https://doi.org/10.1001/archsurg.2010.292
8. Oreskovich, M. R., K. L. Kaups, C. M. Balch, J. B. Hanks, D. Satele, J. Sloan, C. Meredith, A. Buhl, L. N. Dyrbye, and T. D. Shanafelt. 2012. The Prevalence of Alcohol Use Disorders Among American Surgeons. The Archives of Surgery 147(2). https://doi.org/10.1001/archsurg.2011.1481
9. Jackson, E. R., T. D. Shanafelt, O. Hasan, D. V. Satele, L. N. Dyrbye. 2016. Burnout and Alcohol Abuse/Dependence Among U.S. Medical Students. Academic Medicine 91(9). https://doi.org/10.1097/ACM.0000000000001138
10. Shanafelt, T. D., and J. H. Noseworthy. 2016. Executive Leadership and Physician Well-Being: Nine Organizational Strategies to Promote Engagement and Reduce Burnout. Mayo Clinic Proceedings 92(1). https://doi.org/10.1016/j.mayocp.2016.10.004
11. Thomas, L. R., J. A. Ripp, and C. P. West. 2018. Charter on Physician Well-Being. JAMA 319(15). https://doi.org/10.1001/jama.2018.1331
12. Glasgow, R. E. 2013. What Does It Mean to Be Pragmatic? Pragmatic Methods, Measures, and Models to Facilitate Research Translation. Health Education & Behavior 40(3). https://doi.org/10.1177/1090198113486805
13. Glasgow, R. E., and W. T. Riley. 2013. Pragmatic Measures: What They Are and Why We Need Them. American Journal of Preventive Medicine 45(2). https://doi.org/10.1016/j.amepre.2013.03.010
14. Maslach, C., S. E. Jackson, and M. P. Leiter. 1996. Maslach Burnout Inventory Manual. 3rd ed. Palo Alto, CA: Consulting Psychologists Press.
15. Demerouti, E., and A. B. Bakker. 2007. The Oldenburg Burnout Inventory: A Good Alternative to Measure Burnout and Engagement. Available at: https://www.researchgate.net/publication/46704152_The_Oldenburg_Burnout_Inventory_A_good_alternative_to_measure_burnout_and_engagement (accessed September 2, 2020). 
16. Kristense, T. S., M. Borritz, E. Villadsen, and K. B. Christensen.  2005.The Copenhagen Burnout Inventory: A New Tool for the Assessment of Burnout. Work Stress 19(3). https://doi.org/10.1080/02678370500297720
17. Dolan, E. D., D. Mohr, M. Lempa, S. Joos, S. D. Fihn, K. M. Nelson, and C. D. Helfrich. 2014. Using a Single Item to Measure Burnout in Primary Care Staff: A Psychometric Evaluation. Journal of General Internal Medicine 30(5). https://doi.org/10.1007/s11606-014-3112-6
18. Dyrbye, L. N., W. Harper, S. J. Durning, C. Moutier, M. R. Thomas, F. S. Massie, A. Eacker, D. V. Power, D. W. Szydlo, J. A. Sloan, and T. D. Shanafelt. 2011. Patterns of Distress in US Medical Students. Medical Teacher 33(10). https://doi.org/10.3109/0142159X.2010.531158
19. Dyrbye, L. N., A. Schwartz, S. M. Downing, D. W. Szydlo, J. A. Sloan, and T. D. Shanafelt. 2011. Efficacy of a Brief Screening Tool to Identify Medical Students in Distress. Academic Medicine 86(7). https://doi.org/0.1097/ACM.0b013e31821da615
20. Dyrbye, L. N., D. Satele, J. Sloan, and T. D. Shanafelt. 2014. Ability of the Physician Well-Being Index to Identify Residents in Distress. Journal of Graduate Medical Education 6(1). https://doi.org/10.4300/JGME-D-13-00117.1
21. Dyrbye, L. N., D. Satele, J. Sloan, and T. D. Shanafelt. 2013. Utility of a Brief Screening Tool to Identify Physicians in Distress. Journal of General Internal Medicine 28(3). https://doi.org/10.1007/s11606-012-2252-9
22. Dyrbye, L. N., D. Satele, and T. Shanafelt. 2016. Ability of a 9-Item Well-Being Index to Identify Distress and Stratify Quality of Life in US Workers. Journal of Occupational and Environmental Medicine 58(8). https://doi.org/10.1097/JOM.0000000000000798
23. Dyrbye, L. N. P. O. Johnson, L. M. Johnson, D. V. Satele, and T. D. Shanafelt. 2018. Efficacy of the Well-Being Index to Identify Distress and Well-Being in US Nurses. Nursing Research 67(6): 447-455. https://doi.org/10.1097/NNR.0000000000000313
24. Trockel, M., B. Bohman, E. Lesure, M. S. Hamidi, D. Welle, L. Roberts, and T. D. Shanafelt. 2018. A Brief Instrument to Assess Both Burnout and Professional Fulfillment in Physicians: Reliability and Validity, Including Correlation with Self-Reported Medical Errors, in a Sample of Resident and Practicing Physicians. Academic Psychiatry 42(1). https://doi.org/10.1007/s40596-017-0849-3
25. Maslach, C., S. E. Jackson, and M. P. Leiter. 2016. Maslach Burnout Inventory. 4th ed. Palo Alto, CA: Mind Garden, Inc.
26. Demerouti, E., K. Mostert, and A. B. Bakker. 2010. Burnout and Work Engagement: A Thorough Investigation of the Independency of Both Constructs. Journal of Occupational Health Psychology 15(3). https://doi.org/10.1037/a0019408
27. Henriksen, L., and M. Lukasse. 2016. Burnout Among Norwegian Midwives and the Contribution of Personal and Work-Related Factors: A Cross-Sectional Study. Sexual & Reproductive Healthcare: Official Journal of the Swedish Association of Midwives. https://doi.org/10.1016/j.srhc.2016.08.001
28. West, C. P., L. N. Dyrbye, D. V. Satele, J. A. Sloan, and T. D. Shanafelt. 2012. Concurrent Validity of Single-Item Measures of Emotional Exhaustion and Depersonalization in Burnout Assessment. Journal of General Internal Medicine 27(11). https://doi.org/10.1007/s11606-012-2015-7
29. West, C. P., L. N. Dyrbye, J. A. Sloan, and T. D. Shanafelt. 2009. Single Item Measures of Emotional Exhaustion and Depersonalization Are Useful for Assessing Burnout in Medical Professionals. Journal of General Internal Medicine 24(12). https://doi.org/10.1007/s11606-009-1129-z
30. Shanafelt, T., J. Goh, and C. Sinsky. 2017. The Business Case for Investing in Physician Well-Being. JAMA Internal Medicine 177(12). Available at: https://omahamedical.com/wp-content/uploads/2017/10/The-Business-Case-for-Investing-in-Physician-Well-being.pdf (accessed September 2, 2020). 
31. Shanafelt, T. D., C. M. Balch, G. Bechamps, T. Russell, L. Dyrbye, D. Satele, P. Collicott, P. J. Novotny, J. Sloan, and J. Freischlag. 2010. Burnout and Medical Errors Among American Surgeons. Annals of Surgery 251(6). https://doi.org/10.1097/SLA.0b013e3181bfdab3
32. West, C. P., M. M. Huschka, P. J. Novotny, J. A. Sloan, J. C. Kolars, T. M. Habermann, and T. D. Shanafelt. 2006. Association of Perceived Medical Errors with Resident Distress and Empathy: A Prospective Longitudinal Study. JAMA 296(9). https://doi.org/10.1001/jama.296.9.1071
33. West, C. P., A. D. Tan, T. M. Habermann, J. A. Sloan, and T. D. Shanafelt. 2009. Association of Resident Fatigue and Distress with Perceived Medical Errors. JAMA 302(12). https://doi.org/10.1001/jama.2009.1389
34. Balch, C. M., M. R. Oreskovich, L. N. Dyrbye, J. M. Colaiano, D. V. Satele, J. A. Sloan, and T. D. Shanafelt. 2011. Personal Consequences of Malpractice Lawsuits on American Surgeons. Journal of the American College of Surgeons 213(5). https://doi.org/10.1016/j.jamcollsurg.2011.08.005
35. Shanafelt, T. D., K. A. Bradley, J. E. Wipf, and A. L. Back. 2002. Burnout and Self-Reported Patient Care in an Internal Medicine Residency Program. Annals of Internal Medicine 136(5). https://doi.org/10.7326/0003-4819-136-5-200203050-00008
36. Shanafelt, T. D., M. Mungo, J. Schmitgen, K. A. Storz, D. Reeves, S. N. Hayes, J. A. Sloan, S. J. Swensen, and S. J. Buskirk. 2016. Longitudinal Study Evaluating the Association Between Physician Burnout and Changes in Professional Work Effort. Mayo Clinical Proceedings 91(4). https://doi.org/10.1016/j.mayocp.2016.02.001
37. Shanafelt, T. D., L. N. Dyrbye, C. P. West, and C. A. Sinsky. 2016. Potential Impact of Burnout on the US Physician Workforce. Mayo Clinical Proceedings 91(11). https://doi.org/10.1016/j.mayocp.2016.08.016
38. West, C., T. Shanafelt, and J. Kolars. 2011. Quality of Life, Burnout, Educational Debt, and Medical Knowledge Among Internal Medicine Residents. JAMA 306(9). https://doi.org/10.1001/jama.2011.1247
39. Dyrbye, L. N., F. S. Massie, A. Eacker, W. Harper, D. Power, S. J. Durning, M. R. Thomas, C. Moutier, D. Satele, J. Sloan, and T. D. Shanafelt. 2010. Relationship Between Burnout and Professional Conduct and Attitudes Among US Medical Students. JAMA 304(11). https://doi.org/10.1001/jama.2010.1318
40. Dyrbye, L. N., C. P. West, D. Satele, S. Boone, J. Sloan, and T. D. Shanafelt. 2015. A National Study of Medical Students’ Attitudes Toward Self-Prescribing and Responsibility to Report Impaired Colleagues. Academic Medicine 90(4). https://doi.org/10.1097/ACM.0000000000000604
41. Oreskovich, M. R., T. Shanafelt, L. N. Dyrbye, L. Tan, W. Sotile, D. Satele, C. P. West, J. Sloan, and S. Boone. 2015. The Prevalence of Substance Use Disorders in American Physicians. The American Journal on Addictions. https://doi.org/10.1111/ajad.12173
42. Rudman, A., P. Gustavsson, and D. Hultell. 2014. A Prospective Study of Nurses’ Intentions to Leave the Profession During Their First Five Years of Practice in Sweden. International Journal of Nursing Studies 51(4). https://doi.org/10.1016/j.ijnurstu.2013.09.012
43. Rudman, A., and J. P. Gustavsson. 2012. Burnout During Nursing Education Predicts Lower Occupational Preparedness and Future Clinical Performance: A Longitudinal Study. International Journal of Nursing Studies 49(8). https://doi.org/10.1016/j.ijnurstu.2012.03.010
44. Peterson, U., G. Bergstrom, E. Demerouti, P. Gustavsson, M. Asberg, and A. Nygren. 2011. Burnout Levels and Self-Rated Health Prospectively Predict Future Long-Term Sickness Absence: A Study Among Female Health Professionals. Journal of Occupational & Environmental Medicine 53(7). https://doi.org/10.1097/JOM.0b013e318222b1dc
45. Dahlin, M., N. Joneborg, and B. Runeson. 2007. Performance-Based Self-Esteem and Burnout in a Cross-Sectional Study of Medical Students. Medical Teacher 29. https://doi.org/10.1080/01421590601175309
46. Anagnostopoulos, F., E. Demerouti, P. Sykioti, D. Niakas, and P. Zis. 2015. Factors Associated with Mental Health Status of Medical Residents: A Model-Guided Study. Journal of Clinical Psychology in Medical Settings 22(1). https://doi.org/10.1007/s10880-014-9415-2
47. Klein, J., K. G. Frie, K. Blum, O von dem Knesebeck. 2010. Burnout and Perceived Quality of Care Among German Clinicians in Surgery. International Journal for Quality in Health Care 22(6). https://doi.org/10.1093/intqhc/mzq056
48. Kassam, A., J. Horton, I. Shoimer, and S. Patten. 2015. Predictors of Well-Being in Resident Physicians: A Descriptive and Psychometric Study. Journal of Graduate Medical Education 7(1). https://doi.org/10.4300/JGME-D-14-00022.1
49. Shoorideh, F. A., T. Ashtorab, F. Yaghmaei, and H. A. Majd. 2015. Relationship Between ICU Nurses’ Moral Distress with Burnout and Anticipated Turnover. Nursing Ethics 22(1). https://doi.org/10.1177/0969733014534874
50. Chou, L. P., C. Y. Li, and S.C. Hu. 2013. Job Stress and Burnout in Hospital Employees: Comparisons of Different Medical Professions in a Regional Hospital in Taiwan. BMJ Open 4(2). Available at: https://bmjopen.bmj.com/content/bmjopen/4/2/e004185.full.pdf (accessed September 2, 2020). 
51. Borritz, M., R. Rugulies, K. B. Christensen, E. Villadsen, and T. S. Kristensen. 2006. Burnout as a Predictor of Self-Reported Sickness Absence Among Human Service Workers: Prospective Findings from Three Year Follow Up of the PUMA Study. Occupational & Environmental Medicine 63(2). https://doi.org/10.1136/oem.2004.019364
52. Toppinen-Tanner, S., A. Ojajarvi, A. Vaananen, R. Kalimo, and P. Jappinen. 2005. Burnout as a Predictor of Medically Certified Sick-Leave Absences and Their Diagnosed Causes. Journal of Behavioral Medicine 31(1). https://doi.org/10.3200/BMED.31.1.18-32
53. Rohland, B. M., G. R. Kruse, and J. E. Rohrer. 2004. Validation of a Single-Item Measure of Burnout Against the Maslach Burnout Inventory Among Physicians. Stress and Health 20(2). https://doi.org/10.1002/smi.1002
54. Waddimba, A. C., M. Scribani, M. A. Nieves, N. Krupa, J. J. May, and P. Jenkins. 2015. Validation of Single-Item Screening Measures for Provider Burnout on a Rural Health Care Network. Evaluation & the Health Professionals 39(2). https://doi.org/10.1177/0163278715573866
55. Rabatin, J., E. Williams, L. B. Manwell, M. D. Schwartz, R. L. Brown, and M. Linzer. 2016. Predictors and Outcomes of Burnout in Primary Care Physicians. Journal of Primary Care & Community Health 7(1). https://doi.org/10.1177/2150131915607799
56. Williams, E. S., T. R. Konrad, M. Linzer, J. McMurray, D. E. Pathman, M. Gerrity, M. D. Schwartz, W. E. Scheckler, and J. Douglas. 2002.  Physician, Practice, and Patient Characteristics Related to Primary Care Physician Physical and Mental Health: Results from the Physician Worklife Study. Health Services Research 37. https://doi.org/10.1111/1475-6773.00007
57. West, C. P., L. N. Dyrbye, P. J. Erwin, and T. D. Shanafelt. 2016. Interventions to Prevent and Reduce Physician Burnout: A Systematic Review and Meta-Analysis. The Lancet 388. https://doi.org/10.1016/S0140-6736(16)31279-X
58. Panagioti, M., E. Panagopoulou, P. Bower, G. Lewith, E. Kontopantelis, C. Chew-Graham, S. Dawson, H. van Marwijk, K. Geraghty, and A. Esmail. 2017. Controlled Interventions to Reduce Burnout in Physicians: A Systematic Review and Meta-Analysis. JAMA Internal Medicine 177. https://doi.org/10.1001/jamainternmed.2016.7674
59. Cook, D. A., and T. J. Beckman. 2006. Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application. American Journal of Medicine 119(2). https://doi.org/10.1016/j.amjmed.2005.10.036
60. National Academy of Medicine Action Collaborative on Clinician Well-Being and Resilience. 2018. Validated Instruments to Assess Work-Related Dimensions of Well-Being. Available at: https://nam.edu/valid-reliable-survey-instruments-measure-burnout-well-work-related-dimensions (accessed September 2, 2020).
61. Demerouti, E., A. B. Bakker, F. Nachreiner, and W. B. Schaufeli. 2016. The Job Demands-Resources Model of Burnout. Journal of Applied Psychology 86(3). Available at: https://psycnet.apa.org/doiLanding?doi=10.1037%E2%80%9E%2F%E2%80%9E0021-9010.86.3.499 (accessed September 2, 2020). 
62. Halbesleben, J. R. B., and E. Demerouti. 2007. The Construct Validity of an Alternative Measure of Burnout: Investigating the English Translation of the Oldenburg Burnout Inventory. Work & Stress 19(3). https://doi.org/https://doi.org/10.1080/02678370500340728
63. Qiao, H., and W. Schaufeli. 2010. The Convergent Validity of Four Burnout Measures in a Chinese Sample: A Confirmatory Factor-Analytic Approach. Applied Psychology 60(1). https://doi.org/10.1111/j.1464-0597.2010.00428.x
64. Kristensen, T. S., M. Borritz, E. Villadsen, and K. B. Christensen. 2007. The Copenhagen Burnout Inventory: A New Tool for the Assessment of Burnout. Work Stress 19(3). https://doi.org/10.1080/02678370500297720
65. Estabrooks, P. A., M. Boyle, K. M. Emmons, R. E. Glasgow, B. W. Hesse, R. M. Kaplan, A. H. Krist, R. P. Moser, and M. V. Taylor. 2012. Harmonized Patient-Reported Data Elements in the Electronic Health Record: Supporting Meaningful Use by Primary Care Action on Health Behaviors and Key Psychosocial Factors. Journal of the American Medical Informatics Association 19(4). https://doi.org/10.1136/amiajnl-2011-000576

Tweets by @theNAMedicine