The Significance of Data Harmonization for Credentialing Research

 

Improving the quality of health care requires going beyond the basic standards for health care organizations and formal clinician education. One potential strategy to improve health care quality is credentialing. Credentialing describes specific processes that involve licensure, certification, registration, and accreditation (NOCA, 2005; Styles and Affara, 1998). Currently, only limited information is available about the relationship between credentialing and the quality of care, practice, or patient outcomes. The path to building credentialing evidence is difficult. This is because the process of working toward and having certain credentials may support clinical decision making and the goals of quality, safety, and efficiency of health care (Hickey et al., 2014), while the process of attaining or possessing a credential may serve as a mediating, moderating, or modifying variable in research analyses.

Credentialing requirements for individual health care clinicians and health care organizations vary (Kendall-Gallagher and Blegen, 2009), as do evaluation processes and the type of data collected for different credentialing programs. Several organizations are involved in credentialing programs, including the National Committee for Quality Assurance, the Joint Commission, URAC (formerly known as the Utilization Review Accreditation Commission), and the American Nurses Credentialing Center (ANCC). These programs oversee credentialing and qualification standards for health care organizations and individual clinicians. Organizations that manage credentialing processes typically evaluate the training and skill level of individuals and organizations, as well as their capacity to meet certain standards (ICN, 2009). However, each organization sets the standards for the elements within their respective credentialing program, and determines which data elements are to be collected, maintained, and updated. Similar data are not available for organizations and individuals without credentials. As a result, scientists lack both the data and the data harmonization required to determine the impact of credentialing in general, or to compare the impact of specific credentials on individual and organizational outcomes.

Progress in credentialing research is currently obstructed by datasets that are both insufficiently accessible and inadequately standardized across credential types and credentialing organizations. To gain an understanding of the impact credentialing has on health care organizations, patients, and clinicians, researchers must have access to data that are standardized, accessible, and up to date, as well as defined and consistently structured. Additionally, data must report on organizations and individuals with and without credentials or credentialing programs. Data that meet these criteria can be used to identify trends, enable decision making, and permit the analysis of outcomes. Research findings based on data that do not meet these criteria will be inconclusive, as it will be impossible to distinguish between results that are confounded by data artifacts and those that are not. Applying informatics principles to credentialing research holds promise for creating the infrastructure needed to support credentialing research. This paper briefly summarizes the role of informatics in building a credentialing research infrastructure and in addressing key issues involved in securing, managing, and using data and information associated with credentialing.

 

Overview of Health Care Informatics

Health informatics supports health by integrating health care information management with communication technology and information processing (Bernstam et al., 2010). This discipline is at the intersection of information science, computer sciences, and social science, and provides an infrastructure for the discovery of new knowledge. Health informatics involves all aspects of acquiring, organizing, managing, communicating, and using healthcare-related data, information, and knowledge that enable decision making (Kulikowski et al., 2012). Health informatics is derived from the information science processing model, which posits that data captured in the right context, and transformed into information through active data management, becomes knowledge and then wisdom for an organization (ANA, 2013).

This field has facilitated the development of information systems such as the electronic health record (EHR) and activities such as Health Information Exchange (HIE). The EHR is defined as a longitudinal electronic record of patient health information compiled at one or more meetings in any care delivery setting (HIMSS, 2010) that can be used for real-time documentation and retrospective analyses. The EHR can create a comprehensive record of a clinical patient encounter, and directly or indirectly support numerous health care activities, including evidence-based decision support, quality management, and outcomes reporting (HIMSS, 2010). HIE is the movement of health information that allows health care professionals and patients to appropriately and securely access and share a patient’s personal health information electronically (HealthIT.gov, 2014). HIE has evolved with the growth in electronic data and the increased demand for accessible, usable, and interoperable datasets, and has helped overcome some of the barriers associated with data sharing among proprietary EHR systems.

Health informatics principles can be used to identify required data elements, definitions, standard terminologies, and the infrastructure needed for data representation in credentialing research. Whether the data reside in small databases, large data warehouses, or in the “cloud,” informatics approaches (including temporal reasoning, semantic normalization operations, information retrieval, and big data analytics) can be used to systematically identify, integrate, process, and deliver data, information, and knowledge for researchers, clinicians, and decision makers. Informatics provides the infrastructure needed for conducting research on credentialing, and for exploring its role in facilitating health care decision making.

 

Importance of Interoperability

Interoperability is defined as “the ability of two or more systems or components to exchange information and to use the information that has been exchanged” (Geraci et al., 1991, p. 610). This is the ability to effectively and securely exchange data among information networks, while preserving the operational meaning of that data (IEEE, 2000). Interoperability is essential to accessing data across time and care settings, regardless of where the data originated. The ability of information systems to exchange data in a manner that preserves context and meaning requires data governance. When data in one database cannot be linked to or associated with data in other databases, the electronic processes of organizing, storing, and using data become limited.

Interoperability can be achieved at different levels, as shown in Table 1. Data embedded in paper forms are at the lowest level of interoperability, because the forms, data, and information can be shared and used by humans, but it cannot be processed by computers. At the highest level of interoperability, electronic data are structured, coded, and computable. These data are in a format that a computer can comprehend and process; for example, they can provide real-time alerts to clinicians on drug allergies or drug-drug interactions (GAO, 2010).

 

 

In the context of interoperability to support the provision of health care, data standards must be maintained to meet the information-sharing needs of varied care settings, clinicians, patients, and population health care environments. The interoperable exchange of health information can decrease costs by eliminating duplicate tests, improving administrative efficiencies, improving the accessibility of health care information, and providing information to decrease repetitive input. Interoperability can improve the quality of care by minimizing errors related to insufficient information (e.g., allergic reactions to medications), and by enhancing the accessibility of health care information (GAO, 2010; Sensmeier and Halley, 2011, p. 210).

As is true with health care, an interoperable computing environment for information systems is needed to achieve the goals of credentialing research. Considering the importance of assessing the effect of credentialing on patient outcomes and establishing efficient and high-quality care for patients, the EHR must be designed to enable interoperability between different health care organizations directly and indirectly involved in patient care. Interoperability has to meet legal, ethical, and organizational requirements in a flexible and portable manner. Interoperability and the consistent adoption and use of standards are needed for both care delivery and credentialing research.

Achieving interoperability is a challenging, time-consuming, and costly problem. A root cause is that “organizational systems are often conceived, specified, architected, designed, built, and managed independently of one another” (Choi and Sage, 2012, p. 1). Moreover, information systems and applications have been developed for the purpose of supporting the unique business objectives of a single organization. Achieving interoperability requires that business objectives and goals span multiple organizations and settings. Achieving widespread interoperability is costly. According to a report by the Software Engineering Institute of Carnegie Mellon University, achieving semantic interoperability (e.g., Level 4 from Table 1) increases the total cost of a program by 40 percent (Levine et al., 2003). A National Institute of Standards and Technology (NIST) commissioned study reports $15.8 billion in annual interoperability costs for the capital facility industry (NIST, 2004). However, studies conducted in other industries suggest that the costs associated with the lack of interoperability can be even more costly. In fact, a NIST commissioned study states that at least a $1 billion per year loss for the U.S. automotive supply chain is due to imperfect interoperability (NIST, 2004). These difficulties must be addressed if the quality of health care and credentialing research is to improve.

 

The Importance of Data Harmonization

Data harmonization has been defined as the process of standardizing definitions for core data elements from a variety of sources needed to provide effective care and to conduct reliable research (Liu et al., 2010). Health care data that are harmonized can interface with related data from clinical diagnosis and treatment activities (Liu et al., 2010). Harmonization of credentialing data can enable the assessment of events or outcomes that are associated with individual and organizational credentialing and could also inform credentialing research.

Given the lack of harmonization and current data structures and repositories, it is not possible to isolate the independent effect of a credentialed health care clinician on a particular patient or group of patients. Although it may be assumed that information captured in EHRs across sites of care could be used to assess clinical practices or practice patterns of credentialed versus non-credentialed health care clinicians and organizations, robust methods for isolating the effect of the care provided by a single health care clinician (e.g., physician or nurse) generally do not exist. This is further complicated by the lack of a common data model able to define the complete set of data needed to perform this research. In addition to organizational, health care, and credentialing data, a credentialing data model would include potential confounding factors not captured in EHRs, such as differences in clinical setting and the attitudes, critical thinking/reasoning abilities, education, and experience of clinicians.

An important component of data harmonization is the process of standardizing definitions for core data elements from multiple sources critical to effective care delivery and reliable research (Liu et al., 2010). A significant challenge related to research on the impact of credentials or the credentialing process centers on the lack of alignment of definitions and the inconsistency of the data collection processes. Given the array of available credentialing (both organization and individual), efforts to assess trends or comparisons are not currently feasible. Additionally, many different credentials are used within health care. Each credential may have different eligibility requirements and competencies, and there is wide variation in the evaluation criteria of the multitude of available credentialing programs (ICN, 2009). For example, there are many types of credentials for individuals, including state licensure, board certification for physicians, and certifications for skills and knowledge competency such as cardiopulmonary resuscitation (CPR). Simply capturing whether or not a clinician is credentialed makes it impossible to distinguish CPR credentials from the credentials of an advanced practice nurse. Similar issues are involved in comparing credentialed health care organizations. For example, it would not be methodologically sound to compare outcomes at Baldrige-awarded hospitals to outcomes at Magnet-recognized hospitals, without first comparing the differences in the criteria for the two programs. A process is needed for identifying areas of similarity that can be harmonized, and for recognizing differences in organizational and operational characteristics that could serve as mediating, moderating, or modifying variables in the analysis.

Because the credentialing process (e.g., education, assessment, and evaluation of an individual’s competency) varies by type of credential and credentialing organization, clear definitions of each credential and the associated data elements used in assessment and evaluation are needed. Specifically, a credentialing data model is needed to define the core set of data (e.g., health care, organizational, credentialing, individual clinician) required to evaluate the impact of credentialing on patient, clinician, and organizational outcomes. A common data model would explicitly define the data to be captured, how it would be used, and how credentialing data elements would relate to each other. This would promote credentialing data that are both logically and physically organized within organizational databases, and would ensure that consistent credentialing data are available across organizations. A common data model would also facilitate standardization of the credentialing data collected across organizations, an agreement on metrics, the data elements needed to populate the metrics, and the terminologies needed to represent those data elements. A common credentialing data model would facilitate harmonization of credentialing data and support widespread interoperability.

 

Meaningful Credentialing Data

The recent passage of the American Recovery and Reinvestment Act is expected to improve the quality of care by promoting adoption and “Meaningful Use (MU)” of EHRs. (CMS. 2010. Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Final Rule. Federal Register, 75(144), p. 44314.) The MU criteria put in place by the Centers for Medicare & Medicaid Services require health care organizations and clinicians to demonstrate MU of electronic systems, including EHRs, and to ensure that the data within them are used to improve the quality, cost, and outcomes of health care (CMS, 2014). The first stage of MU focused on improved data quality by standardizing the types of data that are captured within EHRs and the data that are available for sharing across sites and levels of care. Before the implementation of MU, decisions involving EHR clinical content selection and the data format for tracking patient encounters and related outcomes were at the discretion of the organization collecting and maintaining the data. This led to wide variation among data formats and the type of data collected. Over time, the goal of MU is to improve the quality of health care in the United States through better patient and population outcomes. (Meaningful Use Definition and Objectives: http://www.healthit.gov/providers-professionals/meaningful use definition-objectives.) A focus on the MU principles of data standardization and sharing is needed for credentialing data to provide a foundation to support assessment of the unique contributions that credentialed clinicians or organizations make to patient outcomes.

As with EHR data, credentialing data are diverse and come from multiple streams. Health care data used, or potentially used, for evaluating the impact of credentialing can be complex. Identifying and standardizing the data elements needed to support credentialing research is needed, but working with health care data can be challenging. In the absence of a defined set of credentialing data elements, existing health care datasets are mined for data that can be used to support credentialing research. But this practice has its limitations. Health care datasets often have unique definitions and content, reflect different time frames, and may be in a state of flux. For example, hospitals and other health care organizations use the Universal Billing Form (UB04) for collecting patient health care services data from their organizations in order to submit bills to Medicare and other payers for reimbursement. Data included in the UB-04 form include: patient status, condition codes, revenue codes, HCPCS or CPT (The Healthcare Common Procedure Coding System (HCPCS) code set is used and maintained by CMS for billing codes assigned to represent tasks and services provided by health care clinicians to Medicare patients. These codes are based on the CPT (Current Procedural Technology) codes developed by the American Medical Association) procedure codes, and diagnosis codes (CMS, 2006). While these codes could provide valuable information related to patient usage and diagnoses, they are not clinically focused, and may not have the level of granularity necessary to evaluate the impact of credentialing on patient outcomes. In addition, the data are aggregated at the hospital level; even if they could be used for assessing the effect of organizational credentialing, they would not be suitable for differentiating care rendered by individual clinicians with and without credentials. These data are also limited in that the consistency, accuracy, and timeliness of the data collected by the UB-04 are subject to frequent updates, due to corrections and payment updates (e.g., adjudication or reversal). This can create multiple versions of essentially the same data. Other examples of data challenges relate to a general lack of attention to the desiderata for controlled medical vocabularies (Cimino, 1998) that are found within many health care databases. These databases are often subject to changes in the definition or name of a variable, or the addition or elimination of new or old variables. For example, a credentialing organization may change the variables in their credentialing database when the requirements for a particular credential are changed or the credential is no longer offered.

As previously stated, the data required to conduct credentialing research are available in many formats, including paper-based documents or summaries, and electronic databases. Even when available electronically, data can be in a free-text format or stored in isolated or relational databases (Wullianallur and Raghupathi, 2014, p. 5). If health care service and credentialing organizations applied the intent and objectives of MU, then the data and information on credentialing and credentials could be effectively linked to operational, economic, and patient outcome data, and be accessible for trend comparison and for research purposes. To achieve credentialing data that are interoperable across organizations, new governance and technology for storage and new methods for capturing and processing data are needed. The data would need to be formalized with clear identification of data elements (Data element: An atomic unit of data that is defined for electronic processing (e.g., size/number of characters, type, and set of possible values).) Credentialing data are diverse, and a range of data is needed to support credentialing research. Data elements associated with the initiation of a license or certification are needed on the individual provider level. Patient, health care, and organizational data are also needed to conduct longitudinal analyses of patient or organizational outcomes. As with MU, specification of credentialing data, standardizing of data elements, and sharing are prerequisites for developing analytic datasets that will enable the identification and analysis of relationships, and identification of the associations needed to conduct credentialing research.

An overarching goal of credentialing research is to determine the impact of credentialing and differentiate the effects of credentialed organizations or clinicians. To achieve this goal, the principles of MU should be applied to credentialing data. This requires first identifying data and information that isolates the factor(s) or characteristic(s) that are associated, or potentially associated, with having a credential. Credentialing data should be clearly defined using standard metrics, which may exist or need to be developed. Consideration of some of the major factors affecting patient outcomes, as presented in Table 2 (see below), may move our understanding of the impact and effect of credentialing beyond what is currently known. However, the manner in which these data are defined, attained, and managed currently vary between credentialing and health care organizations. Moreover, the data collected exist at different levels of granularity and are suited to different purposes. As was true for health care data before MU, the credentialing data and information an organization maintains are organization-specific, and may not be useful for purposes beyond the management of its own credentialing program.

 

 

As noted in Table 2, credentialing data come from many sources and there are advantages and challenges unique to each source of data. Although data from EHRs may be accessible, there is no national data repository for credentialing of clinicians and organizations. Instead, the majority of credentialing data reside in proprietary databases. For example, ANCC collects and maintains data on organizations and individual clinicians that receive credentialing through one of their many programs. While some of these data have been used in research, other data are used for program management purposes, and comparable data are not available for organizations and clinicians who are not certified. Without a national data repository or standardized data requirements, data collected and maintained by credentialing organizations and health care organizations will vary. Other organizations, such as for-profit credentialing verification organizations, collect and maintain proprietary data on clinician credentials (e.g., licensure and practice/employment history), but these databases are not accessible for research and may not be current. A large proportion of data reflecting individual credentialing is derived primarily from self-report. Some credentialing organizations verify information through defined processes and have defined data management methodology, while others have limited or no standardization. The variability in data sources and contents is a key barrier to the depth and breadth of the data needed to support credentialing research.

Previous research on credentialing provides some examples of how data are used to assess the effect of organizational credentialing. For example, ANCC data linked at the hospital level to other databases has been used to compare Magnet Recognition Program hospitals to those without Magnet credentialing. These studies used the Magnet recognition credential as a dichotomous variable (e.g., Magnet vs. non-Magnet) (Evans et al., 2014; Stimpfel et al., 2014; Kalisch and Lee, 2012; Aiken et al., 1994), and as the only characteristic to signal an organization with a potentially different care environment and patient outcomes. While these studies have been able to differentiate the two types of organizations, there tend to be more clinicians with individual credentials in Magnet hospitals. Thus, it may not be easy to differentiate a health care organization by a single variable, nor may a single variable adequately represent the effect of organizational certification. Other studies have assessed additional hospital characteristics, such as work environment (McHugh et al., 2013), nurse staffing and education (Aiken et al., 2011), hospital size, ownership, resources, and teaching status (Abraham et al., 2011; Powers and Sanders, 2013), in order to determine the differences among hospitals that opted for Magnet recognition.

Data use and analysis for purposes of providing health care services, as well as conducting research (including credentialing research), are facilitated by standardization of data elements, and an electronic format that is structured and coded. Yet, for many health care organizations, the value of data they collect, store, and maintain is dependent on several factors, including the way that data and information are stored; which platform is used; the capability of information systems to process data; and the opportunity to retrieve data in a manner that can be used to effectively inform clinical practice, decision making, and research. As a whole, the degree to which electronic data are complete, accurate, and consistent is as important as the ability of that data to be readily extracted and reused for credentialing research.

The ability to access and use health care data also depends on the ability to exchange and link data both meaningfully and accurately in order to facilitate health care and enable researchers to address key questions regarding credentialing research. At a minimum, databases need to be designed and maintained with a governance structure that enables access and use. Informatics principles provide a pragmatic approach to solving problems with data and knowledge, taking into account the limitations of electronic data.

 

Barriers to Optimal Data Management and Use

Despite the increased amount of electronic data, data-driven research approaches to evaluating key questions are dependent on the accessibility of data. This may be problematic for those charged with balancing the desire to make data accessible with the obligation to protect privacy and preserve intellectual capital. In addition to access, researchers must have the technology, knowledge, and expertise necessary to understand and effectively manage the data made available to them.

Other barriers to credentialing research include those related to data content, format, and quality. The majority of the data that may potentially be used to assess the outcomes of credentialing were originally collected for other purposes. Although some of these data can be reused, their contents may be extensively heterogeneous, reflecting the socio-organizational characteristics of the credentialing and health care organizations that first collected the data. Moreover, there are inherent data integrity challenges associated with changes in program goals, lack of standard definitions, and an overall lack of attention to established desiderata (Cimino, 1998) that make it difficult to harmonize existing data. In addition, if data are not near-real–time, comprehensive, consistent, complete, and systematically collected from credentialed organizations and clinicians, then the quality of the data may be questionable. This limits both the ability to conduct research and the methods available to address systematic biases and other limitations of the data. Data quality is a barrier that warrants further research.

To evaluate the impact of credentialing on patient and organizational outcomes, a variety of patient, clinician, organizational, and credentialing data are required. In addition, approaches are needed to link these data, which currently arise from disparate sources. Historically, the electronic availability of health care information has emphasized the formal implementation of EHRs (Wullianallur and Kesh, 2007). Most of these EHRs are composed of applications and relational databases that focus on intra-enterprise information systems. Few EHR systems are fully functional, scalable, distributed, and interoperable. EHR systems capture patient information. External information systems, such as organizational and credentialing systems, are also needed to conduct credentialing research. Creating a technical environment where all of the data needed to conduct credentialing research is available will require a transformation from the current state characterized by a lack of harmonization and interoperability, to the desired future state of widespread harmonization and interoperability. This transformation is an opportunity that must be seized in order to enable researchers to answer the question, “Does credentialing make a difference in patient outcomes?”

 

Moving Forward

An essential requirement for credentialing research is a technical architecture to facilitate harmonization of credentialing data, and interoperability of systems used to capture credentialing data. A common data model is needed to support the business requirements (see above section on Importance of Interoperability) associated with credentialing and credentialing research. Currently, there is a paucity of agreement regarding what data related to individual and organizational credentials should be collected. Given the wide array of data required to evaluate the impact of credentialing, research is needed to explore data models that delineate health care, provider, organizational, and credentialing data. Research is also needed to define the range of data required for large-scale credentialing research, and for exploring the impact of credentialing on patient and organizational outcomes.

In addition to a common data model, cloud computing may offer another technical architecture that supports interoperability. NIST defines cloud computing as “a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction” (NIST, 2011, p. 2). Shared, cloud-based big data solutions can provide a mechanism for accessing the health care, provider, organizational, and credentialing data required for largescale credentialing research. The adoption of cloud computing could facilitate credentialing research and discovery through big data aggregation, mining, and analysis. However, common data models are a prerequisite for both service-oriented architecture and for cloud-computing solutions. Therefore, federal support is needed for research that defines the data, data structures, and relationships required to measure the impact of credentialing.

 

Conclusion

The application of health care informatics principles to transform data into information, knowledge, and wisdom can facilitate the future of credentialing research. This research needs to focus on the effects of individual and organizational credentialing on outcomes. A prerequisite is that data must be captured and structured in a manner that enables the support of this research. The informatics infrastructure currently in place to support credentialing research is deficient. There is great variety in the contents, size, and format of health care datasets that could potentially be used in credentialing research. A common data model and sound technical architectures are needed to bridge this variation, and to promote interoperability among the multitude of systems used to capture pertinent data. Research funding is needed to achieve consensus on a common data model, and to explore technical architectures that are based on standards and are vendor, product, and technology independent. These efforts would allow researchers to isolate the effect that credentials and credentialing have on quality of care and patient outcomes.

 

---

References

1. Abraham, J., B. Jerome-D’Emilia, and J. W. Begun. 2011. The diffusion of Magnet hospital recognition. Health Care Management Review 36(4):306-314. https://doi.org/10.1097/HMR.0b013e318219cd27
2. Aiken, L. H., H. L. Smith, and E. T. Lake. 1994. Lower Medicare mortality among a set of hospitals known for good nursing care. Medical Care 32(8):771-787. https://doi.org/10.1097/00005650-199408000-00002
3. Aiken, L. H., J. P. Cimiotti, D. M. Sloane, H. L. Smith, L. Flynn, and D. F. Neff. 2011. The effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Medical Care 49(12):1047. https://doi.org/10.1097/MLR.0b013e3182330b6e
4. ANA (American Nurses Association). 2013. Nursing informatics: Scope and standards of practice, 2nd ed. Silver Spring, MD: American Nurses Association.
5. Bernstam, E. V., J. W. Smith, and T. R. Johnson. 2010. What is biomedical informatics? Journal of Biomedical Informatics 43(1):104-110. https://doi.org/10.1016/j.jbi.2009.08.006
6. Brull, R. 2012. What is consolidated CDA? Available at: http://www.hl7standards.com/blog/2012/03/22/consolidated-cda (accessed July 1, 2014).
7. Choi, D., and A. P. Sage. 2012. A framework for interoperability assessments in systems of systems and families of systems. Information, Knowledge, Systems Management 11(3):275-295. https://doi.org/10.3233/IKS-2012-0211
8. Cimino, J. J. 1998. Desiderata for controlled medical vocabularies in the twenty-first century. Methods of Information in Medicine 37(4-5):394-403. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3415631/ (accessed June 22, 2020).
9. CMS (Centers for Medicare & Medicaid Services. 2006. Centers for Medicare & Medicaid Services Manual System: Pub. 100-04 Medicare Claims Processing, Transmittal 1104. Washington, DC: CMS. Available at: http://www.cms.gov/Regulations-andGuidance/Guidance/Transmittals/downloads/R1104CP.pdf (accessed July 1, 2014).
10. CMS. 2014. EHR incentive programs. Available at: http://www.cms.gov/Regulations-andGuidance/Legislation/EHRIncentivePrograms/index.html?redirect=/EHRIncentivePrograms/01_Overview.asp (accessed July 1, 2014).
11. Evans, T., K. Rittenhouse, M. Horst, T. Osler, A. Rogers, J. A. Miller, C. Martin, C. Mooney,  and F. B. Rogers. 2014. Magnet hospitals are a magnet for higher survival rates at adult trauma centers. Journal of Trauma and Acute Care Surgery 77(1):89-94. https://doi.org/10.1097/TA.0000000000000262
12. GAO. 2010. Electronic health records: DoD and VA interoperability efforts are ongoing; program office needs to implement recommended improvements (GAO-10-332). Washington, DC: GAO.
13. Geraci, A., F. Katki, L. McMonegal, B. Meyer, J. Lane, P. Wilson, J. Radatz, M. Yee, H. Porteous, and F. Springsteel. 1991. IEEE standard computer dictionary: Compilation of IEEE standard computer glossaries. Piscataway, NJ: IEEE Press.
14. HealthIT.gov. 2014. Health Information Exchange (HIE). Available at: http://www.healthit.gov/HIE (accessed October 7, 2014).
15. Hickey, J. V., L. Y. Unruh, R. P. Newhouse, M. Koithan, M. Johantgen, R. G. Hughes, K. B. Haller, and V. A. Lundmark. 2014. Credentialing: The need for a national research agenda. Nursing Outlook 62(2):119-127. https://doi.org/10.1016/j.outlook.2013.10.011
16. HIMSS. 2011. HIMSS statement on the definition of the electronic health record. Available at: http://www.himss.org/library/ehr/ (accessed October 7, 2014).
17. ICN (International Council of Nurses). 2009. Nursing matters fact sheets: Credentialing. Geneva, Switzerland: ICN. Available at: http://www.icn.ch/images/stories/documents/publications/fact_sheets/1a_FSCredentialing.pdf (accessed July 31, 2014).
18. IEEE. 2000. Recommended practice for architectural description of software-intensive systems. ANSI/IEEE 1471-2000.
19. Kalisch, B., and K. H. Lee. 2012. Missed nursing care: Magnet versus non-Magnet hospitals. Nursing Outlook 60(5):32-39. https://doi.org/10.1016/j.outlook.2012.04.006
20. Kendall-Gallagher, D., and M. A. Blegen. 2009. Competence and certification of registered nurses and safety of patients in intensive care units. American Journal of Critical Care 18(2):106-113. https://doi.org/10.4037/ajcc2009487
21. Kulikowski, C. A., E. H. Shortliffe, L. M. Currie, P. L. Elkin, L. E. Hunter, T. R. Johnson, I. J. Kalet, L. A. Lenert, M. A. Musen, J. G. Ozbolt, J. W. Smith, P. Z. Tarczy-Hornoch, and J. J. Williamson. 2012. AMIA Board white paper: Definition of biomedical informatics and specification of core competencies for graduate education in the discipline. Journal of the American Medical Informatics Association 19:931-938. https://doi.org/10.1136/amiajnl-2012-001053
22. Levine, L., C. Meyers, E. Morris, P. Place, and D. Plakosh. 2003. Proceedings of the System of Systems Interoperability Workshop. Workshop conducted by the Software Engineering Institute, Pittsburgh, PA. Pittsburgh, PA: Carnegie Mellon University. Available at: http://www.sei.cmu.edu/publications/documents/03.reports/03tn016.html (accessed December 1, 2014).
23. Liu, D., X. Wang, F. Pan, P. Yang, Y. Xu, X. Tang, J. Hu, and K. Rao. 2010. Harmonization of health data at national level: A pilot study in China. International Journal of Medical Informatics 79(6):450-458. https://doi.org/10.1016/j.ijmedinf.2010.03.002
24. McHugh, M. D., L. A. Kelly, H. L. Smith, E. S. Wu, J. Vanak, and L. H. Aiken. 2013. Lower mortality in Magnet hospitals. Medical Care 51(5):382-388. https://doi.org/10.1097/MLR.0b013e3182726cc5
25. NIST (National Institute of Standards and Technology). 2004. Cost analysis of inadequate interoperability in the U.S. capital facilities industry (NIST-GCR 04-867). Gaithersburg, MD: NIST. Available at: http://www.bentleyuser.dk/sites/default/files/nist_report.pdf (accessed December 1, 2014).
26. NIST. 2011. The NIST definition of cloud computing: Recommendations of the National Institute of Standards and Technology (special pub. 800-145). Gaithersburg, MD: NIST. Available at: http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf (accessed December 1, 2014).
27. NOCA (National Organization for Competency Assurance). 2005. The NOCA guide to understanding credentialing concepts. Washington, DC: NOCA.
28. Powers, T. L., and T. J. Sanders. 2013. Environmental and organizational influences on Magnet hospital recognition. Journal of Healthcare Management 58(5):353-366. Available at: https://pubmed.ncbi.nlm.nih.gov/24195343/ (accessed June 22, 2020).
29. Sensmeier, J., and E. C. Halley. 2011. Standards and interoperability. In Nursing informatics, edited by M. J. Ball, K. J. Hannah, D. DuLong, S. K. Newbold, J. E. Sensmeier, D. J. Skiba, M. R. Troseth, B. Gugerty, P. Hinton Walker, and J. V. Douglas. London, England: Springer. Pp. 207-217.
30. Stimpfel, A. W., J. E. Rosen, and M. D. McHugh. 2014. Understanding the role of the professional practice environment on quality of care in Magnet® and Non-Magnet hospitals. Journal of Nursing Administration 44(1):10-16. https://doi.org/10.1097/NNA.0000000000000015
31. Styles, M. M., and F. A. Affara. 1998. ICN on regulation: Towards 21st century models. Geneva, Switzerland: International Council of Nurses.
32. Wullianallur, R., and S. Kesh. 2007. Interoperable electronic health records design: Towards a service-oriented architecture. E-Service Journal 5(3):39-57. https://doi.org/10.1353/esj.2008.0003
33. Wullianallur, R., and V. Raghupathi. 2014. Big data analytics in healthcare: Promise and potential. Health Information Science and Systems 2(3). https://doi.org/10.1186/2047-2501-2-3

Tweets by @theNAMedicine