Making the Case for Continuous Learning from Routinely Collected Data

 

Background

Most people who receive health care in the United States recognize that the system is complicated and fragmented. What they are less likely to know is that opportunities to learn from the care provided in hospitals, clinics, and doctors’ offices are most often lost. As health care records move to electronic systems, the data routinely collected as part of medical care (such as blood pressure measurements, weight, medications lists, disease diagnoses, and past medical histories) hold the promise to dramatically increase the opportunities for learning and improving care on a national scale. Turning data collected at the time of care into knowledge that can be used in clinical practice is essential if we are to achieve a learning health system—a system that continuously and seamlessly uses health care data from across the entire system to answer important questions that matter to patients and their health care providers.

New technologies enable the collection, storage, and analysis of vast amounts of data. As consumers we experience the impact of these “big data” every day—from smartphones that collect information about location and behaviors to Internet browsers that serve up personalized ads based on previous shopping habits. As health care becomes more digital, clinical datasets are also becoming larger and more numerous. These data, gathered largely through the normal course of receiving health care, provide great potential for extracting useful knowledge to achieve the “triple aim” in health care—better care for individuals, better health for all, and greater value for dollars spent.

We are individual participants in the Institute of Medicine’s (IOM’s) Clinical Effectiveness Research Innovation Collaborative (CERIC), an affinity group of researchers, health care providers, advocates, and government officials convened under the auspices of the IOM Roundtable on Value & Science-Driven Health Care, who come together regularly to identify gaps and opportunities to generate better evidence for making informed health care decisions. In our discussions, we have recognized that patients and the public can be effective advocates for resetting expectations that routinely collected clinical data should be used to advance knowledge and support continuous learning to ensure better care, lower costs, and improved health, and that, in fact, most feel the information is already being used in this fashion. (Participants drawn from the Clinical Effectiveness Research Innovation Collaborative of the IOM Roundtable on Value & Science-Driven Health Care.)

In this paper, we outline the various sources of clinical data that are increasingly available for learning to better inform health and health care decisions. We explore meaningful case studies reported in the lay press of how data are being used to

• improve disease monitoring and tracking;
• better target medical services for improved health outcomes and cost savings;
• help inform both patients and clinicians to improve how they make decisions during clinical visits;
• avoid harm to patients and unnecessary costs associated with repeat testing and delivery of unsuccessful treatments; and
• accelerate and improve the use of research in routine medical care to answer medical questions more effectively and efficiently.

 

We expand on the critical importance of engaging stakeholders, especially the public, patients, and clinicians, to make a compelling case for the routine collection of data to support a continuously learning health system built on mutual trust and greater transparency. To help us accomplish this, we looked at examples of clinical practices, programs, and research initiatives such as the Patient-Centered Outcomes Research Institute (PCORI), the Distributed Ambulatory Research in Therapeutics Network (DARTNet), the High Plains Research Network (HPRN), and a consumer information strategy effort under way in the United Kingdom (UK).

 

The Opportunity to Learn from Increasing Sources of Data

The availability and reliability of large volumes of relevant longitudinal digital data from a variety of clinical and nonclinical sources are core features of a system that learns from each care experience, a learning health system. Common clinical repositories include data from electronic health record (EHR) systems used to manage patient care and claims data necessary for billing purposes. In some cases, data sources can be linked, using either institution-specific identifiers or matching algorithms, to create disease-specific patient registries that enable research. Integration of large pools of disparate clinical data from EHRs and claims is a major function of health information exchanges, which will be increasingly important to ensure seamless management of health information across institutions. Nonclinical sources of patient information may also include data from retail sales of over-the-counter medications, dietary supplements, walking and running shoes, and personal preferences and behaviors.

Patient-generated sources of data include patient portals, surveys, and online communities. Patient portals allow patients to access their medical records and contribute information that is often not found in the institutional records, including use of over-the-counter drugs or health care preferences. Outcomes data collected directly from patients may include important assessments of health-related quality of life and satisfaction with care. PatientsLikeMe and similar data-driven online communities provide patients with tools to collect and share their experiences of living with common disorders, like diabetes, or rare ones, like amyotrophic lateral sclerosis (ALS). Patient-reported data can offer insights into what worked and what did not, and these data can be shared with others with the same disorder. Additionally, data collected by patients between visits with their health care providers can be used to focus attention on specific symptoms, medication effects, or other issues of concern, and, with appropriate permissions, have been used to conduct research to answer important questions that matter to the patient community. [2] Efforts such as the Collaborative Chronic Care Network at Cincinnati Children’s Hospital have the capability to link patients’ contributions to their health records to providers and facilities to inform their care. In addition to patient portals, there are other innovative versions of patient-centered health records that encourage patients to report not only outcomes, but also unusual reactions to medications and barriers to adherence to prescribed regimens. Eventually, much of this activity is likely to move to mobile platforms, including smartphones and tablets.

Integration of data outside the realm of clinical data (retail data) requires the use of algorithms containing personal data such as credit account information, an address, or a telephone number. Efforts to integrate large datasets have been accomplished in finance applications but have been slow to be applied to health care. There are, however, noteworthy examples that have created a building momentum for the use of routinely collected clinical data for research, surveillance, and the improvement of care in general, culminating in the development of health information exchanges in several states and in partnership with the Department of Health and Human Services’ Office of the National Coordinator. An early example of a health information exchange is the Indiana Health Information Exchange (http://www.ihie.org), which, in partnership with the Regenstrief Institute, actively pursues research in quality and safety to inform improvements in health care for the region.

There are a number of examples of disparate networks of clinical data that have been brought together for research. One example of this work is the HMO Research Network, a consortium of 19 health care delivery organizations that work together to conduct research with administrative and clinical data extracted from their members’ systems. In order to do this work while preserving privacy, data are downloaded at scheduled intervals from clinical systems to the research centers and are maintained “virtually,” using standardized formats. Using this approach, the data remain behind institutional firewalls and are not stored centrally. When a question is asked that requires data from multiple partners—for example, the association of a treatment with a particular outcome—a program can be written and shared with programmers at each site and the results achieved without having to move or physically combine datasets. [3,4] This distributed approach is also used by the Mini-Sentinel, a Food and Drug Administration (FDA)-sponsored safety surveillance project with the potential to monitor drugs, vaccines, devices, and other areas of FDA surveillance and regulatory concern in more than 120 million persons. [5,6] These “distributed” data network approaches allow large insurance companies and integrated delivery systems to maintain control over their vast databases and at the same time contribute data to investigations of great public value.

 

Successes in Improving Care and Health Through the Use of Clinical Data

The clinical data described in the section above are of little use to anyone if they remain unanalyzed and in the same silos that characterize traditional paper records. It is in the analysis of the data, and in the application of the results, that the potential to drive continuous improvement in health lies. The questions that researchers, hospital administrators, and public health officials are asking of these data are targeted to improve safety, efficiency, and value and further our understanding to improve the health of individuals and the population.

As the availability of digital data has increased, accounts of how these data are used have appeared in professional journals as well as in the lay press. We found reports from broad circulation newspapers, including the New York Times, USA Today, and the Chicago Tribune. These articles demonstrate the potential for the use of routinely collected clinical information to detect and respond to disease outbreaks, target medical services to those who need them most, help patients and clinicians make better decisions, avoid errors that can harm patients, and speed medical research. These concrete examples of how the use of health information can improve the lives of individuals, anchor the case for the routine collection and use of health data to drive a continuously learning health system, and suggest that public awareness of the value of routinely collected data may be increasing.

Better Surveillance and Improved Response

Regular collection and analysis of health information holds great promise for earlier detection and response during disease outbreaks. The rapidly expanding use of EHRs allows public health officials at both the federal and state levels to get a better, more rapid picture of what is happening across the country or in local communities. The Escherichia coli (E. coli) cases from the Jimmy John’s restaurant’s clover sprouts in the U.S. Midwest in 2012 and the international H1N1 flu pandemic of 2009 both offer examples of how public health officials were able to use information sent electronically from hospitals to detect a pattern in disease cases. [7] In the case of E. coli, it was a rash of E. coli cases reported by clinical laboratories that alerted officials, and in the case of H1N1, it was a higher-than-usual number of individuals with flu-like symptoms. These incidents were then investigated more closely, and the public health response was tailored to fit the need. The pattern of the E. coli outbreak was detected, and the restaurant chain promptly stopped serving sprouts, limiting the morbidity and potential mortality and the associated costs. In the case of the H1N1 pandemic, officials were able to monitor the evolution of the pandemic and keep clinicians apprised of the rise and eventual taper of cases.

Better Targeting of Health Care Services

Electronic records, including laboratory results and registries, can be used to better target the delivery of health care services. [8] In Massachusetts, Hepatitis B cases in women of childbearing age are flagged for attention, as infection in pregnant mothers can have dire health consequences for their babies. These babies are vaccinated and their progress monitored. This approach uses the results of a routine test to place health care resources where they can have greatest impact on the health of babies, thereby preventing future medical problems and saving the resources needed to treat them. Similarly, in New York City, information from registries is used to follow patients with problematic chronic conditions, such as asthma, over time. This allows clinicians to ensure that their patients have appropriate follow-up and receive needed therapy, and also alerts clinicians when their patients have visited emergency departments or have been hospitalized. This electronic monitoring has been found to have better results than traditional paper-based systems in reducing emergency room visits and hospital admissions, saving 39 percent of costs for children and 25 percent for adults.

Improved Decision Making and Overcoming Misinformation

The use of routinely collected data provides an opportunity for analyses that can help inform health care decisions being made every day by clinicians and patients. Wider availability of this scientifically-based information can trigger discussions in doctors’ offices, hospital rooms, and even around kitchen tables, allowing patients, clinicians, and policy makers to benefit from more informed discussions in their clinical decision making.

As an example, the analysis of Medicaid data and pharmacy files has shown that a relatively common antibiotic, azithromycin, commonly known as Zithromax, which is used to treat respiratory bacterial and mycoplasmal infections, can cause a small but significant increased risk of sudden death. [9,10] Given the extremely wide use of azithromycin—with more than 55 million prescriptions in 2011 alone—the implications of even a small increase in risk are far-reaching. However, analysis of the data showed that this effect is most pronounced in patients at risk for heart disease. Given this information, clinicians and patients are able to make more informed choices about antibiotics to help ensure that patients with the identified risk factors are not put at unnecessary risk.

Analysis of health information can also help ground health care decisions in empirical data and counteract misinformation. A study on the real risks and benefits of vaccinating children against chicken pox, done by looking at the medical records of children treated at Kaiser Permanente Colorado, demonstrates this nicely. [11,12] The study showed that children whose parents refused to have them vaccinated for chicken pox were nine times more likely to become infected and to require medical care. This evidence counters the misperception that refusing vaccination is without serious medical consequences.

Avoiding Harm and Preventing Costly Errors

Perhaps nothing has catalyzed the move toward the digitization of routinely collected clinical data through EHRs more than the promise of a safer medical care delivery system. Despite both formal regulations (i.e., state requirements that hospitals report cases in which medical care harmed a patient) and voluntary measures (i.e., the FDA’s MEDWATCH system for potential medication adverse events), reporting of instances of patient harm from medical interventions often does not occur, due to both lack of awareness and administrative burden. EHRs have the potential to improve the safety of patient care by becoming a surveillance system to identify potential errors in real time, and to serve as the surveillance backbone by collecting data that can be analyzed to identify emerging issues. [13] These data, collected during the course of care, require no additional burden or even knowledge of separate reporting requirements. Further, such systems can also provide patient-specific warnings, assure timely follow-up, or alert medical staff when laboratory test results suggest abnormal findings or possible adverse drug reactions. These same systems can be used to submit new and important adverse events to the relevant agency in an automated manner with little staff effort.

Few drug safety issues have garnered more attention in the public press than the arthritis medication Rofecoxib, commonly known as Vioxx—so much so that the New York Times has a free collection of their 199 articles about Vioxx on their website. The visibility of the Vioxx case in the public eye was due to the contrast between its aggressive marketing, which included a strong message of safety, and the safety questions and potential missteps that came to light after the product had been on the market for a few years. Using Vioxx as a case study, several groups have retrospectively analyzed pooled clinical trial data14,15 and health care claims data16 to reveal an association between Vioxx and acute heart attacks that might have been detected sooner. [17] Although retrospective analyses of large databases are commonly undertaken when investigating an association between a drug and an outcome, investigators attempted to determine whether the association between Vioxx and heart attacks could have been identified more quickly using large health care claims databases. They concluded that they could have detected the adverse events months earlier and potentially prevented suffering and death. However, others have criticized this conclusion because the investigators had the advantage of hindsight.

Accelerate Medical Research

Large electronic record repositories can also accelerate the generation of evidence so that studies that once took years can be done within a matter of months, often at lower costs. For example, MetroHealth, a health system in Cleveland, Ohio, and Explorys, a health care analytics company, used a database of 14 million medical records gathered from 12 major health systems to replicate a longitudinal Norwegian study of heart disease risk. [18] The Norwegian study followed 26,714 people, examining height and weight, among other measures, and determined that the combination of obesity and tall stature increases the risk of blood clots, especially in men. [19] The study took 13 years. In contrast, the MetroHealth/Explorys researchers looked at the MetroHealth database, and found the same patterns within 3 months. Further, because of the larger sample size (959,030), they were able to generate more precise estimates. MetroHealth/Explorys estimated the costs of their analysis at $25,000, while the Norwegian study cost millions of dollars. [20]

 

 

 

Patient and Provider Engagement is Critical

More Active Engagement of Patients, Providers, and the Public is Critical

The availability and usefulness of data collected from routine health care encounters is growing, yet many stakeholders, particularly the public, may be largely unaware of its potential to improve health and health care. Articles such as those cited from the lay press expose the public to compelling examples of using available clinical data beyond the point of care. In addition, the Internet and smartphone applications (apps) have changed our behaviors and attitudes about sharing information on social networking sites such as Facebook and Twitter. Increasingly, the public, patients, and caregivers are using online platforms and apps to track their health (e.g., FitBit), share their stories (e.g., CarePages), and participate in patient-powered research networks (e.g., PatientsLikeMe).

 

 

Yet, ensuring that every clinical encounter becomes an opportunity to learn and improve our health and health care will require effective communication about the associated benefits, resources, costs, and cost-savings. It is especially important that the process not impose undue burdens on health care providers whose time with their patients is already limited. In addition, as discussed in more detail below, patients will need to be assured that their health data will be protected and used responsibly.

As the health care system in the United States adopts a patient-centered framework and seeks to establish a continuously learning health system, there are increasing expectations that patients will assume a more active role in their own health and health care. To achieve the aim of better health by learning from routinely collected health data, we believe patients and clinicians alike should be full participants in viewing every health care encounter as an opportunity to improve outcomes, not only for the individual patient, but also for others like them. Yet, although most people would not expect to be asked to participate in research as a routine part of the health care experience, over one-third of respondents in a Consumers Reports survey believe that their health data is currently being used to improve care for others. [21] Does this suggest that people may be willing to participate in health services and biomedical research as a public good? [22] Although the answer may be yes, there is still much to be learned from patients about their understanding of what constitutes routinely collected data, how those data are used and by whom, and what protections of their data can be expected.

Promoting and Supporting Responsible Uses of Clinical Data

In general, surveys show that patients support provider use of electronic medical records, but they also consistently demonstrate high levels of concern about the privacy and security of electronic data. [23,24] With respect to research uses of information in medical records, surveys demonstrate that “where there are safeguards to protect identity,” at least 68 percent of the public expressed willingness to allow health information to be used “to detect outbreaks, bio-terror attacks, and fraud, and to conduct research and quality and service improvement programs.” [25] A more recent survey conducted with a nationally representative sample of adults who had at least one medical encounter in the previous year found strong support for sharing health data to improve evidence. Eighty-nine percent of respondents strongly or somewhat agreed with the following statement: “My health data should be used to help improve the care of patients who might have the same or similar condition.” [26]

Consequently, the adoption of privacy, confidentiality, and security safeguards by researchers appears to be key to gaining and sustaining widespread public support for research uses of clinical data. There are laws in the United States that set standards for the use of clinical data for research. The Health Insurance Portability and Accountability Act (HIPAA) privacy and security regulations govern how health care providers and health plans (and business associates performing services on their behalf) can access, use, and disclose identifiable health information, such as health plan beneficiary numbers and admission dates, for research purposes. Entities, such as universities, that receive federal tax dollars for conducting research using identifiable health data, are required to comply with the Common Rule.27 In addition, centers that provide substance abuse treatment services using federal funds are required to comply with rules that govern the use of diagnosis and treatment information that identifies (or potentially identifies) an individual as a substance abuser. [28] Health care and commercial groups conducting research using identifiable health data may also be required to comply with state health privacy laws.

These laws generally require researchers to implement protections for health data used in research. Frequently, research uses of identifiable health data must first be authorized by the patient. In addition, researchers are encouraged to use data for research purposes in the least identifiable form. For example, names, addresses, and other identifying information must often be removed or made undetectable in the research data, and use of “de-identified” data for research purposes is common (see, for example, the DARTNet initiative described below). Research can also be conducted using distributed approaches like the HMO Research Network (see above), where identifiable data remains with the source but is made virtually available for analytics purposes. Such measures help ensure the availability of data for research in ways that protect the patient’s interest in confidentiality with respect to his or her health status.

There is much debate about whether the current privacy protections for data used in research are sufficient, [29] or whether they constitute significant obstacles to more robust uses of clinical data to improve population health. [30] Federal regulators have begun to explore modifications to research regulations to address these concerns. [31] The timing and outcomes of those efforts are uncertain, but the need to build public trust in the use of clinical data for research is a significant and pressing priority. Building public support for research uses of clinical data will be important for satisfactory resolution of privacy and security issues.

Examples of Successful Engagement of Patients and Other Stakeholders

Building the case for successful engagement of the public, patients, and their health care providers can be informed by looking at examples in which routinely collected data are being used to improve processes of care delivery and health outcomes for patients and to support research that emanates routinely from point of care.

As has been suggested by Arthur Kaplan, director of the Division of Bioethics at the New York University Langone Medical Center, clinical data is intended for multiple parties and the point of using information in health care is “ultimately, to benefit patients.” [32] Making an effective case to patients and their clinicians about the value of routinely collected data in supporting timely and informed health care decisions may provide the leverage needed to make the case among payers, researchers, industry, regulators, and policy makers.

The sheer volume of data and what it means may be the greatest barrier to effective engagement among stakeholders. Do all stakeholders know what constitutes routinely collected data? What questions do stakeholders have that routinely collected data could answer? Is being a “data donor” important enough to stakeholders to engage in data collection activities? The examples presented here focus on opportunities for engaging the key participants in utilizing routinely collected data through the involvement of patients in all parts of the research process—within a new U.S. research funding organization, with patients in rural Colorado, with clinicians in practices across the United States, and with the general population in the United Kingdom.

Patient-Centered Outcomes Research Institute

PCORI is a unique U.S. research funding entity created by the Patient Protection and Affordable Care Act of 2010. It was set up specifically to conduct research to give patients a better understanding of the prevention, treatment, and care options available, and the science that supports those options. PCORI emphasizes patient and stakeholder input in the design of research questions, in the review of study proposals, and as part of funded research. To inform the design stage, PCORI offers opportunities for public comment, an engagement workshop, and an open call for research questions and other suggestions. In addition, PCORI is developing research methods that support the engagement and meaningful inclusion of patients at every step of the research process. During the funding review process, PCORI involves professional and lay reviewers in order to ensure that proposals have scientific merit and are patient-centered. Finally, research teams working on studies funded through PCORI must meaningfully involve patients and other stakeholders.

 

 

High Plains Research Network’s Community Advisory Council

HPRN consists of primary care practices, hospitals, other health care facilities, and communities located in eastern rural Colorado, and was set up to translate the best scientific evidence into everyday clinical practice at the local grassroots level. With the help of its Community Advisory Council (CAC), HPRN conducts research and quality-improvement programs that matter to the people who live, work, and spend their time in the region’s small, geographically dispersed communities. Comprised of local residents from across eastern rural Colorado, the CAC guides and informs the work done by the HPRN research team to ensure that it is relevant and meaningful to patients and their providers. The CAC sees its role not as a focus group but as a full participant in the research conducted in the region.

Distributed Ambulatory Research in Therapeutics Network

Clinicians make decisions every day about interventions for their patients that may or may not be based on the best available evidence. Emerging opportunities that provide access to clinical data at the point of care for health care decision making may enhance clinician engagement in data-generating activities. The Distributed Ambulatory Research in Therapeutics Network, or DARTNet, a research network that links EHR data across primary care practices,
community health centers, and other delivery organizations, is such an opportunity. The DARTNet system offers member clinicians in any practice the opportunity to actively engage in research that addresses issues of particular interest in primary care. DARTNet has also established learning communities to facilitate timely dissemination of best-practice information and tested clinical decision support tools with member clinicians and researchers.

 

 

United Kingdom National Health Service

The United Kingdom’s National Health Service is uniquely positioned to utilize routinely collected health data from national electronic care records for research and quality improvement purposes. Despite differences between the U.S. and UK health care systems, there are lessons to be learned from the UK experience of using routinely collected health data in biomedical research for public good. [33]

In spring 2012, the UK Department of Health published a consumer-oriented health and social care information strategy. The information, available online and in print, sets a 10-year framework for transforming, recording, and sharing health and care information so that it is accessible to all care providers. The message to the public is clearly stated in the consumer’s voice: “In time, professionals will be able to access bits of information taken from all of our records that will support them to find out things like who has access to services, what services need to improve, which treatments work and how services can be improved to be safer and save money.” [34]

 

Conclusion

Optimizing use of the growing volume of electronic clinical data is a key objective of the learning health system. Clinical data collected during the course of care provides the opportunity to learn from care experiences and continuously improve future efforts toward the triple aim of better care for individuals, better health for the population, and greater value for the costs of care.

There are numerous examples from the scientific community of how digital data is revolutionizing health care in the areas of disease and medication surveillance; targeting evidence-based services that improve the quality and safety of care for fewer dollars; and creating rich opportunities for clinical researchers to bring clarity to important questions about nebulous areas of health care. Increasingly, linked clinical data sources are being repurposed to improve care effectively and efficiently. Such data come not only from institutional EHRs but also directly from patients who are opening windows on their health care experiences in patient portals and online communities.

As participants in CERIC, we envision an exponential rate of progress in the use of health and health-related data from these varied sources to solve important problems in the prevention and treatment of disease with an eye to lowering the costs of care. Engagement of major stakeholders, most notably patients, in the notion that every health care encounter provides an opportunity to improve outcomes, not only for the individual patient but also for others like them, will be a critical component of this success.

 

Appendix A

The following are additional resources for those looking for more information.

1. The U.S. Agency for Healthcare Research and Quality, http://www.ahrq.gov
2. The Observational Medical Outcomes Partnership, http://omop.fnih.org
3. The Patient-Centered Outcomes Research Institute, http://www. pcori.org
4. The power of information: An information strategy for the UK Department of Health, http://informationstrategy.dh.gov.uk

 

---

 

References

1. Unpublished results from ECIC IOM Messaging Poll, Consumer Reports National Research Center 2012.
2. Wicks, P., T. E. Vaughan, M. P. Massagli, and J. Heywood. 2011. Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nature 29:411-414. https://doi.org/10.1038/nbt.1837.
3. Hornbrook, M. C., G. Hart, J. L. Ellis, D. J. Bachman, G. Ansell, S. M. Greene, E. H. Wagner, R. Pardee, M. M. Schmidt, A. Geiger, A. L. Butani, T. Field, H. Fouayzi, I. Miroshnik, L. Liu, R. Diseker, K. Wells, R. Krajenta, L. Lamerato and C. Neslund Dudas. 2005. Building a virtual cancer research organization. Journal of the National Cancer Institute Monographs 35:12-25. https://doi.org/10.1093/jncimonographs/lgi033
4. Go, A. S., D. J. Magid, B. Wells, S. H. Sung, A. E. Cassidy-Bushrow, R. T. Greenlee, R. D. Langer, T. A. Lieu, K. L. Margolis, F. A. Masoudi, C. J. McNeal, G. H. Murata, K. M. Newton, R. Novotny, K. Reynolds, D. W. Roblin, D. H. Smith, S. Vupputuri, R. E. White, J. Olson, J. S. Rumsfeld, and J. H. Gurwitz. 2008. The Cardiovascular Research Network: A new paradigm for cardiovascular quality and outcomes research. Circulation: Cardiovascular Quality and Outcomes 1(2):138-147. https://doi.org/10.1161/CIRCOUTCOMES.108.801654
5. Mini-Sentinel distributed database “At a Glance,” available at http://www.mini-sentinel.org/about_us/MSDD_Ata-Glance.aspx (accessed January 10, 2013).
6. Platt, R., R. M. Carnahan, J. S. Brown, E. Chrischilles, L. H. Curtis, S. Hennessy, J. C. Nelson, J. A. Racoosin, M. Robb, S. Schneeweiss, S. Toh, and M. G. Weiner. 2012. The U.S. Food and Drug Administration’s Mini-Sentinel
   program: Status and direction. Pharmacoepidemiology Drug Safety 21:1-8. https://doi.org/10.1002/pds.2343
7. Freudenheim, M. 2012. Fast access to records helps fight epidemics. New York Times, June 18. http://www.nytimes.com/2012/06/19/health/states-using-electronic-medical-records-to-track-epidemics.html (accessed December 11, 2012).
8. Ibid.
9. Grady, D. 2012. Popular antibiotic may raise risk of sudden death. New York Times, May 16 http://www.nytimes.com/2012/05/17/health/research/popular-antibiotic-may-raise-risk-of-sudden-death.html (accessed December 11, 2012).
10. Ray, W. A., K. Murray, K. Hall, P. Arbogast, and C. M. Stein. 2012. Azithromycin and the risk of cardiovascular death. New England Journal of Medicine (366):1881-1890. https://doi.org/10.1056/NEJMoa1003833
11. Maugh III, T. M. 2010. Study: Risk of serious chickenpox nine times higher for kids whose parents refuse vaccination. Chicago Tribune, January 7. http://articles.chicagotribune.com/2010-01-07/news/1001070183_1_chickenpox-varicella-vaccine-vaccine-refusal (accessed December 11, 2012).
12. Glanz, J. M., D. L. McClure, D. J. Magid, M. F. Daley, E. K. France, and S. J. Hambidge. 2010. Parental refusal of varicella vaccination and the associated risk of varicella infection in children. Archives of Pediatric and Adolescent Medicine 164(1):66-70. https://doi.org/10.1001/archpediatrics.2009.244
13. Kennedy, K. 2012. HHS: Hospitals ignoring requirements to report errors. USA Today, July 20.
14. Singer, N. 2009. Public database is urged to monitor drug safety. New York Times, November 23 http://www.nytimes.com/2009/11/24/health/24vioxx.html?_r=0 (accessed December 11, 2012).
15. Ross, J. S., D. Madigan, K. P. Hill, D. S. Egilman, Y. Wang, and H. M. Krumholz. 2009. Pooled analysis of rofecoxib placebo-controlled clinical trial data: lessons for postmarket pharmaceutical safety surveillance. Archives
    of Internal Medicine 169(21):1976-1985. https://doi.org/10.1001/archinternmed.2009.394
16. Graham, D. J., D. H. Campen, R. Hui, M. Spence, C. Cheetham, S. Shoor, G. Levy, and W. A. Ray. 2005. Risk of acute myocardial infarction and sudden cardiac death with use of COX-2 selective and non-selective NSAIDs: Nested case control study. Lancet 365(9458):475-481. https://doi.org/10.1016/S0140-6736(05)17864-7
17. Brown, J. S., M. Kulldorff, K.A. Chan, R. L. Davis, D. Graham, P. T. Pettus, S. E. Andrade, M. A. Raebel, L. Herrinton, D. Roblin, D. Boudreau, D. Smith, J. H. Gurwitz, M. J. Gunter, and R. Platt. 2007. Early detection of adverse drug events within population-based health networks: application of sequential testing methods. Pharmacoepidemiology Drug Safety 16(12):1275-1284. https://doi.org/10.1002/pds.1509
18. Zeltner, B. 2012. MetroHealth, Explorys use huge patient database to revolutionize medical research. Plain Dealer, August 29.
19. Borch, K. H., C. Nyegaard, J. B. Hansen, E. B. Mathiesen, I. Njolstad, T. Wilsgaard, and S. K. Braekkan. 2011. Joint effects of obesity and body height on the risk of venous thromboembolism: The Tromsø study. Arteriosclerosis, Thrombosis, and Vascular Biology 31(6). https://doi.org/10.1161/ATVBAHA.110.218925
20. Zeltner, B. 2012. MetroHealth, Explorys use huge patient database to revolutionize medical research. Plain Dealer, August 29.
21. Unpublished results from ECIC IOM Messaging Poll, Consumer Reports National Research Center 2012.
22. Schaefer, G., E. Emanuel, and A. Wertheimer. 2009. The obligation to participate in biomedical research. Journal of the American Medical Association 302(1):67-72. https://doi.org/10.1001/jama.2009.931
23. Kaufman, D., J. Murphy, L. Erby, K. Hudson, and J. Scott. 2009. Veterans’ attitudes regarding a database for genomic research. Genetic Medicine 11:329- 337. https://doi.org/10.1097/GIM.0b013e31819994f8
24. National Partnership for Women & Families. 2012. Making IT meaningful: How consumers value and trust health IT. Available at: http://www.nationalpartnership.org/site/DocServer/HIT_Making_IT_Meaningful_National_Partnership_
    February_2.pdf (accessed November 14, 2012).
25. Markle Foundation. 2011. The public and doctors overwhelmingly agree on health IT priorities to improve patient care. Available at: http://www.markle.org/publications/1461-public-and-doctors-overwhelmingly-agree-health-it-prioritiesimprove-patient-care (accessed November 14, 2012).
26. Alston, C., L. Paget, G. Halvorson, B. Novelli, J. Guest, P. McCabe, K. Hoffman, C. Koepke, M. Simon, S. Sutton, S. Okun, P. Wicks, T. Undem, V. Rohrbach, and I. Von Kohorn. 2012. Communicating with Patients on Health Care Evidence. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/201209d
27. Code of Federal Regulations, Protection of human subjects, title 45, sec. 46.
28. Code of Federal Regulations, Confidentiality of alcohol and drug abuse patient records, title 42, sec. 2.11.
29. Institute of Medicine. 2010. Clinical Data as the Basic Staple of Health Learning: Creating and Protecting a Public Good: Workshop Summary. Washington, DC: The National Academies Press. https://doi.org/10.17226/12212
30. Institute of Medicine. 2009. Beyond the HIPAA Privacy Rule: Enhancing Privacy, Improving Health Through Research. Washington, DC: The National Academies Press. https://doi.org/10.17226/12458.
31. “Human Subjects Research Protections: Enhancing Protections for Research Subjects and Reducing Burden, Delay, and Ambiguity for Investigators; Notice of proposed rulemaking,” 45 Federal Register 46, 160, and 164 (26
    July 2011), pp. 44512-44531.
32. Kaplan, A. 2012. Enhancing patient autonomy through peer review to replace the FDA’s rigorous approval process. Health Affairs 31(10):2236-2240. https://doi.org/10.1377/hlthaff.2012.0793
33. Academy of Medical Sciences. 2006. Personal data for public good: Using health information in medical research. http://www.acmedsci.ac.uk/index.php?pid=48&prid=5
34. See http://informationstrategy.dh.gov.uk.

Tweets by @theNAMedicine