Analytic Validity of Genomic Testing

 

The types of evidence needed to support the use of genome sequencing in the clinic varies by stakeholder and circumstance. In this IOM series, seven individually authored commentaries explore this important issue, discussing the challenges involved in and opportunities for moving clinical sequencing forward appropriately and effectively.

The College of American Pathologists (CAP) defines analytic validity as a test’s ability to accurately measure the analyte of interest. (For the College of American Pathologist’s “Immunohistochemical (IHC) Assays: Principles of Analytic Validation Definitions Reference,” see http://www.cap.org/apps/docs/reference/ihc_list_of_definitions.pdf (accessed November 6, 2014).) In the context of genomic testing using next-generation sequencing (NGS), analytic validation of every potential variant (e.g., single nucleotide variants, insertion/deletions, copy number variants, structural chromosomal rearrangements) is not feasible. Therefore, professional guidelines recommend that NGS test validation assures the accuracy of the entire pathway for different types of variants rather than all specific variants (Aziz et al., 2014; Rehm et al., 2013). Currently, there is an FDA-cleared instrument and reagents, but at this time there is no approved test kit for whole genome sequencing (FDA, 2013). Of note, using present-day NGS technologies, we cannot fully sequence the exome or genome because of an inability to generate high-quality data for regions of high GC content (e.g., 5’ end of the gene) and for pseudogenes or repetitive regions. Anticipated technology advancements should provide more complete coverage of the exome or genome in the future.

During test validation, reference materials are used to determine the false negative, false positive, and accuracy rates. Only recently have the National Institute of Standards and Technology (Genome in a Bottle Consortium – see http://www.nist.gov/mml/bbd/biomolecular/genome_in_a_bottle_consortium.cfm (accessed January 13, 2015).)  and the Centers for Disease Control and Prevention focused on reference material for NGS. (Genetic Testing Reference Materials Coordination Program – see http://wwwn.cdc.gov/clia/Resources/GetRM/materialsavailability.aspx (accessed January 13, 2015).)

As with any clinical test, other testing methods are used to confirm novel results. Novel NGS-based test results can be confirmed by focused Sanger sequencing, pyrosequencing, or real-time polymerase chain reaction methods. For genetic tests, novel variant(s) may be confirmed by testing of parents or other informative family members. Because the false positive rate for NGS-based tests is fairly high, especially for exome or genome sequencing, verification of novel results remains important.

Quality management should be applied to NGS testing as with any clinical test. Quality control measures should determine whether each sample in each run is acceptable. Proficiency testing is required by the Clinical Laboratory Improvement Amendments (CLIA) regulation for any clinical test. The CAP has developed proficiency testing to assess both the sequence data generation (commonly referred to as “wet bench”) and the analytical (commonly referred to as the “dry bench”) components of NGS testing. In addition, an NGS accreditation checklist for CLIA accreditation of laboratories has been developed by CAP and is currently in use by laboratories accredited through CAP.

The greatest difficulty in demonstrating analytic validity of an NGS-based test is the interpretation of sequencing data to define the pathogenicity of a variant. Analysis of a genome or exome can take up to 50 hours (Brownstein et al., 2014). Laboratory professionals use public and private databases, sequence analysis programs, clinical correlations, and other resources to predict pathogenicity of variants. The American College of Medical Genetics and Genomics and the Association for Molecular Pathology are developing standards and guidelines for the interpretation of sequence variants, but without access to clinical quality, evidence-based genomic databases, genomic test interpretation will continue to rely heavily on professional judgment.

In summary, some of the key policy issues in analytic validity of NGS tests are as follows:

1. Development of standards for NGS data filtering and analysis;
2. Availability of well-annotated reference materials for controls and validation;
3. Access to clinical quality, evidence-based genomic databases; and
4. Evidence gaps in phenotype-genotype correlation.

 

---

References

1. Aziz, N., Q. Zhao, L. Bry, D. K. Driscoll, B. Funke, J. S. Gibson, W. W. Grody, M. R. Hegde, G. A. Hoeltge, D. G. Leonard, J. D. Merker, R. Nagarajan, L. A. Palicki, R. S. Robetorye, I. Schrijver, K. E. Weck, and K. V. Voelkerding. 2014, in press. College of American Pathologists laboratory standards for next-generation sequencing clinical tests. Archives of Pathology and Laboratory Medicine. Available at: http://www.archivesofpathology.org/doi/abs/10.5858/arpa.2014-0250-CP (accessed January 13, 2015).
2. Brownstein, C. A., A. H. Beggs, N. Homer, B. Merriman, T. W. Yu, K. C. Flannery, E. T. DeChene, M. C. Towne, S. K. Savage, E. N. Price, I. A. Holm, L. J. Luquette, E. Lyon, J. Majzoub, P. Neupert, D. McCallie, Jr., P. Szolovits, H. F. Willard, N. J. Mendelsohn, R. Temme, R. S. Finkel, S. W. Yum, L. Medne, S. R. Sunyaev, I. Adzhubey, C. A. Cassa, P. I. de Bakker, H. Duzkale, P. Dworzynski, W. Fairbrother, L. Francioli, B. H. Funke, M. A. Giovanni, R. E. Handsaker, K. Lage, M. S. Lebo, M. Lek, I. Leshchiner, D. G. MacArthur, H. M. McLaughlin, M. F. Murray, T. H. Pers, P. P. Polak, S. Raychaudhuri, H. L. Rehm, R. Soemedi, N. O. Stitziel, S. Vestecka, J. Supper, C. Gugenmus, B. Klocke, A. Hahn, M. Schubach, M. Menzel, S. Biskup, P. Freisinger, M. Deng, M. Braun, S. Perner, R. J. Smith, J. L. Andorf, J. Huang, K. Ryckman, V. C. Sheffield, E. M. Stone, T. Bair, E. A. BlackZiegelbein, T. A. Braun, B. Darbro, A. P. DeLuca, D. L. Kolbe, T. E. Scheetz, A. E. Shearer, R. Sompallae, K. Wang, A. G. Bassuk, E. Edens, K. Mathews, S. A. Moore, O. A. Shchelochkov, P. Trapane, A. Bossler, C. A. Campbell, J. W. Heusel, A. Kwitek, T. Maga, K. Panzer, T. Wassink, D. Van Daele, H. Azaiez, K. Booth, N. Meyer, M. M. Segal, M. S. Williams, G. Tromp, P. White, D. Corsmeier, S. Fitzgerald-Butt, G. Herman, D. Lamb-Thrush, K. L. McBride, D. Newsom, C. R. Pierson, A. T. Rakowsky, A. Maver, L. Lovrecic, A. Palandacic, B. Peterlin, A. Torkamani, A. Wedell, M. Huss, A. Alexeyenko, J. M. Lindvall, M. Magnusson, D. Nilsson, H. Stranneheim, F. Taylan, C. Gilissen, A. Hoischen, B. van Bon, H. Yntema, M. Nelen, W. Zhang, J. Sager, L. Zhang, K. Blair, D. Kural, M. Cariaso, G. G. Lennon, A. Javed, S. Agrawal, P. C. Ng, K. S. Sandhu, S. Krishna, V. Veeramachaneni, O. Isakov, E. Halperin, E. Friedman, N. Shomron, G. Glusman, J. C. Roach, J. Caballero, H. C. Cox, D. Mauldin, S. A. Ament, L. Rowen, D. R. Richards, F. A. San Lucas, M. L. Gonzalez-Garay, C. T. Caskey, Y. Bai, Y. Huang, F. Fang, Y. Zhang, Z. Wang, J. Barrera, J. M. Garcia-Lobo, D. Gonzalez-Lamuno, J. Llorca, M. C. Rodriguez, I. Varela, M. G. Reese, F. M. De La Vega, E. Kiruluta, M. Cargill, R. K. Hart, J. M. Sorenson, G. J. Lyon, D. A. Stevenson, B. E. Bray, B. M. Moore, K. Eilbeck, M. Yandell, H. Zhao, L. Hou, X. Chen, X. Yan, M. Chen, C. Li, C. Yang, M. Gunel, P. Li, Y. Kong, A. C. Alexander, Z. I. Albertyn, K. M. Boycott, D. E. Bulman, P. M. Gordon, A. M. Innes, B. M. Knoppers, J. Majewski, C. R. Marshall, J. S. Parboosingh, S. L. Sawyer, M. E. Samuels, J. Schwartzentruber, I. S. Kohane, and D. M. Margulies. 2014. An international effort towards developing standards for best practices in analysis, interpretation and reporting of clinical genome sequencing results in the clarity challenge. Genome Biology 15(3):R53. https://doi.org/10.1186/gb-2014-15-3-r53
3. FDA (Food and Drug Administration). 2013. FDA allows marketing of four “next generation” gene sequencing devices. Available at: http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm375742.htm (accessed January 13, 2015).
4. Rehm, H. L., S. J. Bale, P. Bayrak-Toydemir, J. S. Berg, K. K. Brown, J. L. Deignan, M. J. Friez, B. H. Funke, M. R. Hegde, and E. Lyon. 2013. ACMG clinical laboratory standards for next-generation sequencing. Genetics in Medicine 15(9):733–747. https://doi.org/10.1038/gim.2013.92

Tweets by @theNAMedicine