More rigor needed in molecular test development and validation
It is clear that molecular diagnostics will play an increasingly important role in the treatment of patients due to the information we are learning from genomic analysis and the continued introduction of targeted therapies. Despite the potential and importance of molecular testing there are risks to the utilization of these tests, especially as the number of tests grows. Critics of the current state of molecular testing describe it as chaotic and there is significant variation in the rigor of development, validation and implementation of tests into laboratories.
Test claims are made and supported in some cases and not supported in other cases. In addition, few controls are in place to monitor and regulate changes in the claims and performance of products once on the market. For example, companies can change test specifications and reporting methods, without providing new validation studies to support those changes.
This lack of consistency and rigor could risk public health. The field needs clearly articulated standards for the demonstration of analytical and clinical validity and clinical utility.
CLIA and CAP primarily focus on evaluation of laboratory procedures
Currently, laboratories in the United States are evaluated and certified under the Clinical Laboratories Improvement Act (CLIA) and the College of American Pathologists (CAP). These organizations are primarily focused on the quality of the laboratory’s procedures, methods, quality control, proficiency and personnel qualifications and training, all of which are essential to demonstrate that a lab can run tests properly and deliver consistent, reliable results.
However, these certification processes are not focused on nor adequately equipped to evaluate the development, validation and performance claims of many newly developed complex molecular diagnostic tests.
FDA is best equipped to standardize and regulate complex molecular diagnostic tests
Currently the Food and Drug Administration has the most expertise for evaluation of the development, validation and associated performance claims of complex diagnostic tests. However, currently, new diagnostic tests can be launched as Laboratory Developed Tests without FDA review and clearance.
Successful completion of the FDA regulatory clearance process is a reliable indication that a test has been developed in a controlled way and is rigorously validated. For these reasons, Pathwork Diagnostics secured FDA clearance for the Pathwork® Tissue of Origin Test — a molecular test that uses a patient tumor’s RNA expression profile to assist in identifying its origin.
Per the FDA’s requirements, Pathwork followed design control procedures for development of the test, including determination of product specifications prior to product development, careful analysis of hazards and how to mitigate them, designated testing stages and confirmation of successful completion at each step of development.
Pathwork believes that the field would benefit from a transition to the requirement that all new molecular tests require FDA clearance before commercial launch. To be implemented, it would require increased FDA resources, clear guidance from the FDA as to standards for clearance and list of tests exempt from clearance due to low complexity, small patient numbers or minimal risk to public health.
More FDA clarity is needed re standards
Unfortunately, the FDA is still vague regarding what validation and performance standards it expects for each new type of molecular test. In 2007, the FDA issued its draft guidelines, “Guidance on In Vitro Diagnostic Multivariate Index Assays (IVDMIAs),” but these have not gone into effect and there continues to be discussion and debate regarding the appropriate standards.
The FDA faces substantial challenges in regulating molecular diagnostics, including the rapid pace of introduction of new technologies, the multiplicity of different applications and the varying level of unmet medical need and public health risk. Some molecular diagnostics have highly complex informatics approaches, involving computer-based algorithms in contrast to reporting of data for single analytes. These issues, plus the FDA’s limited resources, have made it difficult for the agency to create molecular diagnostics standards.
Standards should incorporate clinical utility
Data supporting clinical utility needs to be different from what is needed for drugs, but currently there are no accepted standards for what clinical utility data should be required for a particular molecular diagnostic test or class of tests.
We believe the standard for diagnostics should vary with the intended use of the test. For example, the standard for a test that predicts response to therapy should be different than a test that aids in diagnosis. In the former, response to the drug is a fundamental aspect of the test’s utility. However, for a diagnostic aid, demonstration that the test does in fact aid in diagnosis and results in a change in physician treatment decisions is appropriate evidence for clinical utility.
Pathwork has proactively collected and analyzed clinical utility data regarding the use of the Tissue of Origin Test. For example, a clinical utility study of the Tissue of Origin Test presented at the ASCO 2011 Gastrointestinal Cancers Symposium showed that after receiving the test results for patients with difficult-to-diagnose primary cancers, oncologists’ determination of the primary diagnosis was changed in the majority of patients and cancer-specific management changed for two-thirds of the patients.1
Lack of payer standards an obstacle
Payers essentially constitute a second tier of regulation and a barrier to the commercial success of new molecular diagnostics. Unfortunately, each payer individually decides what standards should be applied to diagnostic tests, resulting in significant inefficiency and wildly different standards for analytical and clinical validity, clinical utility and cost effectiveness.
The lack of consistent payer standards creates great uncertainty for any company bringing out a new test. It goes without saying that a rational and uniform set of payer standards would greatly benefit the development and commercialization of diagnostics, but that is unlikely to occur until the regulatory bodies and health insurance industry can join together to better articulate them.
Molecular diagnostics companies can help create standards
While the molecular diagnostics industry waits for the regulators and advisory bodies to catch up, it remains an obligation of each diagnostics company to make sure that any test it introduces is rigorously designed and validated and that these steps are all well-documented. The test should be implemented in a laboratory environment that is fully CLIA- and CAP-compliant. Very importantly, companies should be transparent about exactly what claims can be made regarding the performance of a new test and provide the data that backs up any claims.
Validation standards should be appropriate to the type of test and should utilize studies that are appropriately powered and have tight confidence intervals. For the Tissue of Origin Test, which covers 15 different tumor tissues representing 58 subtypes, we performed extensive validation, which included the analysis of 25 samples per tissue type, for a total of 462 formalin-fixed, paraffin-embedded (FFPE) tumor specimens in our study published in the Journal of Molecular Diagnostics.2
Most of all, we as molecular diagnostics test providers need to be transparent about what they have done and how it has been done it at every step of product design, development, validation, regulatory clearance and commercialization.
References
1Hornberger et al. Effect of a gene expression-based tissue of origin test’s impact on patient management for difficult-to-diagnose primary cancers. ASCO Gastrointestinal Cancers Symposium; January 22, 2011; San Francisco, CA. Poster presentation.
2Pillai et al. Validation and reproducibility of a microarray-based gene expression test for tumor identification in FFPE specimens. J Mol Diagn. 2011;13:48-56.
By W. David Henner, MD, PhD, Chief Medical Officer, Pathwork Diagnostics, Inc.
