Validating Process and Action Limits for Automated Liquid Handlers in Volume-Dependent Assays

Errors in automated liquid handling can have a significant impact on data integrity. Since automated liquid handling is a relatively new and rapidly advancing technology, standardized verification practices have not yet emerged. Additionally, many automated liquid handler manufacturers do not provide accuracy specifications. Yet many assays depend on these instruments to dispense volumes with a high degree of accuracy and precision to produce valid results.

One example is the Oncotype DX® breast cancer assay (Genomic Health, Redwood City, CA), a multigene expression test that predicts the likelihood of benefit from chemotherapy as well as disease recurrence. The test is intended to be used by women with early-stage (Stage I or II), node-negative, estrogen receptor-positive (ER+) invasive breast cancer. The assay generates data using several volume-dependent steps to identify the appropriate treatment for patients.

The use of process limits and action limits is one method to build quality into an assay. Process limits dictate the volume range within which an automated liquid handler must perform for confidence in an instrument’s performance. Action limits are points within the process limit range designed to provide statistical confidence that the automated liquid handler is in fact performing within the process limits. If an automated liquid handler performs outside the action limit, action must be taken to correct its performance.

This article describes the use of the MVS® Multichannel Verification System (ARTEL, Westbrook, ME) to validate process and action limits for the Oncotype DX assay and establish the functional performance of the assay within these limits. The study was conducted to streamline quality control operations and confirm the integrity of results generated with the assay. The article also discusses the advantages of using action limits for ongoing quality control of assays relying on accurate and precise liquid handling.

Elements of a quality control strategy

The use of action limits as a quality control tool is beneficial for several reasons. First, action limits provide an objective measurement specification by which to verify the accuracy of an assay’s results. Once the outermost process limits are validated, the action limits can be used as a standard against which to compare automated liquid handler performance. This comparison can then be used to validate assay performance.

The determination of the process limits for a given liquid handling application is critical to provide a baseline for quality control because assays and even steps within assays have varying levels of sensitivity to volume variation. Precision information alone is not sufficient to demonstrate the performance of an automated liquid handler and provide confidence in assay results.

By using action limits as a method for qualifying an automated liquid handler, laboratories can also reduce the need for costly and time-consuming functional testing and streamline performance monitoring of the instrument over time. However, it is important to note that the use of action limits as a quality measure is contingent upon the use of a robust volume verification tool that provides traceable results.

Without the use of action limits and a volume verification tool, more frequent functional testing would need to be conducted to qualify instrument performance. In this case, functional testing would require running actual samples through the Oncotype DX assay and comparing current results to previous results. This process is time consuming and costly, and uses precious resources such as equipment and scientific staff hours. By using validated action limits and a volume verification tool providing standardized results to certify that the automated liquid handler is dispensing within the action limits, confidence in assay results is strengthened.

Validating process limits for the assay

The first step in implementing process controls for a particular assay is to determine the high and low volume outside of which the assay results may be affected. Thus, to validate liquid handling process limits for a given assay, the liquid handler must be deliberately altered to dispense the target volume as well as the high and low process limit volumes throughout the entire assay, at every step that uses a liquid handler.

In the case of the Oncotype DX assay, automated liquid handling is used to handle samples at four distinct steps. These include the RNA quantitation step, followed by genomic DNA detection, and then the reverse transcription step for conversion of RNA into cDNA. The last step is the quantitative PCR assay.

First, the MVS was used to reliably validate that the automated liquid handlers (two Tecan [Männedorf, Switzerland]  Freedom EVO 200 instruments) dispensed the required volumes. For example, at a 2-µL target volume, the tested process limits were 1.6 and 2.4 µL, or ±0.4 µL from target. (Note: These process limits were derived by the Biostatistics Group using statistical analysis of variability data generated previously.)

Because the MVS can rapidly verify the accuracy and precision of volumes dispensed from each channel of an automated liquid handler and provide NIST-traceable results, it was able to be verified with certainty that the instrument was dispensing the volumes required during the process limit study. Table 1 summarizes the output report from the MVS calibration, illustrating that the automated liquid handler was dispensing the desired volume at each level (low, at, high).

After using the MVS to verify that the automated liquid handler was dispensing the desired sample volumes, samples were run at each volume setting (low, at, high) through the Oncotype DX assay. At the conclusion of the assay, results generated with each sample volume level were compared. The output of the assay is Cycle Threshold (CT) data generated by the real-time qPCR assay, which is used to diagnose the likelihood of breast cancer recurrence and likelihood of benefit from chemotherapy for patients. Comparing the CT data from each volume level revealed no statistically significant difference.

When pipetting at the upper and lower process limits, the performance of the Oncotype DX was found to be comparable to its performance when pipetting at target. This finding validated confidence in results generated by the assay when automated liquid handlers perform within the process limits. Action limits were then set inside of the process limits to account for statistical variation in the volume measurements.

Impact of submicroliter variation

This study was especially important for providing confidence in the RNA quantitation step. Tight controls surround the RNA quantification step since it is the most volume-sensitive stage in the Oncotype DX assay. Volume variations as little as 0.1 or 0.2 µL can cause significant variation in the quantification step, causing it to fail rigorous internal quality controls and lead to retesting (see Table 2).

After validating the process limits, we know with confidence that an assay plate that passes the RNA quantitation step will produce accurate results for diagnosing the likelihood of breast cancer recurrence. We also know that our quality control limits at this stage are more than adequate to safeguard the accuracy of the patient diagnosis.

Outcome

Using the MVS, action and process limits can be validated and applied for ongoing liquid handling quality assurance to greatly reduce the need for extensive functional testing, enabling laboratories to improve productivity and data integrity. For assays relying on automated liquid handling, implementation of action and process limits can be an effective method to strengthen confidence in results.

Mr. Volk is Assay Development Engineer, Genomic Health, 301 Penobscot Dr., Redwood City, CA 94063, U.S.A.; tel.: 650-556-9300; fax: 650-556-1132; e-mail: [email protected].