Proficiency Testing: A Useful Process in Cannabis Reformation

Cannabis plants produce a remarkable range of products, including cannabinoids, terpenes, hemp, and popular psychotherapeutics. While cannabis plants have been cultivated by humans for centuries, and perhaps millennia, legal prohibition has precluded cannabis in research, development, and commercial market development for the last 50 years. Society is responding to the excesses of the prohibition by leading a broad-based, popular reformation.

Products of the cannabis reformation need to meet the expectations of today’s consumers. This includes efficacy for intended use, safety, stability, packaging, and labeling. Commerce is based on critical quality attributes (CQAs) that can be measured to assure suitability for intended use. Building a library of validated assay methods guiding the measurement is the typical approach. However, this information is lacking today for cannabis. Yet, the public wants cannabis products—now.

In the absence of a recognized regulatory guidance or prescription, analytical laboratories are developing their own assay methods. Methods have performance measures such as accuracy, precision, detection limits, linearity, interferences, turnaround time, etc. The value of cannabis products is very high, which has raised concern about attracting fraud, including dry-labbing.

Emerald Scientific (San Luis Obispo, CA) has attacked this problem by inviting the laboratories involved in cannabis assays to test samples and report their results as part of an interlaboratory comparison and proficiency testing program. The tests are scheduled in the spring and fall. After five years, the program has grown to include almost 100 participating labs selecting assays from a panel of assays. Interlaboratory testing does not prove the accuracy, but confidence goes up if many independent labs measure similar values. Experience shows that studying these results can uncover important details about the assay method, sample, workflow, and operator.

With this in mind, I had an opportunity to interview Ken Snoke, president and cofounder of Emerald Scientific.

RLS: How did Emerald come about?

KS: My early (<1990s) career in the sciences was in immunology research for several early biotech companies. Even in the lab, I had a strong entrepreneurial interest, and this led me to start several companies over the years. BioSurplus was a company involved in buying and reselling nearly new biotech laboratory instruments such as sequencers, HPLC, and mass spec. Another was Fleet Fuel Testing, which developed kits that enabled transport managers to test the quality of diesel fuel mixed with 20% biofuels used in buses and trucks.

In 2013, it was apparent that the cannabis wave was real and that it needed to be science-based. But due to stigma and legal concerns, the scientific industry was not providing the necessary support that we would typically see in a fledgling industry. As a result, I teamed up with a couple of colleagues and we set up Emerald Scientific, a scientific distribution business to supply quality products and support to the test and production labs in the legal cannabis space.

RLS: So how did Emerald Scientific become involved in proficiency testing?

KS: As the legalization process for medicinal cannabis continued to expand state-by-state, medicinal users wanted assured safety and efficacy. Recreational users wanted high potency. Entrepreneurs jumped into the void by forming labs that offered assays for measuring potency. Then, the states began to implement regulations that require testing for the potency of cannabinoids and the presence of contaminants such as pesticides, solvents, heavy metals, and potentially harmful microbes.

Unfortunately, some early lab operators recognized that their business volume was related to reported THC values, and integrity in the process was undermined. Claimed potency was the key metric, so if 22% THC content sold, they could show a certificate of analysis reporting 30% THC. The numbers would spiral upward. Disputes arose. Consumers were confused.

RLS: This sounds chaotic. It reminds me of the early days of environmental testing in the mid-1970s. Eventually, the EPA had methods published in the Federal Register. Thus, the methods were set. However, they made no provision for error correction or incorporating new technology. What did Emerald do?

KS: It was clear that this situation was untenable. We saw no leadership from the regulatory top (DEA, FDA, NIST, Dept. of Agriculture), plus, the individual states were in no position to provide harmonized methods or uniform quality. So, a few industry stakeholders approached us with the idea of offering a proficiency testing program. Because we were working with the labs on the distribution side, we were a good candidate to organize and manage a PT program. We responded with the Emerald Test, an interlab comparison and proficiency testing program for the cannabis industry. This would help individual labs demonstrate their interest and proficiency in providing good-quality data, and provide them with a means for identifying areas for improvement.

Figure 1 is a results summary in arrow plot format showing the reported potency values from several analysts from a recent interlaboratory challenge. The mean value is for the entire data set of all respondents. The standard deviation of all reports (Z-number) is used to show the scatter of the results from all labs.

ImageFigure 1 – Arrow plots from a recent interlaboratory proficiency test for potency of THC and related cannabinoids organized by Emerald Scientific. The direction of arrows indicates over-reporting (right) and under-reporting (left) compared to the consensus mean. Yellow arrows indicate Z-scores between 2 and 3, and red arrows indicate Z-scores greater than 3. Quick inspection of this plot shows that analyst 0642 is consistently reporting low values compared to the mean. Analyst 1366 is consistently reporting on the high side of the mean.

This data can be reported as a Kernel density plot (Figure 2), where the ideal is a symmetrical narrow peak around the mean. The peak on the right shows a shoulder, which might correlate with a bias between two different methods.

ImageFigure 2 – Scatter plot analysis of the distribution of data from a recent interlaboratory proficiency test. Each lab uses its preferred test protocol. A narrow peak with the majority clustered near the mean is indicative of a good test. A single outlier, as in the plot on the left, is indicative of an individual mistake, while a clustered shoulder (right) may indicate a systematic bias between two or more methods or laboratories that warrants investigation.

Results such as those in Figure 1 are also tabulated by the analytical technology employed. For potency, this includes TLC, HPLC, HPLC-MS, and infrared.

RLS: I see that this PT program has evolved. Can you describe some of the challenges in setting up the Emerald Test?

KS: Cannabis is a very complex matrix compounded by a variety of end products and uses, and there is tremendous financial pressure. We, the stakeholders, are creating a $20 billion multifaceted business in less than a decade. And, we are doing it from the bottom up, since the top is nearly a perfect vacuum, especially in the U.S.A. Creating a program that is relevant across many states with different regulations requires constant adaptation. We created an advisory panel of scientists and regulators to help guide the program, and two years ago we brought in Ken Groggel, a former industry regulator, to direct the Emerald Test program.

We also recognized that the science base of the cannabis industry needed to improve, so we organized the Emerald Conference starting in 2014. The conference provides a focused forum for scientists to meet and talk shop. We’ll meet for the 5th annual Emerald Conference in San Diego at the end of February.

RLS: How has the program worked?

KS: Very well! Integrity and proficiency in the science base has improved dramatically. Many states now require proficiency testing, and the Emerald Test continues to lead the way. Even as we add more analytes and matrices to the test, relative standard deviations continue to tighten. We see state-of-the-art instrumentation, validated methods, and top-tier chemists in our industry now. Again, all from the ground up, without federal oversight. I can’t say enough about my respect for the scientists we work with on a daily basis.

RLS: How do you handle outliers?

KS: Outliers in proficiency tests are important educational opportunities. We learn a lot from those data. From these, we can see if the responses cluster by technology. For example, in the first round for potency testing, two outliers used TLC. We have not had any TLC reports since then. The vast majority now report HPLC.

Personally, I’m technique-agnostic. However, I do see the results of a cross-section of analytical professionals as they select methods for particular assays. Consensus forms very quickly around the best results, especially when investing tens or hundreds of kilo dollars.

RLS: The distribution system for licit products has several steps from nursery to consumer. When is analysis required?

KS: Good question: The distribution system for cannabis products is indeed fragmented. Plus, the natural product is complex. Processed materials such as extracts have stability issues. Infused products have homogeneity issues. Terpenes are volatile. Degree of hydration is important. Also, the value is very high, which attracts fraud.

States want to protect their revenue and the safety of citizens. Thus, one may need a certificate of analysis at multiple points along the supply chain. The products need to be of known quality and suitability for intended purpose, particularly when considering medical applications in children and patients with compromised health. So, the development of adequate sampling schemes will continue to be a very important objective.

RLS: What do you see in the future for PT, especially if the feds get involved?

KS: The Emerald Test will continue to expand, and when the feds come on, we think that they will see PT programs as a major value-add. They certainly have a large hole to fill. They have done an outstanding job with biotech therapeutics, but this has taken 40 years. Public demand for cannabis therapeutics and recreational products will not wait.

We are striving to make the Emerald Test program the model that is adopted, nationally for sure and perhaps globally. The program is working, and it becomes even more relevant with more participants.

RLS: What do you see as the role of ASTM, USP, NIDA, A2LA, and Perry Johnson Lab Accreditation?

KS: The consumer expects products to be fit-for-purpose and safe. Unfortunately, the infrastructure supporting the wide range of products is not entirely worked out. The public and stakeholders are wrestling with how to provide products. These relationships are evolving. I think that ISO 17025 and other accreditations help by providing an overall framework. The tests offered through the Emerald Test program are from ISO 17043-accredited providers to meet the needs of labs obtaining their own ISO accreditation.

For the groups working on standardized methods, such as ASTM, there is a lot of prior technology that needs to be considered and evaluated. Where suitable, the cannabis labs should adopt existing technology. If it is not spot on, then we need to develop and document a method that can be validated and run by a suitably trained staff.

In the early years of standardization, flexibility is important to foster constant improvement. ASTM deserves credit for having a sunset program that calls for updating methods on a five-year cycle. This will be required for the next few decades until things settle down. We have a lot of time to make up.

At the end of the day, we are all working together to develop detailed reference methods. When sensible, these should be harmonized state-by-state, nationally, and internationally. There may be a collection of methods for each stage in production and manufacturing and end use. Stability with time will be an issue as well.

So, together, we must continue on our path of improvement. Patient safety and consumer fairness are at stake. The Emerald Test will serve as an important tool to monitor industry-wide progress and to help individual labs identify areas for improvement. Emerald Scientific will continue to provide a quality supply chain with technical support, and the Emerald Conference will be a meeting place for innovators in the industry. I guess you could say we’re all in.

Robert L. Stevenson, Ph.D., is Editor Emeritus, American Laboratory/Labcompare; e-mail: [email protected]