Statistics in Analytical Chemistry: Part 31—Receiver Operating Characteristic (ROC) Curves: Equations

Please do not despair! This article is not a series of formulas to be memorized. The content is all algebra, resulting in the derivation of the equation for Receiver Operating Characteristic (ROC) curves. (This topic was included in the detection-related “to discuss” list in Part 28 of this series, American Laboratory, Nov/Dec 2007). The material is being presented because the final equation is needed to generate ROC curves for any given data set. For simplicity, all symbols are defined in a Glossary at the end of this article. Now, “let the games begin”! 

Figure 1 - Graphical representation of the Hubaux-Vos detection limit.

This derivation combines the concept of ROC curves (Part 27, American Laboratory, Sept 2007) with the detection-limit theory of Hubaux and Vos (Part 29, American Laboratory, Feb 2008), and the equation for prediction limits (Part 4, American Laboratory, Mar 2003). Figure 1 is the basic plot describing the Hubaux-Vos detection limit. Recall from Part 4 that for a straight-line model, the general equation for a prediction limit is:

The UPL depends on the value of α and the LPL depends on the value of β; the two values are user-chosen and do not have to be equal to each other. Thus, the following equations can be written, starting with Eq. (1):

At x = x₀ and y = T, one can see that:

yUPL,x₀ = response corresponding to the UPL point where x_i = 0 = x₀            (4)

At x = x_D and at y = T, one can see that:

yLPL,x_D = response corresponding to the LPL point where x_i = x_D           (5)

Since these two responses are equivalent, it holds that:

yUPL,x₀ = yLPL,x_D              (6)

Substituting Eqs. (2)–(5) into Eq. (6) gives:

Or,

[a + (b ∗ x₀)] + [t_1-α ∗ (RMSE) ∗ R₀] = [a + (b ∗ x_D)] – [t_1-β ∗ (RMSE) ∗ Rx_D],        (8)

where R₀ and Rx_D are defined for convenience (see Glossary).

The “a” terms cancel, and x₀ = 0, leaving:

[t_1-α ∗ (RMSE) ∗ R₀] = (b ∗ x_D) – [t_1-β ∗ (RMSE) ∗ Rx_D]                (9)

Rearranging gives:

[t_1-α ∗ (RMSE) ∗ R₀] – (b ∗ x_D) = – [t_1-β ∗ (RMSE) ∗ Rx_D]                 (10)

Multiplying through by (–1) yields:

(b ∗ x_D) – [t_1-α ∗ (RMSE) ∗ R₀] = [t_1-β ∗ (RMSE) ∗ Rx_D]                (11)

Isolating t_1-β results in:

{(b ∗ x_D) – [t_1-α ∗ (RMSE) ∗ R₀]}/ [(RMSE) ∗ Rx_D] = t_1-β                  (12)

Or,

{(b ∗ x_D)/[(RMSE) ∗ Rx_D]} – [(t_1-α ∗ R₀)/Rx_D] = t_1-β                   (13)

Eq. (13) is the final equation and is the relationship that was used to generate the ROC curves presented in Part 27. For each graph, the process involved establishing a table with a column for each of the various detection limits (DLs) of interest. Also included were columns for α, β, and the corresponding t values. Creating an overlay plot of the possible α-and-β combinations for each detection limit resulted in each family of ROC curves. Note that Eq. (13) contains three variables: α, β, and x_D. If any one of them is selected, then each of the remaining two can be expressed as a function of the other.

Eq. (13) is also helpful if the user has calculated a detection limit via the widely applied “3-sigma” approach (which will be the theme of the next installment of this series). With this DL-calculation procedure, α is chosen to be 0.01 (i.e., the false-positive rate is set to 1%). The detection limit is calculated by a specific formula involving: 1) the standard deviation of the response and 2) the Student’s t value associated with the number of replicates. Once this computation is complete, Eq. (13) can be used to determine the value of β that is associated with this DL. Then the user has all the information needed to make practical decisions about sample results that are near the DL.

In the scenario discussed in the above paragraph, it would be “the luck of the draw” if the value of β were found to be the same as the value of α. While there is nothing that says that these two probabilities have to be equal, the user can select them to be the same. For example, if one sets α = β = 1%, then what DL will result? Eq. (13) can be rearranged to provide the answer.

The equivalence of the two probabilities means that t_1-α = t_1-β, and Eq. (13) becomes:

{(b ∗ x_D)/[(RMSE) ∗ Rx_D]} – [(t ∗ R₀)/Rx_D] = t            (14)

Combining “t” terms on one side of the equation gives:

(b ∗ x_D)/[(RMSE) ∗ Rx_D] = t + [(t ∗ R₀)/Rx_D]                  (15)

Multiplying through by {[(RMSE) ∗ Rx_D]/b} gives:

x_D = { t + [(t ∗ R₀)/Rx_D] } ∗ {[(RMSE) ∗ Rx_D]/b}             (16)

The “t” term can be multiplied by (Rx_D/Rx_D) to give:

x_D = {[(Rx_D ∗ t)/Rx_D] + [(t ∗ R₀)/Rx_D]} ∗ {[(RMSE) ∗ Rx_D]/b}              (17)

The Rx_D terms cancel, resulting in:

x_D = [Rx_D ∗ t) + (t ∗ R₀)] ∗ [RMSE/b]                        (18)

Or,

x_D = t ∗ (Rx_D + R₀) ∗ (RMSE/b)                   (19)

This equation holds for any equal pair of α and β. The value of x_D will correspond to the graphically determined DL that was illustrated in Parts 29 and 30 of this series. Eq. 19 is also helpful for developing a deeper understanding of how to achieve a lower Hubaux-Vos detection limit. The RMSE/b term represents the noise (standard deviation) in the measurement data, converted to concentration units. The detection limit is proportional to noise; reduce the noise by a factor of, e.g., two, and the detection limit will be reduced by a factor of two. The t factor behaves similarly, but the options may be less palatable. The only ways to reduce t are to increase α and β, or to increase the sample size, n, greatly. Increasing n will also have a slight decreasing effect on Rx_D and R₀. Altogether, the best opportunity to reduce the DL is to reduce noise. This reduction can be achieved by improving the technology and/or the analytical method (including sample size and sample preparation). Most simply, single-sample measurements can be replaced with multiple-sample measurements, which are averaged and reported. (In general, reporting the average of k measurements reduces RMSE by a factor of √k.)

Glossary

a = y-intercept of the straight-line model

b = slope of the straight-line model

CL = calibration line

LPL = lower prediction limit

n = number of data points in the data set

R₀ = (1 + (1/n) + [(x₀ – x_avg)²/S_xx])^1/2

Rx_D = (1 + (1/n) + [(x_D – x_avg)²/S_xx])^1/2

RMSE = Root Mean Square Error

S_xx = Σ(x_i – x_avg)²

T = threshold value, on the y-axis

t_1-α = Student’s t when probability is (1-α)

t_1-β = Student’s t when probability is (1-β)

t_1-γ = Student’s t when probability is (1-γ)

UPL = upper prediction limit

x₀ = true concentration of zero

x_avg = mean of all true concentrations in the data set

x_D = true concentration equal to the detection limit

x_i = i^th true concentration

y = measurement response

yLPL,x_D = response corresponding to the point on LPL where x = x_D

yUPL,x₀ = response corresponding to the point on UPL where x = x₀ = 0

Mr. Coleman is an Applied Statistician, Alcoa Technical Center, MST-C, 100 Technical Dr., Alcoa Center, PA 15069, U.S.A.; e-mail: [email protected]. Ms. Vanatta is an Analytical Chemist, Air Liquide-Balazs™ Analytical Services, 13546 N. Central Expressway, Dallas, TX 75243-1108, U.S.A.; tel.: 972-995-7541; fax: 972-995-3204; e-mail: [email protected].