03 Tools03-01
Confusion matrix
From the number of cases, the prevalence and a model’s sensitivity and specificity: the four counts, eight rates, and what the errors cost at a price per false alarm and per miss.
03-01 · Classification · confusion matrixBinary · one operating point · costs per error
Inputs
Costs
Limit: whole cases rounded from the rates, one operating point. The threshold curve behind it is not modelled.
Errors cost €4,400 across 500 cases: 91% from the 20 churners missed, 9% from the 40 false alarms.
Fig. 03-01 · Confusion matrixn = 500 · rows actual · columns predicted
Predicted positive
Predicted negative
Actual positive
TP80churners flagged
FN20churners missed
Actual negative
FP40loyal customers flagged
TN360loyal customers cleared
Accuracy
88.0%
(TP + TN) / n
Precision
66.7%
TP / (TP + FP)
Recall
80.0%
TP / (TP + FN) · sensitivity
Specificity
90.0%
TN / (TN + FP)
NPV
94.7%
TN / (TN + FN)
F1
0.727
harmonic mean of P and R
False positive rate
10.0%
FP / (FP + TN)
False negative rate
20.0%
FN / (FN + TP)
Total cost
€4,400
FP · c_FP + FN · c_FN
Cost per case
€8.80
total cost / n
Rule
cost = FP · c_FP + FN · c_FN
cost = 40 · €10 + 20 · €200 = €400 + €4,000 = €4,400 · €8.80 per case