ML Foundations
Confusion matrix axes: why accuracy cannot catch a transpose
Keep true labels on rows and predictions on columns, then verify precision and recall with an asymmetric six-sample fixture.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
Write the axis contract before calculating metrics
In the scikit-learn convention, a confusion matrix cell counts samples whose true class is the row label and whose predicted class is the column label. Keep one explicit label order for both axes. The API accepts a labels argument for that order; without it, the labels observed in either input are sorted. [1]
A transpose can look plausible because the diagonal and total count stay unchanged. Their ratio, ordinary accuracy for this unweighted single-label example, therefore stays unchanged too. Checking only accuracy cannot detect that the meanings of the two axes have been reversed. Use an asymmetric fixture and inspect row and column totals as well.
References: [1]
Check a case where precision and recall differ
The example orders classes as cat, dog and bird. Three samples are cats, but only one is predicted as cat. Another sample is a bird predicted as cat. Cat precision is therefore one correct cat prediction out of two cat predictions, while cat recall is one recovered cat out of three actual cats. This deliberately asymmetric case exposes an axis swap.
Precision divides true positives by true positives plus false positives; recall divides them by true positives plus false negatives. Under the axis contract above, these denominators are the class column total and class row total respectively. The documented support is the true-label count, so it agrees with the row total. [2]
Run python examples/confusion_matrix_axes.py. This is a small standard-library implementation of the counts, not a call to scikit-learn. Exact fractions keep rounding out of the checks. Assertions verify the full matrix, both margins, all per-class ratios, unchanged accuracy after transposition, and rejection of mismatched lengths, unknown labels and duplicate class names.
Executed with Python 3.12.14 using only the standard library. Exact stdout and source-byte SHA-256 are preserved in private evidence.
"""Hand-check a single-label confusion matrix and its precision/recall axes."""
from fractions import Fraction
def counts(truth, predictions, labels):
if not labels or len(set(labels)) != len(labels):
raise ValueError('labels must be nonempty and unique')
positions = {label: i for i, label in enumerate(labels)}
matrix = [[0 for _ in labels] for _ in labels]
for actual, predicted in zip(truth, predictions, strict=True):
if actual not in positions or predicted not in positions:
raise ValueError('unknown label')
matrix[positions[actual]][positions[predicted]] += 1
return matrix
def ratio(numerator, denominator):
return None if denominator == 0 else Fraction(numerator, denominator)
labels = ['cat', 'dog', 'bird']
truth = ['cat', 'cat', 'cat', 'dog', 'dog', 'bird']
predictions = ['cat', 'dog', 'dog', 'dog', 'dog', 'cat']
matrix = counts(truth, predictions, labels)
assert matrix == [[1, 2, 0], [0, 2, 0], [1, 0, 0]]
rows = [sum(row) for row in matrix]
columns = [sum(row[j] for row in matrix) for j in range(len(labels))]
assert rows == [3, 2, 1] and columns == [2, 4, 0]
assert sum(rows) == len(truth)
precision = [ratio(matrix[i][i], columns[i]) for i in range(len(labels))]
recall = [ratio(matrix[i][i], rows[i]) for i in range(len(labels))]
assert precision == [Fraction(1, 2), Fraction(1, 2), None]
assert recall == [Fraction(1, 3), Fraction(1), Fraction(0)]
transposed = [list(column) for column in zip(*matrix)]
accuracy = ratio(sum(matrix[i][i] for i in range(len(labels))), sum(rows))
wrong_accuracy = ratio(sum(transposed[i][i] for i in range(len(labels))), sum(map(sum, transposed)))
assert accuracy == wrong_accuracy == Fraction(1, 2)
print('rows=true, columns=predicted:', matrix)
print('true support:', rows)
print('predicted counts:', columns)
print('precision:', [str(x) for x in precision])
print('recall:', [str(x) for x in recall])
print('accuracy before/after transpose:', str(accuracy), str(wrong_accuracy))
rejected = 0
for actual, predicted, order in [(['cat'], [], labels), (['fox'], ['cat'], labels), (truth, predictions, ['cat', 'cat'])]:
try:
counts(actual, predicted, order)
except ValueError:
rejected += 1
assert rejected == 3
print('invalid fixtures rejected:', rejected)
Recorded output
rows=true, columns=predicted: [[1, 2, 0], [0, 2, 0], [1, 0, 0]] true support: [3, 2, 1] predicted counts: [2, 4, 0] precision: ['1/2', '1/2', 'None'] recall: ['1/3', '1', '0'] accuracy before/after transpose: 1/2 1/2 invalid fixtures rejected: 3
Keep undefined precision visible
No sample in this fixture is predicted as bird, so bird precision has a zero denominator. The helper returns None to make that undefined ratio visible. Bird recall is nevertheless defined and equals zero: there is one actual bird, and it was missed. None is this example's reporting policy, not a claim about the scikit-learn default.
When reviewing a metrics export, retain the label order, axis names, raw counts and denominator policy alongside the ratios. Here the matrix is dense, unweighted and single-label, with every observed class included. The helper intentionally rejects unknown labels rather than selecting a subset; it does not implement sample weights, multilabel targets or averaged scores. The six invented records test arithmetic and orientation only; they provide no evidence about a trained model's quality.
References: [2]