ML Foundations
Average precision is not trapezoidal precision-recall area
Compute recall-weighted precision at distinct score thresholds, keep ties together, and avoid relabeling a trapezoidal PR integral as average precision.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
Name the summary you actually calculate
A classifier evaluation can report two different numbers from the same precision-recall points. Scikit-learn defines average precision as a sum of precision values weighted by the increase in recall at successive thresholds. It explicitly distinguishes this non-interpolated calculation from trapezoidal integration, which joins points with straight lines. [1]
The fixture ranks four observations by scores 0.9, 0.8, 0.7 and 0.6; their labels are positive, negative, positive and negative. Recall increases twice, by one half each time. Precision at those increases is one and two thirds, giving average precision 5/6, or approximately 0.833333. Trapezoidal integration over the same constructed curve gives 19/24, approximately 0.791667. Neither calculation error nor different predictions explains the gap: they summarize the curve differently.
References: [1]
Build one operating point per distinct score
The NumPy helper sorts scores from highest to lowest, accumulates true positives and emits a curve point only after the last row of each tied-score group. Under a score-at-least-threshold rule, equal scores enter together. Scikit-learn documents curve thresholds in terms of unique scores and that threshold comparison. [2]
Run python examples/average_precision_steps.py. The code adds the no-predictions endpoint with recall zero and precision one, then traverses toward recall one. It computes AP as the sum of recall increments multiplied by the precision after each increment. For a positive and negative observation with identical scores, both input row orders give AP 0.5. Processing the tied rows as separate attainable thresholds would describe a different rule.
Assertions check the hand-derived four-row result, both tied-row permutations, an all-positive case and five invalid or unsupported inputs. A separate local reference check compared all four valid fixtures with scikit-learn 1.8.0: both AP and the complete precision-recall arrays matched after reversing curve direction.
Executed with Python 3.12.14 and NumPy 2.3.5 on CPU. Separate scikit-learn 1.8.0 reference comparisons passed. Exact example stdout and SHA-256 are preserved in private evidence.
"""Compare step-weighted AP and trapezoidal precision-recall area."""
import numpy as np
def binary_curve(labels, scores):
labels = np.asarray(labels)
scores = np.asarray(scores, dtype=np.float64)
if labels.ndim != 1 or scores.ndim != 1 or labels.shape != scores.shape or not labels.size:
raise ValueError("nonempty aligned one-dimensional inputs required")
if not np.all(np.isin(labels, [0, 1])) or not np.all(np.isfinite(scores)):
raise ValueError("binary labels and finite scores required")
positives = int(np.sum(labels))
if positives == 0:
raise ValueError("this teaching helper requires a positive label")
order = np.argsort(-scores, kind="stable")
ranked_scores = scores[order]
# One attainable threshold per distinct score; tied rows enter together.
ends = np.r_[np.flatnonzero(ranked_scores[:-1] != ranked_scores[1:]), labels.size - 1]
true_positives = np.cumsum(labels[order], dtype=np.int64)[ends]
precision = np.r_[1.0, true_positives / (ends + 1)]
recall = np.r_[0.0, true_positives / positives]
ap = float(np.sum(np.diff(recall) * precision[1:]))
return recall, precision, ap
labels = [1, 0, 1, 0]
scores = [0.9, 0.8, 0.7, 0.6]
recall, precision, ap = binary_curve(labels, scores)
trapezoid = float(np.trapezoid(precision, recall))
assert np.allclose(recall, [0, 0.5, 0.5, 1, 1])
assert np.allclose(precision, [1, 1, 0.5, 2 / 3, 0.5])
assert np.isclose(ap, 5 / 6) and np.isclose(trapezoid, 19 / 24)
tie_a = binary_curve([1, 0], [0.8, 0.8])[2]
tie_b = binary_curve([0, 1], [0.8, 0.8])[2]
assert tie_a == tie_b == 0.5
assert binary_curve([1, 1], [0.8, 0.2])[2] == 1.0
rejected = 0
for bad_labels, bad_scores in (([], []), ([1], [0.1, 0.2]), ([2], [0.1]), ([1], [np.nan]), ([0, 0], [0.1, 0.2])):
try:
binary_curve(bad_labels, bad_scores)
except ValueError:
rejected += 1
assert rejected == 5
print("recall in threshold traversal:", recall.tolist())
print("step-weighted average precision:", format(ap, ".6f"))
print("trapezoidal PR area:", format(trapezoid, ".6f"))
print("AP for both tied-row orders:", tie_a, tie_b)
print("all-positive fixture AP:", binary_curve([1, 1], [0.8, 0.2])[2])
print("invalid/unsupported fixtures rejected:", rejected)
Recorded output
recall in threshold traversal: [0.0, 0.5, 0.5, 1.0, 1.0] step-weighted average precision: 0.833333 trapezoidal PR area: 0.791667 AP for both tied-row orders: 0.5 0.5 all-positive fixture AP: 1.0 invalid/unsupported fixtures rejected: 5
Keep direction and scope visible
Scikit-learn returns recall in decreasing order, ending with recall zero and precision one. This example uses the reverse traversal so recall increments are nonnegative. If integrating a curve from another API, inspect its direction and endpoint convention before comparing values. [2]
The helper supports finite, unweighted binary scores with positive label one and at least one positive observation. It deliberately rejects all-negative input rather than reproducing a library-specific fallback. It does not implement sample weights, multiclass averaging or alternate interpolation conventions. Use the established metric implementation for a full evaluation; this small implementation makes the calculation and tie behavior inspectable.