ML Foundations
Why Python threshold callbacks can all use the final cutoff
Bind each loop value when building delayed classification callbacks, then test the callbacks after their builder returns.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
Construct first, call later to expose the binding
A threshold sweep can appear to create a separate classifier for each cutoff while every classifier actually uses the final cutoff. In a loop, lambda score: score >= threshold refers to the enclosing threshold binding. Calling those functions after construction reads the value left in that binding. Python's FAQ explains this delayed lookup and notes that ordinary nested functions have the same issue. [1]
The fixture builds callbacks for cutoffs 0.2, 0.5 and 0.8, then evaluates scores 0.1, 0.5 and 0.9 after the builder has returned. The broken callbacks all behave like the 0.8 cutoff. Distinct function objects do not establish that distinct settings were captured. The visible failure is three identical prediction rows when two of them should accept the middle score.
References: [1]
Bind the numeric cutoff during construction
The compact fix is lambda score, threshold=threshold: score >= threshold. The right-hand threshold is evaluated when that function is created and stored as its default. At invocation, the body uses the local parameter. The Python tutorial documents definition-time evaluation of default argument expressions. Here the captured values are immutable numbers. [2]
Run python examples/threshold_callback_binding.py. The file executes the broken builder and two corrected versions. Assertions compare their complete prediction matrices against a hand-derived expectation. It also checks equality at the middle cutoff, a reordered four-cutoff configuration and an empty configuration. Testing only the final cutoff would miss the defect, because that callback already gives the intended answer.
The scores are fixed synthetic inputs and the comparison includes equality. These checks establish callback behavior for the fixture; selecting an effective threshold still requires the appropriate evaluation data.
Executed with Python 3.12.14, using no third-party dependencies. Exact stdout and source-byte SHA-256 are preserved in private evidence.
"""Reproduce delayed threshold callbacks sharing the final loop binding."""
def make_bad(thresholds):
callbacks = []
for threshold in thresholds:
callbacks.append(lambda score: score >= threshold)
return callbacks
def make_bound(thresholds):
callbacks = []
for threshold in thresholds:
callbacks.append(lambda score, threshold=threshold: score >= threshold)
return callbacks
def make_one(threshold):
def classify(score):
return score >= threshold
return classify
thresholds = (0.2, 0.5, 0.8)
scores = (0.1, 0.5, 0.9)
bad = make_bad(thresholds)
bound = make_bound(thresholds)
factories = [make_one(threshold) for threshold in thresholds]
def predictions(callbacks):
return [[int(callback(score)) for score in scores] for callback in callbacks]
expected = [[0, 1, 1], [0, 1, 1], [0, 0, 1]]
assert predictions(bad) == [[0, 0, 1]] * 3
assert predictions(bound) == predictions(factories) == expected
assert [callback(0.5) for callback in bound] == [True, True, False]
# Reordering and changing the length must still bind each requested threshold.
other = (0.8, 0.2, 0.5, 0.9)
assert [callback(0.5) for callback in make_bound(other)] == [False, True, True, False]
assert make_bound(()) == []
try:
factories[0](0.5, 0.99)
except TypeError:
print('factory rejects a second positional argument: True')
else:
raise AssertionError('factory callback must accept one argument')
print('thresholds:', thresholds)
print('scores:', scores)
print('late-bound predictions:', predictions(bad))
print('default-bound predictions:', predictions(bound))
print('factory predictions:', predictions(factories))
print('reordered thresholds and empty configuration: passed')
Recorded output
factory rejects a second positional argument: True thresholds: (0.2, 0.5, 0.8) scores: (0.1, 0.5, 0.9) late-bound predictions: [[0, 0, 1], [0, 0, 1], [0, 0, 1]] default-bound predictions: [[0, 1, 1], [0, 1, 1], [0, 0, 1]] factory predictions: [[0, 1, 1], [0, 1, 1], [0, 0, 1]] reordered thresholds and empty configuration: passed
Use a factory when the callback must accept one argument
The default-parameter form exposes a second optional argument that a caller could override. The alternative make_one(threshold) factory creates a separate enclosing binding on each call and returns classify(score). The example verifies that its predictions match the default-bound version and that passing a second positional argument raises TypeError. This keeps the intended single-argument callback interface.
Default binding stores the object supplied at creation; it does not deep-copy mutable configuration. The tutorial warns that mutable defaults can retain changes across calls. If a callback needs a list or dictionary configuration, decide separately whether shared state or a copied snapshot is intended. The numeric-cutoff fixture avoids that additional ownership question. [2]
References: [2]