ML Foundations
Avoid raw-moment subtraction when checking feature variance
A large feature offset can make mean(x*x) - mean(x)**2 report a negative variance. Compare centered reduction with exact evidence and distinguish prior precision loss.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
A negative result can come from the calculation
A feature-quality check may use mean(x*x) minus mean(x) squared to estimate variance. In exact arithmetic this equals the population mean of squared deviations. In floating-point arithmetic, subtracting two very large, nearly equal moments can lose the small difference that carries the feature spread. A negative result should prompt inspection of the calculation before marking the feature constant or invalid.
The fixture uses four consecutive integer values starting at one trillion. Their deviations from the mean are minus 1.5, minus 0.5, 0.5 and 1.5, so their population variance is 1.25. The executed float64 raw-moment expression instead produces minus 134217728. Python statistics.pvariance applied to Fraction inputs independently returns the exact rational value 5/4; its documentation explicitly supports Fraction data. [2]
References: [2]
Compute spread around the mean
NumPy documents var in terms of squared deviations from the mean, divided by N minus ddof. The example calls np.var with dtype=float64 and ddof=0 and obtains 1.25. Subtracting the common trillion-unit offset first gives the same result in this fixture. These checks isolate the arithmetic without fitting a model or changing the feature distribution. [1]
Run python examples/feature_variance_cancellation.py. Assertions compare the raw-moment result against the rational reference, verify both centered calculations and check zero population variance for singleton and constant inputs. The example also uses ddof=1 and obtains approximately 1.666667. That denominator change is a different variance definition, not a numerical repair. [1]
For an existing array, use a variance routine with an explicit denominator and suitable computation dtype rather than reproducing the raw-moment subtraction. The example measures one bounded failure case; it does not establish exactness for arbitrary magnitudes, input orders or floating-point datasets.
Executed with Python 3.12.14 and NumPy 2.3.5 on CPU, including a standard-library rational reference. Exact stdout and source-byte SHA-256 are preserved in private evidence.
"""Compare raw-moment variance, centered variance and prior quantization."""
from fractions import Fraction
from statistics import pvariance
import numpy as np
integers = [10**12 + i for i in range(4)]
exact = pvariance([Fraction(x) for x in integers])
assert exact == Fraction(5, 4)
values = np.array(integers, dtype=np.float64)
raw_moments = np.mean(values * values) - np.mean(values) ** 2
centered = np.var(values, dtype=np.float64, ddof=0)
shifted = np.var(values - 10**12, dtype=np.float64, ddof=0)
assert raw_moments != float(exact)
assert centered == shifted == float(exact)
assert np.var(np.array([7.0]), ddof=0) == 0.0
assert np.var(np.full(4, 10**12, dtype=np.float64), ddof=0) == 0.0
sample = np.var(values, dtype=np.float64, ddof=1)
assert np.isclose(sample, 5.0 / 3.0)
# A wider accumulator cannot recover distinctions lost in stored float32 data.
quantized = values.astype(np.float32)
assert np.unique(quantized).size == 1
after_widening = np.var(quantized, dtype=np.float64, ddof=0)
assert after_widening == 0.0
print("exact population variance:", str(exact))
print("raw-moment subtraction:", float(raw_moments))
print("centered float64 variance:", float(centered))
print("shifted-data variance:", float(shifted))
print("float32 distinct stored values:", np.unique(quantized).size)
print("float64 accumulation after float32 storage:", float(after_widening))
print("sample variance (ddof=1):", format(float(sample), ".6f"))
Recorded output
exact population variance: 5/4 raw-moment subtraction: -134217728.0 centered float64 variance: 1.25 shifted-data variance: 1.25 float32 distinct stored values: 1 float64 accumulation after float32 storage: 0.0 sample variance (ddof=1): 1.666667
References: [1]
Separate accumulator precision from stored precision
The same four observations collapse to one distinct stored value after conversion to float32. Computing their variance with a float64 accumulator then returns zero. Widening the accumulator cannot reconstruct differences already removed by that conversion. The fixture explicitly counts distinct stored values to make this second failure visible.
Choose storage precision before discarding feature detail. NumPy notes that floating-point variance normally uses the input floating type and that a higher-precision dtype can improve calculation accuracy. That option addresses computation precision, while the demonstrated conversion loss has already changed the input data. Empty inputs, non-finite values and extreme-range overflow need their own handling; none is evaluated here. [1]
References: [1]