Inference
State the quantile method when reporting p95 latency
The same five latency samples yield p95 values of 88 ms and 100 ms under two NumPy methods. Keep the estimator and probability scale in the report.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
A percentile label does not identify the estimator
Two reports can use exactly the same latency samples and still disagree on p95 because their quantile methods differ. NumPy exposes a method argument; its default is linear interpolation. A result can therefore fall between measured values rather than equal an observed latency. [1]
The fixture uses five invented request durations in milliseconds: 10, 20, 30, 40 and 100. It is an arithmetic example, not a benchmark of any inference service. Linear p95 is 88 ms, while inverted_cdf returns 100 ms. The changed method alone explains this difference.
References: [1]
Verify both estimates against the sorted sample
Run python examples/latency_quantile_method.py. For linear interpolation, the zero-based position is 0.95 times four, or 3.8. Moving 80 percent of the way from the fourth value, 40, to the fifth value, 100, gives 88. The example independently computes that interpolation and checks it against NumPy.
For the unweighted inverted empirical-CDF method at a positive probability, select the sorted observation at one-based rank ceil(n times q). Here ceil(5 times 0.95) is five, so the result is 100. The code checks this rank formula as well. This method corresponds to the empirical-distribution interpretation described in NumPy's notes. [1]
Only four of the five samples are at or below 88, while all five are at or below 100. An interpolated p95 estimate need not place at least 95 percent of this finite sample below its cutoff. The fixture asserts these actual counts and confirms that 88 was never observed.
NumPy percentile takes a percentage between zero and 100; quantile takes a probability between zero and one. The example verifies that percentile(samples, 95, method='linear') matches quantile(samples, 0.95, method='linear'). Passing 0.95 to percentile would request the 0.95th percentile instead. [2]
Executed with Python 3.12.14 and NumPy 2.3.5. Manual interpolation and rank calculations check actual NumPy results. Exact stdout and source-byte SHA-256 are preserved in private evidence.
"""A fixed latency fixture, not a service benchmark."""
from math import ceil
import numpy as np
def checked_latencies(values):
x = np.asarray(values, dtype=float)
if x.ndim != 1 or x.size == 0 or not np.isfinite(x).all() or (x < 0).any():
raise ValueError("nonempty finite nonnegative latency vector required")
return x
def main():
samples = checked_latencies([10, 20, 30, 40, 100])
q = 0.95
linear = float(np.quantile(samples, q, method="linear"))
empirical = float(np.quantile(samples, q, method="inverted_cdf"))
# Independent formulas for this fixture, with 0 < q <= 1.
sorted_samples = sorted(samples.tolist())
h = (len(sorted_samples) - 1) * q
j = int(h)
manual_linear = sorted_samples[j] + (h - j) * (sorted_samples[j + 1] - sorted_samples[j])
manual_empirical = sorted_samples[ceil(len(sorted_samples) * q) - 1]
assert np.isclose(linear, 88.0) and np.isclose(linear, manual_linear)
assert empirical == manual_empirical == 100.0
below_linear = int(np.count_nonzero(samples <= linear))
below_empirical = int(np.count_nonzero(samples <= empirical))
assert (below_linear, below_empirical) == (4, 5)
assert linear not in samples
assert np.isclose(np.percentile(samples, 95, method="linear"), linear)
assert np.isclose(np.quantile(samples[::-1], q, method="linear"), linear)
assert float(np.quantile(samples, 0.0095, method="linear")) < 11
rejected = 0
for bad in ([], [10, float("nan")], [-1, 10], [[10, 20]]):
try:
checked_latencies(bad)
except ValueError:
rejected += 1
assert rejected == 4
print("latency fixture milliseconds:", samples.astype(int).tolist())
print(f"p95 linear / inverted_cdf: {linear:.1f} {empirical:.1f}")
print("observations at or below cutoffs:", below_linear, below_empirical, "of", samples.size)
print("linear cutoff is an observed latency:", bool(linear in samples))
print("percentile(95) matches quantile(0.95):", bool(np.isclose(np.percentile(samples, 95, method="linear"), linear)))
print("invalid latency fixtures rejected:", rejected)
if __name__ == "__main__":
main()
Recorded output
latency fixture milliseconds: [10, 20, 30, 40, 100] p95 linear / inverted_cdf: 88.0 100.0 observations at or below cutoffs: 4 5 of 5 linear cutoff is an observed latency: False percentile(95) matches quantile(0.95): True invalid latency fixtures rejected: 4
Keep reporting choices separate from population certainty
Neither estimate is automatically a bug or a universally superior choice. State the method, sample count, units and sampling window when comparing reports. If a requirement refers to the fraction of observed requests below a cutoff, check that fraction explicitly instead of assuming an interpolated percentile enforces it.
The helper accepts one nonempty vector of finite nonnegative durations and rejects four invalid fixtures. It does not silently drop failed or missing measurements. The example uses equal weights and complete observations; weighted requests, censored timeouts and streaming percentile approximations require separate definitions. Five invented observations do not establish a population tail percentile, a confidence bound or a service-level guarantee.