Inference
Clip quantized codes before narrowing them to int8
A narrowing cast does not implement saturation. Make the rounding, scale and clipping interval explicit in a small symmetric quantization check.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
Define the integer contract before converting
For a simple symmetric quantizer, form an integer code by rounding the input-to-scale ratio, then limiting it to a chosen code interval. Reconstruction multiplies that integer by the same scale. This example chooses zero point zero and codes from -127 through 127. It deliberately leaves the int8 value -128 unused; the interval is part of this teaching contract, not a universal backend requirement.
A narrowing cast is a conversion operation. NumPy documents astype with unsafe casting as its default; it does not supply the clipping operation in this recipe. With scale 1/127, inputs 1.2 and -1.2 round to codes 152 and -152. Narrowing those small integers directly from int64 to int8 yields -104 and 104 in the executed fixture, reversing their signs. [3]
References: [3]
Round, clip, then store
The helper first validates finite inputs and a finite positive scalar scale. It divides in float64, rounds with np.rint, clips to the declared interval and only then converts to int8. NumPy documents clip as replacing values outside an interval with its edges. Its rint function rounds halfway cases to the nearest even integer. [1, 2]
Run python examples/saturating_int8_quantization.py. The four-value fixture produces codes 127, -127, zero and 64, reconstructing approximately 1, -1, zero and 0.503937. Assertions check the dtype, endpoints, zero, empty input and positive and negative rounding ties. Four invalid fixtures cover zero, negative and infinite scales plus a NaN input.
The code enables floating-point exceptions while scaling and rounding. Extremely large input-to-scale ratios can therefore fail before clipping; accepting all finite inputs regardless of arithmetic range is not promised. The demonstrated narrowing outputs are runtime evidence, not a portable recipe for intentional overflow.
Executed with Python 3.12.14 and NumPy 2.3.5 on bounded synthetic CPU inputs. Exact stdout and source-byte SHA-256 are preserved in private evidence.
"""Use an explicit symmetric saturation contract before narrowing to int8."""
import numpy as np
def quantize(values, scale):
values = np.asarray(values, dtype=np.float64)
if not np.isscalar(scale) or not np.isfinite(scale) or scale <= 0:
raise ValueError("a finite positive scalar scale is required")
if not np.all(np.isfinite(values)):
raise ValueError("finite values required")
with np.errstate(over="raise", divide="raise", invalid="raise"):
rounded = np.rint(values / scale)
return np.clip(rounded, -127, 127).astype(np.int8)
scale = 1.0 / 127
values = np.array([1.2, -1.2, 0.0, 0.5])
rounded = np.rint(values / scale)
# Narrowing is not a saturating operation; intermediate integers are small here.
naive = rounded.astype(np.int64).astype(np.int8)
encoded = quantize(values, scale)
decoded = encoded.astype(np.float64) * scale
assert rounded.tolist() == [152.0, -152.0, 0.0, 64.0]
assert naive.tolist() == [-104, 104, 0, 64]
assert encoded.dtype == np.int8 and encoded.tolist() == [127, -127, 0, 64]
assert np.allclose(decoded[:3], [1, -1, 0])
in_range = np.abs(values) <= 1.0
assert np.all(np.abs(decoded[in_range] - values[in_range]) <= scale / 2 + 1e-15)
assert np.all(np.abs(decoded[:2] - values[:2]) > scale / 2)
assert quantize([-1.0, 1.0], scale).tolist() == [-127, 127]
assert quantize([0.5, 1.5, -0.5, -1.5], 1.0).tolist() == [0, 2, 0, -2]
assert quantize([], scale).size == 0
rejected = 0
for x, s in (([0], 0), ([0], -1), ([0], np.inf), ([np.nan], scale)):
try:
quantize(x, s)
except ValueError:
rejected += 1
assert rejected == 4
print("rounded pre-cast codes:", rounded.tolist())
print("direct narrowing codes:", naive.tolist())
print("saturated int8 codes:", encoded.tolist())
print("reconstructed values:", np.round(decoded, 6).tolist())
print("outside representable interval:", int(np.count_nonzero(~in_range)))
print("nearest-even ties:", quantize([0.5, 1.5, -0.5, -1.5], 1.0).tolist())
print("invalid value/scale fixtures rejected:", rejected)
Recorded output
rounded pre-cast codes: [152.0, -152.0, 0.0, 64.0] direct narrowing codes: [-104, 104, 0, 64] saturated int8 codes: [127, -127, 0, 64] reconstructed values: [1.0, -1.0, 0.0, 0.503937] outside representable interval: 2 nearest-even ties: [0, 2, 0, -2] invalid value/scale fixtures rejected: 4
Keep clipping error separate from rounding error
With this scale, the reconstruction interval is minus one through one. The fixture checks that in-range values have error no larger than half a scale step, allowing a small numerical tolerance. The two out-of-range values instead lose 0.2 each through saturation. Clipping prevents invalid stored codes but cannot preserve values beyond the chosen range.
Count inputs outside the representable interval when evaluating a fixed scale. A correct clipping implementation does not establish that its calibration range is suitable. This example loads no weights, performs no quantized matrix multiplication and measures no inference accuracy or speed. Real model formats may use different scales, zero points, granularity, code ranges and rounding conventions.