Inference
Why dict.get can run inference even on a cache hit
Python evaluates a fallback call before dict.get chooses a value. Use an explicit miss branch to avoid unnecessary computation and retain false-valued results.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
The fallback expression runs before the lookup method
cache.get(key, predict(key)) does not defer prediction until a cache miss. Python evaluates call arguments before attempting the call, so predict(key) runs first. Only afterward does get choose between the cached value and the already-computed default. The language reference documents argument evaluation, while the dictionary reference specifies which value get returns. [1, 2]
This distinction can be invisible if a test checks only the returned prediction. In the fixture, the cached answer is 42 and the returned answer is correctly 42, yet the fallback call counter increases. In an inference service that same expression could perform unwanted model work on a hit. The example counts calls using a small stand-in function; it measures no model latency.
Put computation and insertion in the missing-key branch
The get_or_compute helper first checks whether the key exists and returns the stored value when it does. Otherwise it calls the supplied compute function, stores the successful result, and returns it. Dictionary membership tests key presence, independently of the value stored under that key. [1]
Run python examples/lazy_inference_cache.py. The first assertion reproduces the unnecessary fallback evaluation. Subsequent assertions show no computation for a known key and exactly one computation across two requests for the same initially missing key. The new value is explicitly checked in the cache as well as in both return values.
The final case supplies a function that raises a synthetic RuntimeError. The helper lets the exception propagate and leaves the key absent because insertion follows successful computation. That ordering is a deliberate behavior of this helper, demonstrated by the executed assertion.
Executed with Python 3.12.14 using no third-party dependencies. Exact stdout and source-byte SHA-256 are preserved in private evidence.
"""Count fallback evaluations on cache hits without running a real model."""
def get_or_compute(cache, key, compute):
if key in cache:
return cache[key]
value = compute(key)
cache[key] = value
return value
calls = []
def predict(key):
calls.append(key)
return len(key)
cache = {'known': 42}
value = cache.get('known', predict('known'))
assert value == 42 and calls == ['known']
print('dict.get cached value:', value)
print('fallback calls despite hit:', len(calls))
calls.clear()
assert get_or_compute(cache, 'known', predict) == 42
assert calls == []
first = get_or_compute(cache, 'fresh', predict)
second = get_or_compute(cache, 'fresh', predict)
assert first == second == 5 and calls == ['fresh']
assert cache['fresh'] == 5
print('computed once for repeated missing key:', calls)
calls.clear()
for stored in (None, False, 0, ''):
cache['special'] = stored
result = get_or_compute(cache, 'special', predict)
assert result is stored
assert calls == []
print('false-valued cached results retained:', 4)
# A failed computation must not create a successful cache entry.
def fail(key):
raise RuntimeError('synthetic failure')
try:
get_or_compute(cache, 'failed', fail)
except RuntimeError:
assert 'failed' not in cache
print('failed computation cached:', False)
else:
raise AssertionError('expected the computation error to propagate')
Recorded output
dict.get cached value: 42 fallback calls despite hit: 1 computed once for repeated missing key: ['fresh'] false-valued cached results retained: 4 failed computation cached: False
Treat presence separately from truth and concurrency
A cached None, False, zero or empty string may be a legitimate result. The fixture checks that all four are returned without recomputation. A shortcut based on cache.get(key) or compute(key) would treat these false-valued results as missing. Python documents those false values and the short-circuit behavior of or. [1]
This helper is intentionally limited to sequential access to an ordinary dictionary. Its membership check, computation and insertion do not form an atomic operation, so concurrent callers can still compute the same missing key. Cache keys must also represent the input and model configuration whose result is being reused; eviction, invalidation and shared-process coordination require separate design decisions.
References: [1]