Estimate KV-cache memory before choosing a context length
Calculate the dense cache payload from layers, KV heads, token count and dtype, and see why the answer is not a GPU-fit guarantee.
2026-10-21
FIELD NOTES
Inference
Memory, latency, and the practical work of serving models.