https://www.jonam.io/tags/kv-cache/Recent content in Kv-Cache on jonam'LogHugo -- gohugo.ioen© 2026 Manoj. All Rights Reserved.Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/product-ideas/docvault/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/product-ideas/docvault/Compute any document’s context once, serve it to every user forever.https://www.jonam.io/journal/inference-engineering/research-topics/position-invariant-document-kv-cache/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/research-topics/position-invariant-document-kv-cache/Can document KV states be cached independent of prompt position and reused across RAG queries?https://www.jonam.io/journal/inference-engineering/product-ideas/prefillx/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/product-ideas/prefillx/Cut TTFT for long-context document applications by precomputing and repairing reusable KV states.https://www.jonam.io/journal/inference-engineering/research-topics/temporal-turboquant-kv-tiering/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/research-topics/temporal-turboquant-kv-tiering/Recent tokens stay high precision, older tokens degrade to INT4 or INT2, and TurboQuant makes the low-bit tiers less painful.https://www.jonam.io/journal/inference-engineering/research-topics/speculative-prefill/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/research-topics/speculative-prefill/Speculative decoding is common; this asks whether speculation can reduce long-prompt prefill latency.https://www.jonam.io/journal/inference-engineering/research-topics/slo-aware-kv-cache-tiering/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/research-topics/slo-aware-kv-cache-tiering/Premium users get hot KV blocks; batch users spill to cheaper memory tiers.https://www.jonam.io/journal/inference-engineering/product-ideas/convocache/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/product-ideas/convocache/Store and rehydrate the conversation state that actually mattered.https://www.jonam.io/journal/inference-engineering/research-topics/Mon, 18 May 2026 00:00:00 +0000https://www.jonam.io/journal/inference-engineering/research-topics/Novel and practical research directions around KV cache compression, scheduling, speculation, quantization, and hardware-aware serving.