Blog & resources

Notes from the lab.

Engineering notes, launch news, and plain-English explainers on durable long-context AI — method, measurements, and the occasional argument for changing how a model is served instead of changing the model.

Featured · durable context

The KV cache stops being a GPU problem

In a standard stack the KV cache is scratch space — built during a request, sized by that request, thrown away at the end, and competing with the model for the same scarce memory. Make it durable instead and long context stops being a memory-allocation question.

Michael Beck· August 6, 2026

announcements
Your model, unmodified: durable long context without touching the weights
lab358 serves the open-weight model you already use exactly as its authors published it, and makes long context durable in the serving layer — index a document once, reuse it on every call. Run it managed on lab358 Cloud, or self-hosted in your own AWS account.
Michael Beck· August 5, 2026
inference economics
Inference is where the AI budget goes
Training gets the headlines, but the money is in inference — the large majority of AI compute spend, paid again on every request. And the workloads growing fastest are exactly the ones standard attention punishes most.
Michael Beck· July 28, 2026
durable context
Index once, reuse forever: durable long-context AI
Most AI stacks re-read your documents from scratch on every call. lab358 indexes a document once and reuses that context across every chat, agent, and API request — no repeated prefill, no fixed context ceiling.
Michael Beck· July 20, 2026

Loading…

Notes from the lab.

The KV cache stops being a GPU problem

Your model, unmodified: durable long context without touching the weights

Inference is where the AI budget goes

Index once, reuse forever: durable long-context AI

Notes from the lab.

The KV cache stops being a GPU problem

Your model, unmodified: durable long context without touching the weights

Inference is where the AI budget goes

Index once, reuse forever: durable long-context AI