announcements
lab358 serves the open-weight model you already use exactly as its authors published it, and makes long context durable in the serving layer — index a document once, reuse it on every call. Run it managed on lab358 Cloud, or self-hosted in your own AWS account.
lab358 serves the open-weight model you already use exactly as its authors published it, and makes long context durable in the serving layer — index a document once, reuse it on every call. Run it managed on lab358 Cloud, or self-hosted in your own AWS account.
Inference, not training, is where AI budgets go: roughly 60–80% of total AI compute spend, in a market estimated at $106B in 2025 and projected to reach $255B by 2030. And the workloads growing fastest — contracts, case law, patient histories, filings, codebases — are exactly the ones a standard stack punishes, because every call re-reads the whole document from scratch and the cache it builds devours GPU memory.
The usual escape routes all give something up. State-space conversions compress history into a fixed-size state and lose long-range recall. Cache-eviction tricks discard context at inference time, trading quality for speed. Retraining a frontier model on a new architecture costs more than most companies will ever spend on inference.
And modifying the model has a tax of its own that rarely gets counted. The moment a checkpoint is converted or retrained, it stops being the model anyone else has benchmarked. Your evaluations don't transfer. The vendor's quality claims are the only ones available, and you have to take them. Every upstream release restarts the process.
So we took a different route: don't change the model — change how it's served.
You bring an open-weight model you already use. We run it unmodified — the same weights, the same tokenizer, the same behaviour its authors published and the community measured. There is no conversion step, no retraining, and no fine-tune.
What we build is everything around it. A document is processed once into a stored form the model can read back later, and retrieved on demand with approximate nearest-neighbor search instead of being pushed through the prompt again on every call. That single change is what makes long context durable:
In practice, three steps:
It reads like a limitation and it is closer to the opposite:
Numbers here will be published measured, not projected, and we would rather be late than approximate. The serving-side quality and cost measurements are in progress; this section will carry them when they exist, with the method and eval set named so they're reproducible.
What can be said today without waiting: the model's own quality is a matter of public record, because it is the public model.
The following are the intended effects of the design above; per-workload benchmarks are in progress and will be published when measured.
lab358 ships two ways — the same model and the same OpenAI-compatible API either way — so you pick where it runs:
If you run long-context workloads, join the lab358 Cloud waitlist — or, to self-host in your own AWS account, or to ask about the model you want on the supported list, talk to us.
Investors: hello@lab358.ai.