Introducing Tensormesh Platform
Reuse more of the GPU work you’ve already paid for.
Before a model can answer, it processes your prompt: the instructions, documents, and conversation so far. This step, called prefill, builds an internal record the model uses as it writes its response. That record is the key-value cache, or KV cache. For long prompts, rebuilding it can be a substantial part of the GPU work.
Today we’re launching Tensormesh Platform for teams serving open-weight models on their own infrastructure. It manages that cached work across memory, storage, and nodes, so more requests can reuse it.
Keep reusable work within reach
vLLM already reuses matching prompt prefixes, the identical beginning of a prompt. The next challenge is keeping useful cache available as workloads grow beyond GPU memory and requests reach different nodes.
Tensormesh Platform manages three tiers: GPU memory for active use, host memory for the hot set, and filesystem-backed storage for a larger cache. State saved outside an inference pod can be reused after that pod restarts. Our platform’s peer-to-peer capabilities, which we call cross-node sharing, let a node retrieve compatible cached work from another node instead of recomputing it.
That makes storage part of inference capacity planning. You can retain more reusable work without keeping it all in expensive GPU memory, with one management layer across tiers.
Conceptual view: cached work can be reused across memory tiers and nodes, with filesystem-backed storage providing a larger retained cache. Storage topology varies by deployment; only saved, compatible state can be reused after a pod restart.
Reuse beyond the start of a prompt
Prefix caching helps when requests share the same beginning. But if a document moves later in a prompt or follows different content, that shared prefix can end before the document begins.
Tensormesh Platform adds non-prefix reuse through CacheBlend, which our research team published at EuroSys 2025. It combines cached document state with selective recomputation to account for the surrounding context. This extends reuse in retrieval-augmented generation, or RAG, where an application looks up documents before asking the model to answer.
When document A follows different content, CacheBlend can reuse its cached state with selective recomputation. The diagram shows where reuse is possible, not that every token avoids computation.
Built on LMCache, operated in Kubernetes
We’re the team behind LMCache, the open-source KV caching engine with more than 10,000 GitHub stars. LMCache remains open source. Tensormesh Platform adds the enterprise management layer for teams running inference at scale in production.
Helm handles installation. A web UI and CLI support ongoing operations, alongside observability and multi-tenancy. The platform also supports prefill/decode disaggregation: processing prompts and generating output on separate workers.
Tensormesh Platform runs with vLLM on Kubernetes with NVIDIA or supported AMD Instinct GPUs through ROCm, and filesystem-backed storage. Non-prefix caching supports dense models, which use all their model parameters for each token. SGLang support is next.
The cache runs alongside the models you serve in your own environment. It doesn’t attach to a model accessed only through a hosted provider’s API.
Get started with Tensormesh Platform
Talk to an engineer about your workload to identify repeated context, discuss deployment requirements, and assess potential savings.
Read the getting-started guide for cluster prerequisites and installation options.
