Cache LLM responses by exact match or semantic similarity.

The Semantic Cache module routes repeated or semantically similar prompts through a cache layer on your Knotbind Gateway, reducing cost and latency for common queries.

When to use it

Enable Semantic Cache when your application issues many near-duplicate or closely related requests — FAQs, product lookups, classification prompts, documentation Q&A — where the exact answer is likely to repeat. The cache sits in front of your configured providers and applies to all keys in the workspace that have the module enabled.

Configuration

Using the cache key header

By default the cache key is derived from the request body. To name an entry yourself, send an x-knotbind-cache-key header on your request to /v1/chat/completions. Requests that share a cache key can be served from the same entry. This header is only forwarded when Semantic Cache is enabled for your workspace; otherwise the Gateway ignores it.

curl https://YOUR-SITE/v1/chat/completions \
  -H "Authorization: Bearer kb_live_..." \
  -H "content-type: application/json" \
  -H "x-knotbind-cache-key: faq-returns-policy" \
  -d '{"model":"groq/openai/gpt-oss-20b",
       "messages":[{"role":"user","content":"What is your return policy?"}]}'

Two things decide whether a request is cached at all:

• A cache key must be present — either your x-knotbind-cache-key header or the Default cache key you set in the dashboard. Without one, the request goes straight to the provider.
• The conversation must stay within the Conversation history threshold. Longer conversations are not cached, because their context makes a repeat answer unlikely.

Direct matching runs first. Only when it misses does semantic similarity run, and only in Direct + Semantic mode. Writes are asynchronous: on a miss you get the provider's answer immediately and the entry is stored in the background, so a first request never waits on the cache. Entries survive a Gateway restart and expire on their TTL.

Status in the dashboard

Gateway → Gateway modules reports whether Semantic Cache is loaded and, if it could not be applied, a short reason. Per-key request counts, latency and spend remain on the Gateway page.

Limits

Semantic mode needs an embedding provider configured on your Gateway. Direct-only mode needs none, which makes it the cheapest way to start: exact repeats are served from cache with no embedding call at all.