Semantic Cache

Cache LLM responses by meaning using Redis vector search. Similar questions return cached answers instantly instead of making expensive API calls.

How It Works

1. 1. User asks a question or makes an LLM request
2. The question is embedded into a vector using OpenAI text-embedding-3-small
3. Redis vector search finds semantically similar cached queries (cosine similarity > 0.80)
4. Cache hit: Return the cached response instantly (~100ms)
5. Cache miss: Pass through to the LLM, cache the response for future similar queries

Commands

Cache a query and response

Check cache for a similar query

Cache stats

Clear all cached entries

Interactive mode — wraps any LLM call with caching

node scripts/cache.js query "Your question here"

When to Use This Skill

• - Before making any LLM API call, check if a semantically similar query was already answered
• When building agents that answer repetitive questions (support bots, FAQ systems)
• When you want to reduce OpenAI/Anthropic API costs by 40-80%
• When you need faster response times for common queries

Configuration

Set these environment variables:

• - REDIS_URL — Redis connection string with vector search support (Redis Cloud or Redis Stack)
• INLINECODE1 — For generating embeddings
• INLINECODE2 — Similarity threshold 0-1 (default: 0.80, higher = stricter matching)
• INLINECODE3 — Cache TTL in seconds (default: 86400 = 24 hours)

Example Workflow

CODEBLOCK5

Performance

• - Cache lookup: ~5-15ms (vs 1-5 seconds for LLM call)
• Embedding generation: ~50-100ms
• Storage per entry: ~6KB (1536-dim vector + metadata)
• Supports millions of cached entries