Ultramemory

Structured agent memory: extracts atomic facts from text, detects relations to existing knowledge (updates, contradicts, extends), embeds for semantic search, and auto-builds entity profiles.

PyPI: ultramemory | GitHub: INLINECODE1

Setup

CODEBLOCK0

Requirements:

• - Python 3.10+
• An LLM API key for fact extraction (Anthropic recommended, OpenAI also works)
• Local embeddings load automatically (sentence-transformers/all-MiniLM-L6-v2, ~80MB first run)

Environment:
CODEBLOCK1

Quick Start

CODEBLOCK2

CLI Usage

The scripts/memory.sh wrapper handles venv activation and API key loading.

Store memories (ingest)

CODEBLOCK3

Categories: person, preference, project, decision, event, INLINECODE8

Relations auto-detected: updates (supersedes old fact), contradicts, extends, supports, INLINECODE13

When a memory updates an existing one, the old memory is marked superseded and the new one gets an incremented version.

Recall (agent-optimized search)

Compact context block for injecting into agent prompts:

CODEBLOCK4

Output:
CODEBLOCK5

Search (full JSON)

CODEBLOCK6

Entity operations

CODEBLOCK7

Integration Patterns

Session startup

At the start of any session, hydrate context:
CODEBLOCK8

Post-conversation ingest

After meaningful conversations, pass the text directly:
CODEBLOCK9

API Server

For multi-agent setups, run the API server (requires separate install from PyPI):
CODEBLOCK10

Endpoints: POST /api/ingest, POST /api/search, POST /api/recall, INLINECODE19

Advanced: Auto-ingest from session transcripts

For continuous ingestion from session files, see the GitHub repo which includes auto_ingest.py and live_ingest.sh scripts.

Why Not Just Use MEMORY.md?

You'll outgrow a flat file fast. But you also can't replace it entirely. We tried.

At 18,000+ memories, search results get noisy. The DB is great at answering "what happened Tuesday?" but terrible as a session primer. Meanwhile, MEMORY.md is perfect for "who am I, who's my human, what are we working on" but can't hold 18K facts in 2K tokens.

The architecture we landed on uses three layers:

Layer 1: MEMORY.md (always loaded, zero cost)
Curated essentials under 2K tokens. Loaded every session, no API calls, no latency. Contains identity, active projects, key preferences. Think of it as working memory.

Layer 2: Ultramemory plugin (opportunistic injection)
When a message arrives, the plugin searches the DB and injects relevant memories if they score above a similarity threshold (we use 0.55). The agent never explicitly asks for this. It just gets richer context when the DB has something relevant.

Layer 3: Ultramemory direct (precision recall)
The agent explicitly searches when it needs specifics. "What was the benchmark result?" or "When did we decide to drop NYC?" This is the full 18K+ memory DB with semantic search, temporal filtering, and entity profiles.

MEMORY.md is the backup and the bootstrap. Ultramemory is the brain. You need both.

Architecture

• - Storage: Single SQLite DB with WAL mode. Agent-ID tagging for multi-agent isolation.
• Extraction: LLM extracts atomic facts, categorizes, detects entities, finds relations to existing memories.
• Embeddings: Local sentence-transformers (384-dim). No API calls for search.
• Relations: UPDATE, EXTEND, CONTRADICT, SUPPORT, DERIVE. Version chain with superseded tracking.
• Profiles: Auto-built entity profiles from accumulated facts.
• Events: Structured event extraction with canonical clustering and dedup.
• Temporal: Deterministic temporal expression parsing and date arithmetic (no LLM needed).

Cost

• - Ingest: ~$0.01-0.02 per call (3 LLM calls: extract, relate, profile)
• Search/recall: Free (local embeddings + SQLite)
• Embedding model: ~80MB download on first run, then cached

Benchmark

80% accuracy on LongMemEval_s (production-relevant questions). 32ms median search latency.