Tiered Memory System v2.2.0

A mind that remembers everything is as useless as one that remembers nothing. The art is knowing what to keep. 🧠

EvoClaw-compatible three-tier memory system inspired by human cognition and PageIndex tree retrieval.

What's New in v2.2.0

🔄 Automatic Daily Note Ingestion

• - Consolidation (daily/monthly/full modes) now auto-runs INLINECODE3
• Bridges memory/YYYY-MM-DD.md files → tiered memory system
• No more manual ingestion required — facts flow automatically
• Fixes the "two disconnected data paths" problem

What's New in v2.1.0

🎯 Structured Metadata Extraction

• - Automatic extraction of URLs, shell commands, and file paths from facts
• Preserved during distillation and consolidation
• Searchable by URL fragment

✅ Memory Completeness Validation

• - Check daily notes for missing URLs, commands, and next steps
• Proactive warnings for incomplete information
• Actionable suggestions for improvement

🔍 Enhanced Search

• - Search facts by URL fragment
• Get all stored URLs from warm memory
• Metadata-aware fact storage

🛡️ URL Preservation

• - URLs explicitly preserved during LLM distillation
• Fallback metadata extraction if LLM misses them
• Command-line support for adding metadata manually

Architecture

CODEBLOCK0

Design Principles

From Human Memory

• - Consolidation — Short-term → long-term happens during consolidation cycles
• Relevance Decay — Unused memories fade; accessed memories strengthen
• Strategic Forgetting — Not remembering everything is a feature
• Hierarchical Organization — Navigate categories, not scan linearly

From PageIndex

• - Vectorless Retrieval — LLM reasoning instead of embedding similarity
• Tree-Structured Index — O(log n) navigation, not O(n) scan
• Explainable Results — Every retrieval traces a path through categories
• Reasoning-Based Search — "Why relevant?" not "how similar?"

Cloud-First (EvoClaw)

• - Device is replaceable — Soul lives in cloud (Turso)
• Critical sync — Hot + tree sync after every conversation
• Disaster recovery — Full restore in <2 minutes
• Multi-device — Same agent across phone/desktop/embedded

Memory Tiers

🔴 Hot Memory (5KB max)

Purpose: Core identity and active context, always in agent's context window.

Structure:
CODEBLOCK1

Auto-pruning:

• - Lessons: Max 20, removes lowest-importance when full
• Events: Keeps last 10 only
• Tasks: Max 10 pending
• Total size: Hard limit at 5KB, progressively prunes if exceeded

Generates: MEMORY.md — auto-rebuilt from structured hot state

🟡 Warm Memory (50KB max, 30-day retention)

Purpose: Recent distilled facts with decay scoring.

Entry format:
CODEBLOCK2

Scoring:
CODEBLOCK3

Tier classification:

• - score >= 0.7 → Hot (promote to hot state)
• INLINECODE7 → Warm (keep)
• INLINECODE8 → Cold (archive)
• INLINECODE9 → Frozen (delete after retention period)

Eviction triggers:

1. 1. Age > 30 days AND score < 0.3
2. Total warm size > 50KB (evicts lowest-scored)
3. Manual consolidation

🟢 Cold Memory (Unlimited, Turso)

Purpose: Long-term archive, queryable but never bulk-loaded.

Schema:
CODEBLOCK4

Retention: 10 years (configurable)
Cleanup: Monthly consolidation removes frozen entries older than retention period

Tree Index

Purpose: Hierarchical category map for O(log n) retrieval.

Constraints:

• - Max 50 nodes
• Max depth 4 levels
• Max 2KB serialized
• Max 10 children per node

Example:
CODEBLOCK5

Operations:

• - --add PATH DESC — Add category node
• INLINECODE11 — Remove node (only if no data)
• INLINECODE12 — Remove dead nodes (no activity in 60+ days)
• INLINECODE13 — Pretty-print tree

Distillation Engine

Purpose: Three-stage compression of conversations.

Pipeline:
CODEBLOCK6

Stage 1→2: Raw → Distilled

Input: Raw conversation text
Output: Structured JSON

CODEBLOCK7

Modes:

• - rule: Regex/heuristic extraction (fast, no LLM)
• INLINECODE15: LLM-powered extraction (accurate, requires endpoint)

Usage:
CODEBLOCK8

Stage 2→3: Distilled → Core Summary

Purpose: One-line summary for tree index

Example:
CODEBLOCK9

Target: <30 bytes

LLM-Powered Tree Search

Purpose: Semantic search through tree structure using LLM reasoning.

How it works:

1. 1. Build prompt with tree structure + query
2. LLM reasons about which categories are relevant
3. Returns category paths with relevance scores
4. Fetches memories from those categories

Example:

Query: "What did we decide about the hackathon deadline?"

Keyword search returns:

• - projects/evoclaw (0.8)
• INLINECODE17 (0.4)

LLM search reasons:

• - projects/evoclaw/bsc (0.95) — "BSC integration for hackathon"
• INLINECODE19 (0.85) — "Deadline mentioned here"

LLM prompt template:
CODEBLOCK10

Usage:
CODEBLOCK11

Multi-Agent Support

Agent ID scoping — All operations support --agent-id flag.

File layout:
CODEBLOCK12

Cold storage: Agent-scoped queries via agent_id column

Usage:
CODEBLOCK13

Consolidation Modes

Purpose: Periodic memory maintenance and optimization.

Quick (hourly)

• - Warm eviction (score-based)
• Archive expired to cold
• Recalculate all scores
• Rebuild MEMORY.md

Daily

• - Everything in Quick
• Tree prune (remove dead nodes, 60+ days no activity)

Monthly

• - Everything in Daily
• Tree rebuild (LLM-powered restructuring, future)
• Cold cleanup (delete frozen entries older than retention)

Full

• - Everything in Monthly
• Full recalculation of all scores
• Deep tree analysis
• Generate consolidation report

Usage:
CODEBLOCK14

Recommended schedule:

• - Quick: Every 2-4 hours (heartbeat)
• Daily: Midnight via cron
• Monthly: 1st of month via cron

Critical Sync (Cloud-First)

Purpose: Cloud backup of hot state + tree after every conversation.

What syncs:

• - Hot memory state (identity, owner profile, active context, lessons)
• Tree index (structure + counts)
• Timestamp

Recovery: If device lost, restore from cloud in <2 minutes

Usage:
CODEBLOCK15

Retry strategy: Exponential backoff if cloud unreachable (5s, 10s, 20s, 40s)

Metrics & Observability

Tracked metrics:
CODEBLOCK16

Usage:
CODEBLOCK17

Key metrics:

• - contexttokenssaved — Estimated tokens saved vs. flat MEMORY.md
• retrievalcount — How often memories are accessed
• evictionstoday — Memory pressure indicator
• warmmemorysize_kb — Storage usage

Commands Reference

Store

CODEBLOCK18

Importance guide:

• - 0.9-1.0 — Critical decisions, credentials, core identity
• INLINECODE23 — Project decisions, architecture, preferences
• INLINECODE24 — General facts, daily events
• INLINECODE25 — Casual mentions, low priority

Example:
CODEBLOCK19

Validate (v2.1.0)

CODEBLOCK20

Purpose: Check daily notes for incomplete information (missing URLs, commands, next steps).

Example:
CODEBLOCK21

Output:
CODEBLOCK22

Extract Metadata (v2.1.0)

CODEBLOCK23

Purpose: Extract structured metadata (URLs, commands, paths) from a file.

Example:
CODEBLOCK24

Output:
CODEBLOCK25

Search by URL (v2.1.0)

CODEBLOCK26

Purpose: Search facts by URL fragment.

Example:
CODEBLOCK27

Output:
CODEBLOCK28

Retrieve

CODEBLOCK29

Modes:

• - Default: Keyword-based tree + warm + cold search
• INLINECODE26: LLM-powered semantic tree search

Example:
CODEBLOCK30

Distill

CODEBLOCK31

Example:
CODEBLOCK32

Output:
CODEBLOCK33

Hot Memory

CODEBLOCK34

Keys:

• - identity — Agent/owner identity info
• INLINECODE28 — Owner preferences, personality
• INLINECODE29 — Add critical lesson
• INLINECODE30 — Add event to active context
• INLINECODE31 — Add task to active context
• INLINECODE32 — Add/update project

Examples:
CODEBLOCK35

Tree

CODEBLOCK36

Examples:
CODEBLOCK37

Cold Storage

CODEBLOCK38

Examples:
CODEBLOCK39

Configuration

File: config.json (optional, uses defaults if not present)

CODEBLOCK40

Integration with OpenClaw Agents

After Conversation

CODEBLOCK41

Before Responding (Retrieval)

CODEBLOCK42

Heartbeat Consolidation

CODEBLOCK43

LLM Integration

Model Recommendations

For Distillation & Tree Search:

• - Claude 3 Haiku (fast, cheap, excellent structure)
• GPT-4o-mini (good balance)
• Gemini 1.5 Flash (very fast)

For Tree Rebuilding:

• - Claude 3.5 Sonnet (better reasoning)
• GPT-4o (strong planning)

Cost Optimization

1. 1. Use cheaper models for frequent operations (distill, search)
2. Batch distillation — Queue conversations, distill in batch
3. Cache tree prompts — Tree structure doesn't change often
4. Skip LLM for simple — Use rule-based for short conversations

Example LLM Endpoint

CODEBLOCK44

Performance Characteristics

Context Size:

• - Hot: ~5KB (always loaded)
• Tree: ~2KB (always loaded)
• Retrieved: ~1-3KB per query
• Total: ~8-15KB (constant, regardless of agent age)

Retrieval Speed:

• - Keyword: 10-20ms
• LLM tree search: 300-600ms
• Cold query: 50-100ms

5-Year Scenario:

• - Hot: Still 5KB (living document)
• Warm: Last 30 days (~50KB)
• Cold: ~50MB in Turso (compressed distilled facts)
• Tree: Still 2KB (different nodes, same size)
• Context per session: Same as day 1

Comparison with Alternatives

System	Memory Model	Scaling	Accuracy	Cost
Flat MEMORY.md	Linear text	❌ Months	⚠️ Degrades	❌ Linear
Vector RAG

Embeddings | ✅ Years | ⚠️ Similarity≠relevance | ⚠️ Moderate | | EvoClaw Tiered | Tree + tiers | ✅ Decades | ✅ Reasoning-based | ✅ Fixed |

Why tree > vectors:

• - Accuracy: 98%+ vs. 70-80% (PageIndex benchmark)
• Explainable: "Projects → EvoClaw → BSC" vs. "cosine 0.73"
• Multi-hop: Natural vs. poor
• False positives: Low vs. high

Troubleshooting

Tree size exceeding limit

CODEBLOCK45

Warm memory filling up

CODEBLOCK46

Hot memory exceeding 5KB

CODEBLOCK47

LLM search failing

CODEBLOCK48

Migration from v1.x

Backward compatible: Existing warm-memory.json and memory-tree.json files work as-is.

New files:

• - config.json (optional, uses defaults)
• INLINECODE37 (auto-created)
• INLINECODE38 (auto-created)

Steps:

1. 1. Update skill: INLINECODE39
2. Run consolidation to rebuild hot state: INLINECODE40
3. Initialize cold storage (optional): INLINECODE41
4. Configure agent to use new commands (see Integration section)

Migration from v2.0 to v2.1

Fully backward compatible: Existing memory files work without changes.

What's new:

• - ✅ Metadata automatically extracted from existing facts when loaded
• ✅ New commands: validate, extract-metadata, INLINECODE44
• ✅ store command now accepts --url, --command, --path flags
• ✅ Distillation preserves URLs and technical details
• ✅ No action required - just update and use new features

Testing the upgrade:
CODEBLOCK49

References

• - Design: /docs/TIERED-MEMORY.md (EvoClaw)
• Cloud Sync: /docs/CLOUD-SYNC.md (EvoClaw)
• Inspiration: PageIndex (tree-based retrieval)

---

v2.1.0 — A mind that remembers everything is as useless as one that remembers nothing. The art is knowing what to keep. Now with structured metadata to remember HOW, not just WHAT. 🧠🌲🔗