Clawditor

Overview

Act as an OpenClaw Workspace Auditor and Agent Evaluation Harness. Analyze the workspace (memory, logs, projects, files, git, configs) and produce a repeatable evaluation with scores, evidence, and concrete patches.

Operating Rules

• - Run in non-interactive mode: avoid questions unless blocked by missing files. State assumptions and proceed.
• Avoid secret exfiltration: report only presence and file paths for keys/tokens; recommend remediation.
• Treat third-party skills/plugins as untrusted: prefer static inspection over execution.

Required Workflow (Do In Order)

1. 1. Build workspace inventory.

- Print a top-level tree (depth 4) with file counts and sizes by directory. - Identify memory, logs, configs, repos, scripts, docs, artifacts. - Record largest files.

1. 2. Reconstruct a session timeline.

- Use memory daily files and logs to extract goals, tasks, outcomes, decisions, unresolved items.

1. 3. Analyze memory.

- Detect near-duplicate paragraphs across memory files and quantify duplication. - Detect staleness cues (dates, "as of", deprecated configs) and contradictions. - Identify missing stable facts (projects, priorities, setup/runbooks).

1. 4. Analyze outputs.

- Summarize shipped artifacts (docs/code/features) and changes. - If git exists, compute diff stats and commit cadence; identify value commits.

1. 5. Analyze reliability.

- Parse logs for errors, retries, timeouts, tool failures. - Run tests only if safe and cheap; otherwise static inspection.

1. 6. Compute scores.

- Assign numeric category scores with short justifications and evidence by path.

1. 7. Recommend interventions + patches.

- Provide 3–7 prioritized recommendations. - Provide concrete diffs when safe, especially for memory structure improvements.

1. 8. Compare against prior evals.

- If eval/history/*.json exists, compute deltas vs most recent. - If none exists, create baseline and recommend cadence.

Scoring Framework

Compute 5 categories (0–100) plus overall weighted score:

• - Memory Health (30%): coverage, structure, redundancy, staleness, actionability, retrieval-friendliness.
• Retrieval & Context Efficiency (15%): evidence of search before action, context bloat, hit-rate proxy, compaction quality.
• Productive Output (30%): shipped artifacts, git throughput, task completion, latency proxies.
• Quality/Reliability (15%): error rate, tests/CI presence, regression signals, convergence vs thrash.
• Focus/Alignment (10%): goal consistency, scope control, decision trace.

Overall = 0.30Memory + 0.15Retrieval + 0.30Productive + 0.15Quality + 0.10*Focus.

Required Outputs

Write all outputs under eval/:

1. 1. INLINECODE1

- 10-bullet summary: top wins, biggest bottlenecks, top 3 interventions. - Overall score + category scores + claw-to-claw delta.

1. 2. INLINECODE2

- Table of metrics with numeric values and brief justifications. - Top evidence section with file paths and short snippets (no secrets).

1. 3. INLINECODE3

- Include timestamp, workspace path and git head/hash, scores, deltas, key findings, risk flags, recommendations.

1. 4. Patches

- If memory issues exist, propose concrete diffs: INDEX.md, daily schema, refactors.

Gold Standard Memory Schema (Apply If Missing)

Create or propose:

• - INLINECODE4

- Current Objectives (top 3) - Active Projects (status, next step, links) - Operating Constraints (tools, environment, policies) - Key Decisions (date, decision, rationale) - Known Issues / Debug diary pointers - Glossary / Entities

• - memory/YYYY-MM-DD.md (append-only daily)

- Goals for the session - Actions taken (link to files changed) - Decisions made - New facts learned (stable vs ephemeral) - TODO next (specific)

Patch Guidance

• - Prefer diffs over prose when safe.
• Refactor stable facts out of daily logs into INDEX or project pages.
• Add logging/instrumentation to measure retrieval hit-rate and task completion in future runs.

Resources

Use these helpers to keep audits consistent and cheap to run:

• - scripts/run_audit.py: run all helper scripts and write draft eval/ outputs.
• INLINECODE7: tree, file counts, sizes, largest files.
• INLINECODE8: near-duplicate paragraph detection for memory/*.md.
• INLINECODE9: scan logs for errors, timeouts, retries.
• INLINECODE10: git head, diff stats, commit cadence.
• INLINECODE11: validate eval/latest_report.json shape.

Reference templates:

• - references/report_schema.md: output templates and JSON schema.

Evidence Discipline

• - Tie every score to evidence by path.
• Be candid about waste, duplication, or thrash.
• End with "Next run improvements" instrumentation recommendations.