LLM Router

An intelligent proxy that classifies incoming requests by complexity and routes them to appropriate LLM models. Use cheaper/faster models for simple tasks and reserve expensive models for complex ones.

Works with OpenClaw to reduce token usage and API costs by routing simple requests to smaller models.

Status: Tested with Anthropic, OpenAI, Google Gemini, Kimi/Moonshot, and Ollama.

Quick Start

Prerequisites

1. 1. Python 3.10+ with pip
2. Ollama (optional - only if using local classification)
3. Anthropic API key or Claude Code OAuth token (or other provider key)

Setup

CODEBLOCK0

Verify Installation

CODEBLOCK1

Start the Server

CODEBLOCK2

Options:

• - --port PORT - Port to listen on (default: 4001)
• INLINECODE1 - Host to bind (default: 127.0.0.1)
• INLINECODE2 - Config file path (default: config.yaml)
• INLINECODE3 - Enable verbose logging
• INLINECODE4 - Enable OpenClaw compatibility (rewrites model name in system prompt)

Configuration

Edit config.yaml to customize:

Model Routing

CODEBLOCK3

Note: Reasoning models are auto-detected and use correct API params.

Classifier

Three options for classifying request complexity:

Local (default) - Free, requires Ollama:
CODEBLOCK4

Anthropic - Uses Haiku, fast and cheap:
CODEBLOCK5

OpenAI - Uses GPT-4o-mini:
CODEBLOCK6

Google - Uses Gemini:
CODEBLOCK7

Kimi - Uses Moonshot:
CODEBLOCK8

Use remote (anthropic/openai/google/kimi) if your machine can't run local models.

Supported Providers

• - anthropic:claude-* - Anthropic Claude models (tested)
• INLINECODE7, openai:o1-*, openai:o3-* - OpenAI models (tested)
• INLINECODE10 - Google Gemini models (tested)
• INLINECODE11, kimi:moonshot-* - Kimi/Moonshot models (tested)
• INLINECODE13 - Local Ollama models (tested)

Complexity Levels

Customizing Classification

Edit ROUTES.md to tune how messages are classified. The classifier reads the table in this file to determine complexity levels.

API Usage

The router exposes an OpenAI-compatible API:

CODEBLOCK9

Testing Classification

CODEBLOCK10

Running as macOS Service

Create ~/Library/LaunchAgents/com.llmrouter.plist:

CODEBLOCK11

Important: Replace /path/to/llmrouter with your actual install path. Must use the venv python, not system python.

CODEBLOCK12

OpenClaw Configuration

Add the router as a provider in ~/.openclaw/openclaw.json:

CODEBLOCK13

Note: Cost is set to 0 because actual costs depend on which model the router selects. The router logs which model handled each request.

Set as Default Model (Optional)

To use the router for all agents by default, add:

CODEBLOCK14

Using with OAuth Tokens

If your config.yaml uses an Anthropic OAuth token from OpenClaw's ~/.openclaw/auth-profiles.json, the router automatically handles Claude Code identity headers.

OpenClaw Compatibility Mode (Required)

If using with OpenClaw, you MUST start the server with --openclaw:

CODEBLOCK15

This flag enables compatibility features required for OpenClaw:

• - Rewrites model names in responses so OpenClaw shows the actual model being used
• Handles tool name and ID remapping for proper tool call routing

Without this flag, you may encounter errors when using the router with OpenClaw.

Common Tasks

• - Check server status: INLINECODE21
• View current config: INLINECODE22
• Test a classification: INLINECODE23
• Run classification tests: INLINECODE24
• Restart server: Stop and run python server.py again
• View logs (if running as service): INLINECODE26

Troubleshooting

"externally-managed-environment" error

"Connection refused" on port 4001

Classification returns wrong complexity

Ollama errors / "model not found"

OAuth token not working

LaunchAgent not starting