Ollama — Herd Your Ollama LLMs Into One Endpoint

You have Ollama running on multiple machines. This skill gives you one Ollama endpoint that routes every Ollama request to the best available device automatically. No more hardcoding Ollama IPs, no more manual Ollama load balancing, no more "which Ollama machine has that model loaded?"

Setup Ollama Herd

CODEBLOCK0

Now point everything at http://localhost:11435 instead of http://localhost:11434. Same Ollama API, same Ollama models, smarter Ollama routing.

Package: ollama-herd | Repo: github.com/geeks-accelerator/ollama-herd

Use your Ollama models through the fleet

OpenAI SDK (drop-in Ollama routing)

CODEBLOCK1

Ollama API (same as before, different port)

CODEBLOCK2

What the Ollama router does

When an Ollama request comes in, the Ollama router scores every online Ollama node on 7 signals:

1. 1. Ollama Thermal — is the Ollama model already loaded in GPU memory? (+50 for hot)
2. Ollama Memory fit — how much headroom does the Ollama node have?
3. Ollama Queue depth — how many Ollama requests are waiting?
4. Ollama Wait time — estimated latency based on Ollama history
5. Ollama Role affinity — large Ollama models prefer big machines
6. Ollama Availability — is the Ollama node reliably available?
7. Ollama Context fit — does the loaded Ollama context window fit the request?

The highest-scoring Ollama node handles the request. If it fails, the Ollama router retries on the next best node automatically.

Supported Ollama models

Any model that runs on Ollama works through the Ollama fleet. Popular Ollama models:

Ollama Resilience features

• - Ollama Auto-retry — re-routes to next best Ollama node on failure (before first chunk)
• Ollama VRAM-aware fallback — routes to a loaded Ollama model in the same category instead of cold-loading
• Ollama Context protection — prevents num_ctx from triggering expensive Ollama model reloads
• Ollama Zombie reaper — cleans up stuck in-flight Ollama requests
• Ollama Auto-pull — downloads missing Ollama models to the best node automatically

Also available via Ollama Herd

The same Ollama fleet router handles three more workloads:

Ollama Image generation

CODEBLOCK3

Ollama Speech-to-text

CODEBLOCK4

Ollama Embeddings

CODEBLOCK5

Ollama Dashboard

INLINECODE14 — 8 tabs: Ollama Fleet Overview, Trends, Ollama Model Insights, Apps, Benchmarks, Ollama Health, Recommendations, Settings. Real-time Ollama queue visibility with [TEXT], [IMAGE], [STT], [EMBED] badges.

Ollama Request tagging

Track per-project Ollama usage:

CODEBLOCK6

Full Ollama documentation

Ollama Agent Setup Guide

Ollama Guardrails

• - Never restart the Ollama router or Ollama node agents without user confirmation.
• Never delete or modify files in ~/.fleet-manager/ (Ollama data).
• Never pull or delete Ollama models without user confirmation.