🦞 Gradient AI — Serverless Inference

⚠️ This is an unofficial community skill, not maintained by DigitalOcean. Use at your own risk.

"Why manage GPUs when the ocean provides?" — ancient lobster proverb

Use DigitalOcean's Gradient Serverless Inference to call large language models without managing infrastructure. The API is OpenAI-compatible, so standard SDKs and patterns work — just point at https://inference.do-ai.run/v1 and swim.

Authentication

All requests need a Model Access Key in the Authorization: Bearer header.

CODEBLOCK0

Where to get one: DigitalOcean Console → Gradient AI → Model Access Keys → Create Key.

📖 Full auth docs

Tools

🔍 List Available Models

Window-shop for LLMs before you swipe the card.

CODEBLOCK1

Use this before hardcoding model IDs — models are added and deprecated over time.

Direct API call:
CODEBLOCK2

📖 Models reference

💬 Chat Completions

The classic. Send structured messages (system/user/assistant roles), get a response. OpenAI-compatible, so you probably already know how this works.

CODEBLOCK3

Direct API call:
CODEBLOCK4

📖 Chat Completions docs

⚡ Responses API (Recommended)

DigitalOcean's recommended endpoint for new integrations. Simpler request format and supports prompt caching — a.k.a. "stop paying twice for the same context."

CODEBLOCK5

Direct API call:
CODEBLOCK6

When to use which:

📖 Responses API docs

🖼️ Generate Images

Turn text prompts into images. Because sometimes a chart isn't enough.

CODEBLOCK7

Direct API call:
CODEBLOCK8

📖 Image generation docs

🧠 Model Selection Guide

Not all models are created equal. Choose wisely, young crustacean:

🦞 Pro tip: Cost-aware routing. Use a fast model (e.g., qwen3-32b) to score or triage, then only escalate to a strong model (e.g., openai-gpt-oss-120b) when depth is needed. Enable prompt caching for repeated context.

Always run python3 gradient_models.py to check what's currently available — the menu changes.

📖 Available models

💰 Model Pricing Lookup

Check what models cost before you rack up a bill. Scrapes the official DigitalOcean pricing page — no API key needed.

CODEBLOCK9

How it works:

• - Fetches live pricing from DigitalOcean's docs (public page, no auth)
• Caches results for 24 hours in INLINECODE11
• Falls back to a bundled snapshot if the live fetch fails

🦞 Pro tip: Run python3 gradient_pricing.py --model "gpt-oss" before choosing a model to see the cost difference between gpt-oss-120b ($0.10/$0.70) and gpt-oss-20b ($0.05/$0.45) per 1M tokens.

📖 Pricing docs

CLI Reference

All scripts accept --json for machine-readable output.

CODEBLOCK10

External Endpoints

Security & Privacy

• - All requests go to inference.do-ai.run — DigitalOcean's own endpoint
• Your GRADIENT_API_KEY is sent as a Bearer token in the Authorization header
• No other credentials or local data leave the machine
• Model Access Keys are scoped to inference only — they can't manage your DO account
• Prompt caching entries are scoped to your account and automatically expire

Trust Statement

By using this skill, prompts and data are sent to DigitalOcean's Gradient Inference API.
Only install if you trust DigitalOcean with the content you send to their LLMs.

Important Notes

• - Run python3 gradient_models.py before assuming a model exists — they rotate
• All scripts exit with code 1 and print errors to stderr on failure