human_test() — Real Human Feedback for AI Products

AI agents cannot judge human perception, emotion, or usability. This skill lets you call real humans to test any product URL and get structured feedback back.

What it does

1. 1. You call human_test() with a product URL or description (URL is optional — also works for mobile apps, desktop software, etc.)
2. AI auto-generates a structured test plan
3. Real human testers claim the task on the web platform
4. Each tester records their screen and microphone (up to 15 min) while completing a guided feedback flow — first impression, task steps, NPS rating
5. AI extracts key frames from each recording and uses vision AI to analyze usability issues, then aggregates all feedback into a structured report with severity-ranked findings

Setup

Option A: Hosted (zero setup)

Use the hosted version at https://human-test.work — no installation needed. Register to get an API key, then skip to Create a test task below using BASE_URL=https://human-test.work.

Option B: Self-hosted (auto-install)

human_test() can run locally. Before creating a task, check if the server is reachable:

CODEBLOCK0

If the server is not running, install and start it:

CODEBLOCK1

This auto-detects AI API keys from your environment (ANTHROPIC_API_KEY, OPENAI_API_KEY, DEEPSEEK_API_KEY, or GEMINI_API_KEY), creates a local SQLite database, builds the app, and starts it on port 3000.

A default admin user is created automatically — no registration needed.

Set BASE_URL: Ask the user once for their preferred base URL. Default: INLINECODE7

Quick start

Create a test task

CODEBLOCK2

Response:
CODEBLOCK3

Check progress and get the report

CODEBLOCK4

Response (when completed):
CODEBLOCK5

Note for agents: If repoUrl was provided, code fix generation starts automatically after the report is ready — no need to trigger it manually. Keep polling until codeFixStatus is COMPLETED or FAILED, or use codeFixWebhookUrl to get notified.

Parameters

Parameter	Required	Default	Description
INLINECODE13	No	—	Product URL to test (optional — leave empty for mobile apps or non-web products)
INLINECODE14

No | Auto from hostname | Task title | | focus | No | — | What testers should focus on | | maxTesters | No | 5 | Number of testers (1-50) | | estimatedMinutes | No | 10 | Expected test duration | | creator | No | admin | Name of the agent/user creating the task (auto-creates a user if needed) | | webhookUrl | No | — | HTTPS URL to receive the report on completion | | codeFixWebhookUrl | No | — | HTTPS URL to receive code fix results on completion | | repoUrl | No | — | GitHub repo URL for code-level fix suggestions | | repoBranch | No | repo default | Branch to analyze (only used with repoUrl) | | locale | No | en | Report language: en (English) or zh (Chinese) |

Async webhooks

There are two separate webhooks for the two stages:

Report webhook (webhookUrl)

If you provide a webhookUrl, the platform will POST the report to that URL when it's ready:

CODEBLOCK6

Code fix webhook (codeFixWebhookUrl)

If you provide a codeFixWebhookUrl, the platform will POST the code fix result when done:

CODEBLOCK7

Report format (structured for AI agents)

The report is returned as a markdown string in the report field. It uses a consistent, machine-parseable structure designed for AI agents to read and act on directly — for example, to automatically file issues, create PRs, or prioritize a fix backlog.

Section structure

Every report contains these exact sections in order:

CODEBLOCK8

Parsing tips for agents

• - Severity levels: [CRITICAL], [MAJOR], [MINOR] — always in brackets in issue headers
• Priority tags: P0, P1, P2, P3 — in the Recommendations section
• Each issue has 3 fields: Evidence, Impact, Recommendation — always bolded labels
• Metadata table: always the first section, machine-readable key-value pairs
• NPS scores: appear in Metadata (average) and NPS Analysis (per-tester breakdown)

Agent auto-fix workflow

The structured report format is designed for a closed-loop workflow: your agent calls human_test(), receives the report, and automatically fixes the issues found — no human intervention needed after testing.

Recommended flow

1. 1. Call human_test() with your product URL (include webhookUrl to get notified)
2. Wait for the report (poll /api/skill/status/<taskId> or receive webhook)
3. Parse the ## Issues section — each issue has [SEVERITY], Evidence, Impact, and Recommendation
4. For [CRITICAL] and [MAJOR] issues, use the Recommendation field to generate targeted code fixes
5. Create commits or PRs for each fix
6. (Optional) Call human_test() again to verify the fixes

Each issue's Evidence tells you what went wrong, Impact tells you why it matters, and Recommendation tells you exactly what to fix. This gives your agent enough context to write a targeted fix without guessing.

Repo-aware code fix suggestions

If you pass a repoUrl, the platform automatically triggers code fix generation as soon as the report is ready. It clones your repo, analyzes the code against reported issues, and produces file-level code fix suggestions (with unified diffs) appended to the report as a ## Code Fix Suggestions section.

Two modes (auto-detected)

Mode 1 — Read-only: Grant GitHub user avivahe326 read access to your repo. After the report, the platform clones the repo, analyzes the code against reported issues, and appends code-level diffs to the report.

Mode 2 — Developer access: Grant avivahe326 write access. Same as Mode 1, plus: creates a branch human-test/fixes-<taskId>, applies the diffs, pushes, and opens a PR. The PR URL is returned in the webhook payload as codeFixPrUrl and in the status API.

Example with repoUrl

CODEBLOCK9

Links

• - Web platform: https://human-test.work
• GitHub: https://github.com/avivahe326/humantest