Agent Security Harness
332 security tests across 24 modules for AI agent systems. 4 wire protocols (MCP, A2A, L402, x402), 20+ enterprise platforms, GTG-1002 APT simulation, false positive rate testing, supply chain provenance, jailbreak resistance, AIUC-1 certification prep. Zero external dependencies for core protocol modules.
Current version: v3.8.1 | PyPI | GitHub | Apache 2.0
New in v3.8.1: MCP Server (expose harness as MCP tools for any AI agent), Attestation Registry (opt-in, Ed25519 signed), Telemetry (opt-in, GDPR compliant), GitHub Action for CI/CD, Free MCP Security Scan, AIUC-1 Certification Prep, Monthly Security Report pipeline, Discord Scan Bot. Validated at 97.9% pass rate (HRAO-E, 146 tests, Wilson 95% CI [0.943, 0.994]). 22 rounds of critical evaluation, 10/10 final score.
Safety
Non-destructive by default. All 332 tests send crafted inputs and analyze responses. No tests modify target state, delete data, or execute write operations.
Do NOT run against production systems without explicit authorization. Use isolated staging/test environments and test accounts, especially for payment endpoints (L402, x402).
Payment tests (L402/x402): Send crafted payment challenges and analyze responses. They do NOT execute real transactions, transfer funds, or interact with live payment networks.
Required Environment
Python 3.10+ and pip are required.
Environment variables:
| Variable | Required | Purpose |
|---|
| (none by default) | - | Most tests only need a target URL passed via CLI --url flag |
| INLINECODE1 |
Only for enterprise adapter tests | Platform-specific API key (SAP, Salesforce, Workday, etc.) - use scoped test credentials only |
|
ALPACA_PAPER_API_KEY | No | Only for trading-related integration tests |
No environment variables are required for standard protocol testing (MCP, A2A, L402, x402, over-refusal, provenance, jailbreak). The target URL is passed as a CLI argument, not an environment variable.
Credential guidance: If you use enterprise adapter tests that require API keys, store credentials securely using environment variables or .env files. Never commit API keys to version control. Only provide scoped test credentials, never production keys.
Install
CODEBLOCK0
Source verification:
Dependencies: Core protocol modules (MCP, A2A, L402, x402, over-refusal, provenance, jailbreak) use Python stdlib only (zero external dependencies). Application-layer suite requires requests and geopy.
Quick Reference
CODEBLOCK1
MCP Server Mode
Use the harness as an MCP tool that any AI agent can call:
CODEBLOCK2
Tools: scan_mcp_server (quick scan), full_security_audit (332 tests), aiuc1_readiness, get_test_catalog, validate_attestation.
Harness Modules (24 modules, 332 tests)
| Command | Tests | What It Tests |
|---|
| INLINECODE12 | 13 | MCP wire-protocol (JSON-RPC 2.0): tool poisoning, capability escalation, protocol downgrade, resource traversal, sampling hijack, context displacement |
| INLINECODE13 |
12 | A2A protocol: Agent Card spoofing, task injection, push notification redirect, skill injection, context isolation |
|
test l402 | 14 | L402 payments: macaroon tampering, preimage replay, caveat escalation, invoice validation |
|
test x402 | 25 | x402 payments: recipient manipulation, session theft, facilitator trust, cross-chain confusion, spending limits, health checks. Includes Agent Autonomy Risk Score (0-100) |
|
test enterprise | 31 | Tier 1 enterprise: SAP, Salesforce, Workday, Oracle, ServiceNow, Microsoft, Google, Amazon, OpenClaw |
|
test extended-enterprise | 27 | Tier 2 enterprise: IBM Maximo, Snowflake, Databricks, Pega, UiPath, Atlassian, Zendesk, IFS, Infor, HubSpot, Appian |
|
test framework | 11 | Framework adapters: LangChain, CrewAI, AutoGen, OpenAI Agents SDK, Bedrock |
|
test identity | 18 | NIST NCCoE Agent Identity: identification, authentication, authorization, auditing, data flow, standards compliance |
|
test gtg1002 | 17 | GTG-1002 APT simulation: 6 campaign phases + hallucination detection |
|
test advanced | 10 | Advanced patterns: polymorphic injection, stateful escalation, multi-domain chains, jailbreak persistence |
|
test over-refusal | 25 | False positive rate: legitimate requests across all protocols that should NOT be blocked. Measures FPR with Wilson CI |
|
test provenance | 15 | Supply chain: fake provenance, spoofed attestation, marketplace integrity, CVE-2026-25253 attack patterns |
|
test jailbreak | 25 | Jailbreak resistance: DAN variants, token smuggling, authority impersonation, context manipulation, persistence |
|
test return-channel | 8 | Return channel poisoning: output injection, ANSI escape, context overflow, encoded smuggling, structured data poisoning |
|
test capability-profile | 10 | Executor capability boundary validation, profile escalation prevention |
|
test harmful-output | 10 | Toxicity, bias, scope violations, deception (AIUC-1 C003/C004) |
|
test cbrn | 8 | Chemical/biological/radiological/nuclear content safeguards (AIUC-1 F002) |
|
test incident-response | 8 | Alert triggering, kill switch, log completeness, recovery (AIUC-1 E001-E003) |
|
test aiuc1 | 12 | AIUC-1 compliance: all 24 certification requirements mapped |
|
test cloud | 25 | Cloud agent platforms: AWS Bedrock, Azure AI, GCP Vertex, Anthropic, OpenAI |
|
test cve-2026 | 8 | CVE-2026-25253 reproduction: supply chain tool poisoning at scale |
CI/CD Integration (v3.8+)
CODEBLOCK3
CODEBLOCK4
Output Format
All harnesses produce JSON reports with:
- - Pass/fail per test with test ID and OWASP ASI mapping
- Full request/response transcripts for audit
- Elapsed time per test
- Wilson score confidence intervals (with
--trials N) - x402 harness adds: CSG mapping, financial impact estimation, Agent Autonomy Risk Score
When to Use Each Harness
- - Building an MCP server? Run
test mcp before deploying - Exposing an A2A agent? Run
test a2a to check Agent Card and task security - Adding agent payments? Run
test l402 (Lightning) or test x402 (USDC) before going live - Deploying on enterprise platforms? Run
test enterprise with your platform name - Red-teaming an agent system? Run
test gtg1002 for full APT campaign simulation - Need compliance evidence? Use
--trials 10 for NIST AI 800-2 aligned statistical reports - Preparing for AIUC-1 certification? Run all harnesses for B001/C010/D004 evidence
- Checking false positive rate? Run
test over-refusal to verify security controls don't break legitimate use - Validating supply chain integrity? Run
test provenance (especially relevant after CVE-2026-25253) - Testing jailbreak resistance? Run
test jailbreak for DAN variants and encoding evasion - Checking agent capability boundaries? Run
test capability-profile to verify escalation prevention - Validating safety controls? Run
test harmful-output and test cbrn for content safeguards - Testing incident response? Run
test incident-response for kill switch and recovery validation
Research
This harness is part of a published research program on autonomous AI agent governance:
Source & Provenance
Agent Security Harness
332项安全测试，覆盖24个模块，适用于AI代理系统。支持4种有线协议（MCP、A2A、L402、x402）、20+企业平台、GTG-1002 APT模拟、误报率测试、供应链溯源、越狱防御、AIUC-1认证准备。核心协议模块零外部依赖。
当前版本：v3.8.1 | PyPI | GitHub | Apache 2.0
v3.8.1 新特性： MCP服务器（将测试框架以MCP工具形式暴露给任意AI代理）、认证注册表（可选加入，Ed25519签名）、遥测（可选加入，符合GDPR）、CI/CD的GitHub Action、免费MCP安全扫描、AIUC-1认证准备、月度安全报告流水线、Discord扫描机器人。验证通过率97.9%（HRAO-E，146项测试，Wilson 95%置信区间[0.943, 0.994]）。22轮关键评估，最终得分10/10。
安全性
默认非破坏性。 全部332项测试仅发送精心构造的输入并分析响应。不修改目标状态、不删除数据、不执行写操作。
未经明确授权，请勿在生产系统上运行。 请使用隔离的预发布/测试环境和测试账户，尤其是针对支付端点（L402、x402）。
支付测试（L402/x402）： 发送精心构造的支付挑战并分析响应。不执行真实交易、不转移资金、不与真实支付网络交互。
环境要求
需要 Python 3.10+ 和 pip。
环境变量：
| 变量 | 是否必需 | 用途 |
|---|
| （默认无） | - | 大多数测试仅需通过CLI --url 参数传递目标URL |
| PLATFORMAPIKEY |
仅企业适配器测试需要 | 平台特定API密钥（SAP、Salesforce、Workday等）——仅使用受限测试凭证 |
| ALPACA
PAPERAPI_KEY | 否 | 仅用于交易相关集成测试 |
标准协议测试（MCP、A2A、L402、x402、过度拒绝、溯源、越狱）无需任何环境变量。 目标URL通过CLI参数传递，而非环境变量。
凭证指南： 如果使用需要API密钥的企业适配器测试，请使用环境变量或.env文件安全存储凭证。切勿将API密钥提交到版本控制系统。仅提供受限测试凭证，切勿使用生产密钥。
安装
bash
从PyPI安装（建议指定版本）
pip install agent-security-harness==3.8.1
验证安装
agent-security version
预期输出：3.8.1
源码验证：
依赖项： 核心协议模块（MCP、A2A、L402、x402、过度拒绝、溯源、越狱）仅使用Python标准库（零外部依赖）。应用层套件需要requests和geopy。
快速参考
bash
列出所有测试框架和测试
agent-security list
agent-security list mcp
测试MCP服务器（仅需URL）
agent-security test mcp --transport http --url http://localhost:8080/mcp
测试A2A代理
agent-security test a2a --url https://agent.example.com
测试L402支付端点（Lightning）——非破坏性
agent-security test l402 --url https://l402-endpoint.com
测试x402支付端点（Coinbase/USDC）——非破坏性
agent-security test x402 --url https://x402-endpoint.com
使用特定付费端点路径测试x402
agent-security test x402 --url https://apibase.pro --paid-path /api/v1/tools/geo.geocode/call
测试误报率（过度拒绝）
agent-security test over-refusal --url http://localhost:8080/mcp
测试供应链溯源和认证
agent-security test provenance --url http://localhost:8080/mcp
测试越狱防御
agent-security test jailbreak --url http://localhost:8080/mcp
测试能力配置文件边界
agent-security test capability-profile --url https://agent.example.com
测试有害输出防护
agent-security test harmful-output --url https://agent.example.com
测试CBRN内容防护
agent-security test cbrn --url https://agent.example.com
测试事件响应准备度
agent-security test incident-response --url https://agent.example.com
统计置信区间（符合NIST AI 800-2）
agent-security test mcp --url http://localhost:8080/mcp --trials 10
生产端点限速（测试间毫秒延迟）
agent-security test a2a --url https://agent.example.com --delay 1000
无服务器测试（内置模拟MCP服务器）
python -m testing.mock
mcpserver # 终端1：在端口8402启动
agent-security test mcp --transport http --url http://localhost:8402/mcp # 终端2
MCP服务器模式
将测试框架作为MCP工具使用，任何AI代理均可调用：
bash
stdio（适用于Cursor、Claude Desktop）
python -m mcp_server
HTTP
python -m mcp_server --transport http --port 8400
工具：scanmcpserver（快速扫描）、fullsecurityaudit（332项测试）、aiuc1readiness、gettestcatalog、validateattestation。
测试框架模块（24个模块，332项测试）
| 命令 | 测试数 | 测试内容 |
|---|
| test mcp | 13 | MCP有线协议（JSON-RPC 2.0）：工具投毒、能力升级、协议降级、资源遍历、采样劫持、上下文置换 |
| test a2a |
12 | A2A协议：代理卡片伪造、任务注入、推送通知重定向、技能注入、上下文隔离 |
| test l402 | 14 | L402支付：macaroon篡改、preimage重放、caveat升级、发票验证 |
| test x402 | 25 | x402支付：接收方操纵、会话窃取、协调者信任、跨链混淆、消费限额、健康检查。包含代理自主风险评分（0-100） |
| test enterprise | 31 | 一级企业：SAP、Salesforce、Workday、Oracle、ServiceNow、Microsoft、Google、Amazon、OpenClaw |
| test extended-enterprise | 27 | 二级企业：IBM Maximo、Snowflake、Databricks、Pega、UiPath、Atlassian、Zendesk、IFS、Infor、HubSpot、Appian |
| test framework | 11 | 框架适配器：LangChain、CrewAI、AutoGen、OpenAI Agents SDK、Bedrock |
| test identity | 18 | NIST NCCoE代理身份：标识、认证、授权、审计、数据流、标准合规 |
| test gtg1002 | 17 | GTG-1002 APT模拟：6个攻击阶段 + 幻觉检测 |
| test advanced | 10 | 高级模式：多态注入、状态升级、多域链、越狱持久化 |
| test over-refusal | 25 | 误报率：所有协议中不应被拦截的合法请求。使用Wilson置信区间测量FPR |
| test provenance | 15 | 供应链：伪造溯源、伪造认证、市场完整性、CVE-2026-25253攻击模式 |
| test jailbreak | 25 | 越狱防御：DAN变体、令牌走私、权限冒充、上下文操纵、持久化 |
| test return-channel | 8 | 返回通道投毒：输出注入、ANSI转义、上下文溢出、编码走私、结构化数据投毒 |
| test capability-profile | 10 | 执行器能力边界验证、配置文件升级防护 |
| test harmful-output | 10 | 毒性、偏见、范围违规、欺骗（AIUC-1 C003/C004） |
| test cbrn | 8 | 化学/生物/放射/核内容防护（AIUC-1 F002
