闲社服务运行正常AI智能体自动化平台
客户端下载帮助中心服务商入驻
免费注册
技能集市 > AI智能 > llm-evaluator
l

大模型评估器

技能标识：llm-evaluator

>

作者：admin | 来源记录：ClawHub
下载
登记来源
ClawHub
版本
V 1.0.0
检测标记
后台标记通过
免费
免费获取
1
收藏
来源与检测标记为平台登记信息，并不代表已展示可复核的检测报告。使用前请核对版本、依赖和所需权限，建议先在隔离环境中运行。
概述
安装方式
版本历史

大模型评估器

LLM 评估器 ⚖️

基于Langfuse的LLM即评判者评估系统。使用GPT-5-nano对AI输出进行评分。

适用场景

  • - 评估搜索结果或AI回复的质量
  • 对追踪记录进行相关性、准确性、幻觉检测评分
  • 批量评分近期未评分的追踪记录
  • 对智能体输出进行质量保证

使用方法

bash

使用示例用例进行测试


python3 {baseDir}/scripts/evaluator.py test

对特定Langfuse追踪记录进行评分

python3 {baseDir}/scripts/evaluator.py score

仅使用特定评估器进行评分

python3 {baseDir}/scripts/evaluator.py score --evaluators relevance

对近期未评分的追踪记录进行回填评分

python3 {baseDir}/scripts/evaluator.py backfill --limit 20

评估器

评估器衡量指标评分范围
相关性回复与查询的相关性0–1
准确性
事实正确性 | 0–1 | | 幻觉 | 虚构信息检测 | 0–1 | | 有用性 | 整体实用性 | 0–1 |

致谢

由 M. Abidi 构建 | agxntsix.ai
YouTube | GitHub
属于OpenClaw智能体的 AgxntSix技能套件 的一部分。

📅 需要为您的业务搭建OpenClaw？ 预约免费咨询

标签

skill ai

通过对话安装

以下为平台配置的接入选项，并非逐项实测通过。能否安装取决于客户端支持、技能来源和运行环境：

OpenClaw WorkBuddy QClaw Kimi Claude

方式一：安装 SkillHub 和技能

帮我安装 SkillHub 和 llm-evaluator-pro-1776420065 技能

方式二：设置 SkillHub 为优先技能安装源

设置 SkillHub 为我的优先技能安装源，然后帮我安装 llm-evaluator-pro-1776420065 技能

通过命令行安装

skillhub install llm-evaluator-pro-1776420065

下载

⬇ 下载 llm-evaluator v1.0.0（免费）

文件大小: 5.1 KB | 发布时间: 2026-4-17 18:57

v1.0.0 最新 2026-4-17 18:57
LLM-as-a-Judge evaluator via Langfuse

相关推荐

s

self-improvement

Captures learnings, errors, and corrections to enable continuous improvement. Use when: (1) A command or operation fails unexpectedly, (2) User corrects Claude ('No, that's wrong...', 'Actually...'), (3) User requests a capability that doesn't exist, (4) An external API or tool fails, (5) Claude realizes its knowledge is outdated or incorrect, (6) A better approach is discovered for a recurring task. Also review learnings before major tasks.

AI智能
s

self-improvement

Captures learnings, errors, and corrections to enable continuous improvement. Use when: (1) A command or operation fails unexpectedly, (2) User corrects Claude ('No, that's wrong...', 'Actually...'), (3) User requests a capability that doesn't exist, (4) An external API or tool fails, (5) Claude realizes its knowledge is outdated or incorrect, (6) A better approach is discovered for a recurring task. Also review learnings before major tasks.

AI智能
s

self-improvement

Captures learnings, errors, and corrections to enable continuous improvement. Use when: (1) A command or operation fails unexpectedly, (2) User corrects Claude ('No, that's wrong...', 'Actually...'), (3) User requests a capability that doesn't exist, (4) An external API or tool fails, (5) Claude realizes its knowledge is outdated or incorrect, (6) A better approach is discovered for a recurring task. Also review learnings before major tasks.

AI智能
s

self-improvement

Captures learnings, errors, and corrections to enable continuous improvement. Use when: (1) A command or operation fails unexpectedly, (2) User corrects Claude ('No, that's wrong...', 'Actually...'), (3) User requests a capability that doesn't exist, (4) An external API or tool fails, (5) Claude realizes its knowledge is outdated or incorrect, (6) A better approach is discovered for a recurring task. Also review learnings before major tasks.

AI智能
返回顶部