模型介绍
中文整理LycheeAI-coder-2b-II-pro · MLX 4bit
- *v7** —— 用「精选 1000 条 × 1 epoch」替代「全量 5341 条 × 0.22 epoch」,
验收 26/27 通过(v6.1 是 25/27)。
中文 English
基于 MiniCPM5-2B 微调的轻量编程助手 · Apache 2.0
- *2B 参数,主打本地工具调用**
这是个人开发者的项目,2B 参数注定它只是个"本地小助手"。** 它比 1B 版更能干,但请别用 GPT / Claude / DeepSeek 的标准要求它。
它最大的用处是:**跑在你自己的机器上,不用联网,能调工具,响应飞快。**
本仓库是 MLX 4bit 量化版本**,1.4 GB,Apple Silicon 上开箱即用。
四个版本,按需要选:**
版本 体积 用在什么场景
- -- --- ---
- *MLX 4bit**(本仓库) 1.42 GB Mac 上日常跑,最省内存、最快
MLX 无损 bf16 5.03 GB **要质量基准 / 二次量化 / 继续微调**
- f16 5.03 GB 同上(**同一份权重**,方便 transformers 用户找)
GGUF 1.45 / 2.50 / 4.69 GB Ollama、llama.cpp
- *MLX 无损版(bf16)**是给"要在它身上再加工"的人用的:4bit 是有损的,不能作为二次量化或续训的基础。
它和 -f16 仓库是**同一份权重**(字节相同),两个名字只是为了让 MLX 用户和 transformers 用户都能找到。
格式是标准 safetensors,**MLX 和 transformers 都能直接加载**。
中文
这一版做了什么
上一版(1b-II)最大的短板是**工具调用完全不会**——训练数据里一条工具样本都没有。这一版专门补上了这块:
能力 1b-II **2b-II-pro**
- -- --- ---
工具调用格式 ❌ 全错、字段编造 ✅ 裸 JSON,格式稳定
单步调用 ❌ ✅
多步调用 ❌ ✅
拒绝不该做的事 ⚠️ 时好时坏 ✅
结果解读 ❌ ✅
基座换了**:从 MiniCPM5-1B 换成 MiniCPM5-2B(面壁智能,Apache 2.0,512K 上下文)。
快速开始
MLX(Apple Silicon,本仓库)
- *要在 MLX 上用无损版**(二次量化、继续微调、要质量基准),把模型名换成
whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16 即可(5.03 GB,MLX 直接加载)。
手工拼 prompt 时记得套 chat template 并传 enable thinking=False ,否则会复读乱码。
transformers
本仓库是 **MLX 4bit 格式**,transformers 不能直接加载。
需要 HF safetensors 请用无损版 → whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16 (或等价的 -f16 )。
Ollama(GGUF)
已发布 → whcl412/LycheeAI-coder-2b-II-pro-GGUF
三档可选: q4 k m (1.45 GiB)/ q8 0 (2.50 GiB,推荐)/ f16 (4.69 GiB 无损)。
仓库里带 Modelfile ,下载后 ollama create 即可用。国内可用 ModelScope 镜像。
⚠️ 注意:Ollama 的**自动 tool calls 解析对这个小模型不可靠**——它是按「直接输出 JSON」训练的,不带运行时期望的包裹标签。生产环境请从 content 里自己解析 JSON,详见 GGUF 仓库 README。
工具调用
输出格式
它输出的是**裸 JSON**(OpenAI function calling 风格),不带任何包裹:
没有**
system: 你是 LycheeAI-coder-2b-II-pro。可以使用工具来帮助用户。
可用工具:
get weather(city: string): 查询指定城市的天气
user: 北京今天天气怎么样?
json
{"name": "get weather", "arguments": {"city": "北京"}}
json
{"name": "get weather", "arguments": {"city": "北京"}}
json
{"name": "get weather", "arguments": {"city": "北京"}}
{"name": "get weather", "arguments": {"city": "上海"}}
1234 + 5678 = 2000 (正确 6912)
35 ** 2 = 70 (正确 1225)
- 5 × 4 = 7.8 (正确 54)
199 × 257 = 40043 (正确 51143)
需要任何数值计算时,一律调用 calculate,不要自己心算。
你是一个AI助手,可以使用工具来帮助用户。
可用工具:
get weather(city: str): 查询指定城市的天气
...
需要调用工具时,请直接输出 JSON:{"name": "工具名", "arguments": {...}}
你是 LycheeAI-coder-2b-II-pro,由 MiniCPM5-2B 通过 LoRA 微调而来的编程助手。
你是 LycheeAI-coder-2b-II-pro。可以使用工具来帮助用户。
可用工具:
...
user: 789*123 等于多少
assistant: {"name": "calculate", "arguments": {"expression": "789*123"}}
user: [工具调用结果] {"name": "calculate", "result": {"value": 97047}}
assistant: 789 × 123 = **97047**。
user: 跑一下这段代码
[工具调用结果] {"exit code": 0, "stdout": "45\n"}
assistant: 执行成功,输出 45 。
一头补稀缺:K9 生成 90 条,按稀缺度加权(小数加法最缺 → 补最多)
一头削过剩:对基线里超量的乘法样本下采样,删掉 48 条
bash
pip install mlx-lm
python -m mlx lm.generate \
- -model whcl412/LycheeAI-coder-2b-II-pro \
- -prompt "write a quicksort"
python -m mlx lm.server --model whcl412/LycheeAI-coder-2b-II-pro --port 8080
json
{"name": "get weather", "arguments": {"city": "Beijing"}}
json fences, **no** tags, **no** "Sure, let me call the tool" narration.
Multi-step calls (only the first step is emitted when dependent)
For "Is it a good day to go out in Beijing?", it needs to check the weather first. In that case it **emits only the first step**:
Feed the tool result back and it continues. **Your host must drive the loop**: parse → execute → feed result back → parse again.
Independent calls are emitted in parallel
It refuses clearly when it should
Not every request becomes a tool call. When something is out of scope or disallowed, it says so and offers alternatives instead of fabricating a tool call.
When to give it tools, and when not to
Category Example Behavior
- -- --- ---
Tool gives a definite answer 789*123 , 1000 USD to CNY calls the tool
Out of scope / disallowed booking flights, cracking WiFi refuses + offers alternatives
Tool doesn't help "what is recursion", "write binary search" answers directly
- *Host has no tools** "search for me" in a plain chat app says it can't, doesn't emit JSON
The last row matters**: if your host is a plain chat app, **give it no tool definitions at all**. It will say "I have no web access" instead of emitting JSON your host can't handle.
Training
Item Value
- -- ---
Base openbmb/MiniCPM5-2B
Method LoRA (rank 8, dropout 0, scale 20, 16 layers)
Data 5,341 samples
Epochs 1 (2,671 steps)
LR 5e-5 (adam)
batch / seq 2 / 2048
Framework MLX-LM 0.31.3
Quantization 4-bit affine (group size 64)
⚠️ Known issues
- It does mental arithmetic without tools — and gets it wrong
Root cause** (confirmed by controlled experiment): the model decides based on **operation type**, and this maps exactly onto per-type sample counts in the training data — multiplication 432 samples → calls the tool; addition 86 / power 61 / subtraction 49 → does it in its head.
It is **not** string memorization (38% vs 50%, no real difference), and **not** difficulty ( 1234+5678 also gets mental math).
Workaround**: state it explicitly in the system prompt:
- Identity questions fail under a generic system prompt
If your system prompt is the generic "You are an AI assistant with tools..." style, asking "what's your name" may produce nonsense (observed: "I'm Qwen").
Put the identity in the system prompt:
Cause: every identity sample in the training data used the specific system prompt above; the model never saw the generic-tools-plus-identity combination.
- Tool results come in two shapes — your host should know which it's sending
Both are understood:
Shape A** — an explicit call turn (what mainstream agent frameworks do):
Shape B** — the host already executed it and hands the result straight over (used by 67 early samples):
Difference**: shape B has no name in the result, because there is no call turn in context — the tool identity is inferable from the system tool list and the question itself.
Recommendation**: use **shape A** for normal integration (multi-turn loop). Use shape B only when you already know which tool to run and just want the model to interpret the result.
- Why v6.1 and not v6.4
Five iterations were trained. v6.2 fixed one item and broke two; v6.3 was item-for-item identical to v6.1; v6.4 collapsed the tool-call rate to 12%. **v6.1 is the only version with a real improvement over v6** (25/2 vs 24/3 on a 27-item eval).
Lesson: each round patched ~5% of the data and retrained on everything, reshuffling the whole behavior distribution. The next version will validate on small-scale runs first.
License
Apache 2.0, same as the MiniCPM5-2B base.
LycheeAI —— 小的,但能干活。
LycheeAI — small, but it gets the job done.
优先采用原作者提供的中文模型卡片;仅有英文说明时由闲社AI翻译整理。重要参数、许可证和商用范围请以原文为准。
你将获得什么
完整仓库方式包含该固定版本中的权重、配置、分词器及说明文件;指定版本方式只下载所选文件或整组分片。多模态模型的投影文件、配置等依赖,请按作者说明配齐。
完整仓库可能包含多个权重格式,因此需要的磁盘空间可能大于单一格式的模型大小。下载文件不等于完成模型部署。
使用前须知
运行模型需要的设备、工具与依赖,以原作者说明为准。
是否允许商用、修改和分发,请查看模型许可证。
闲社提供资料与镜像下载方法,不代表已运行模型或完成安全审计。下载的代码应先检查,再执行。
资料或下载有问题?
登录后可提交反馈。
