模型介绍
中文整理DeepSeek-R1-0528-GPTQ-Int4-Int8Mix-Medium
基础型 deepseek-ai/DeepSeek-R1-0528
本仓库提供一份 Int4 + 混合Int8 的 GPTQ 模型：仅将对量化敏感的层保留为 Int8，其余维持 Int4，以尽可能小的体积保有模型的生成质量。
初步试验表明，若直接按 vLLM 默认布局将模型整体转换为纯 Int4（AWQ/GPTQ），推理精度不足，可能导致输出异常。按层精细量化可以显著缓解这一问题。
临时补丁：
vLLM == 0.9.0 目前尚未原生支持 MoE 模块的逐层量化。
我们在 gptq marlin.py 中增补了 get moe quant method 以作过渡。
在官方 PR 合并之前，请先用随仓库附带的文件替换原文件。
版本选择一览
版本 特点 文件大小 适用场景
- -------- --------------------------------- -------- -------------------------------
Lite 仅将最必要的层提升至 Int8，体积接近纯 Int4 355 GB 资源受限、轻量化的服务器部署环境
Compact Int8 层更多，精度更高 414 GB 显存较为充裕、对作答质量要求高（例如 8xA800）
Medium Compact基础上，注意力层全数Int8，精度高、长文损失小 445 GB 显存充裕、追求作答质量与并发能力的均衡（例如 8xH20）
请根据硬件条件与精度需求选择合适版本。
【Vllm 单机（8x141GB）启动命令】
【H200吞吐性能】
- 8 × H200 (141 GB) 、 context = 163840 tokens
concurrent reqs total tok/s tok/s per req
- ---------------- ------------- ---------------
1 60 60.0
50 1350 27.0
100 2200 22.0
200 3400 17.0
400 5100 12.7
- 4 × H200 (141 GB) 、 context = 63840 tokens
concurrent reqs total tok/s tok/s per req
- ---------------- ------------- ---------------
1 56 56.0
50 1100 22.0
100 1700 17.0
200 2600 13.0
400 3900 9.7
【模型更新日期】
【依赖】
【💡gptq marlin.py 补丁💡】
目前 vllm==0.9.0 对于 moe 模块的逐层量化设置没有实现出来，会导致模型加载错误。
我这边简易实现了一下，即在 gptq marlin.py 文件中补充 get moe quant method 函数。
在pr merge之前，请先将附件中的 gptq marlin.py 文件替换至
【模型列表】
文件大小 最近更新时间
- -------- --------------
445GB 2025-06-04
【模型下载】
【介绍】
DeepSeek-R1-0528
Paper Link👁️
- Introduction
The DeepSeek R1 model has undergone a minor version upgrade, with the current version being DeepSeek-R1-0528. In the latest update, DeepSeek R1 has significantly improved its depth of reasoning and inference capabilities by leveraging increased computational resources and introducing algorithmic optimization mechanisms during post-training. The model has demonstrated outstanding performance across various benchmark evaluations, including mathematics, programming, and general logic. Its overall performance is now approaching that of leading models, such as O3 and Gemini 2.5 Pro.
Compared to the previous version, the upgraded model shows significant improvements in handling complex reasoning tasks. For instance, in the AIME 2025 test, the model’s accuracy has increased from 70% in the previous version to 87.5% in the current version. This advancement stems from enhanced thinking depth during the reasoning process: in the AIME test set, the previous model used an average of 12K tokens per question, whereas the new version averages 23K tokens per question.
Beyond its improved reasoning capabilities, this version also offers a reduced hallucination rate, enhanced support for function calling, and better experience for vibe coding.
- Evaluation Results
DeepSeek-R1-0528
For all our models, the maximum generation length is set to 64K tokens. For benchmarks requiring sampling, we use a temperature of $0.6$, a top-p value of $0.95$, and generate 16 responses per query to estimate pass@1.
Category Benchmark (Metric) DeepSeek R1 DeepSeek R1 0528
- --------- ---------------------------------- ----------------- ---
General
MMLU-Redux (EM) 92.9 93.4
MMLU-Pro (EM) 84.0 85.0
GPQA-Diamond (Pass@1) 71.5 81.0
SimpleQA (Correct) 30.1 27.8
FRAMES (Acc.) 82.5 83.0
Humanity's Last Exam (Pass@1) 8.5 17.7
Code
LiveCodeBench (2408-2505) (Pass@1) 63.5 73.3
Codeforces-Div1 (Rating) 1530 1930
SWE Verified (Resolved) 49.2 57.6
Aider-Polyglot (Acc.) 53.3 71.6
Math
AIME 2024 (Pass@1) 79.8 91.4
AIME 2025 (Pass@1) 70.0 87.5
HMMT 2025 (Pass@1) 41.7 79.4
CNMO 2024 (Pass@1) 78.8 86.9
Tools
BFCL v3 MultiTurn (Acc) - 37.0
Tau-Bench (Pass@1) - 53.5(Airline)/63.9(Retail)
Note: We use Agentless framework to evaluate model performance on SWE-Verified. We only evaluate text-only prompts in HLE testsets. GPT-4.1 is employed to act user role in Tau-bench evaluation.
- License
This code repository is licensed under MIT License. The use of DeepSeek-R1 models is also subject to MIT License. DeepSeek-R1 series (including Base and Chat) supports commercial use and distillation.
- Citation
- Contact
If you have any questions, please raise an issue or contact us at service@deepseek.com.
优先采用原作者提供的中文模型卡片；仅有英文说明时由闲社AI翻译整理。重要参数、许可证和商用范围请以原文为准。
你将获得什么
完整仓库方式包含该固定版本中的权重、配置、分词器及说明文件；指定版本方式只下载所选文件或整组分片。多模态模型的投影文件、配置等依赖，请按作者说明配齐。
完整仓库可能包含多个权重格式，因此需要的磁盘空间可能大于单一格式的模型大小。下载文件不等于完成模型部署。
使用前须知
运行模型需要的设备、工具与依赖，以原作者说明为准。
是否允许商用、修改和分发，请查看模型许可证。
闲社提供资料与镜像下载方法，不代表已运行模型或完成安全审计。下载的代码应先检查，再执行。
Problem with the information or download?
Sign in to submit feedback.