闲社服务运行正常AI智能体自动化平台
中文
语言 / Language简体中文SelectedEnglish

选择显示语言

DeepSeek-R1-0528-Qwen3-8B-Int8-W8A16

QuantTrio/DeepSeek-R1-0528-Qwen3-8B-Int8-W8A16

发布机构QuantTrio

下载访问条件魔搭来源

重要:友情提醒,推理本模型时,请按照如下官方指引: 对于思考模式(enable thinking=True, 默认为 True) 请使用 Temperature=0.6 、 TopP=0.95 、 TopK=20 和 MinP=0 ( generation config.json 中的默认设置)。请勿使用贪婪解码(greedy decoding),因为它可能导致性能下降和无限循环。如需更详细的指导,请参阅最佳实践部分。

文字对话
模型详情

模型介绍

中文整理

重要:友情提醒,推理本模型时,请按照如下官方指引:

对于思考模式(enable thinking=True, 默认为 True)

请使用 Temperature=0.6 、 TopP=0.95 、 TopK=20 和 MinP=0 ( generation config.json 中的默认设置)。请勿使用贪婪解码(greedy decoding),因为它可能导致性能下降和无限循环。如需更详细的指导,请参阅最佳实践部分。

对于非思考模式(enable thinking=False)

建议使用 Temperature=0.7 、 TopP=0.8 、 TopK=20 和 MinP=0 。如需更详细的指导,请参阅最佳实践部分。

📖 关于量化损失方面的研究,可阅读公众号“觉察流”文章👇

《Reason Model 的“瘦身计划”:量化技术的得与失》

DeepSeek-R1-0528-Qwen3-8B-Int8-W8A16 量化高精校准

原模型 deepseek-ai/DeepSeek-R1-0528-Qwen3-8B

作者在此 👇🏻 扫一扫

下载

SDK下载

Git下载

模型概述

DeepSeek-R1-0528-Qwen3-8B-Int8-W8A16 是一个基于 DeepSeek-R1-0528-Qwen3-8B 的 INT8 量化并校准的模型。

  • *模型名称:** DeepSeek-R1-0528-Qwen3-8B-Int8-W8A16
  • *模型架构:** Qwen3
  • *权重量化:** INT8

该模型通过将 deepseek-ai/DeepSeek-R1-0528-Qwen3-8B 的权重量化为 INT8 数据类型而实现。

量化过程将每个参数从 16bit 减少到 8bit,将模型占用磁盘空间大小,以及推理时加载模型需要的GPU显存空间,减少到了大约为原模型的1/2。

量化过程中,只有 transformer 中的 Linear 层权重是量化的,其他层均保持为原模型数据类型,采用混合精度的计算方式,尽可能减少模型量化后所带来的精度损失。

在量化过程中,使用了AutoGPTQ,采用 Symmetric group-wise 方式量化。并在此次量化过程中,进行了数据校准,以提升模型生成精度。(相对比 BF16 做到几乎无损)

注意:本模型需要 compute capability > 8.0(Ampere、Ada Lovelace、Hopper 架构)的 Nvidia GPU 来支持 INT8 **混合精度**计算。

部署推理

推荐使用 vLLM>=0.8.4 ( transformers>=4.51.0 )

Openai api 兼容模式

附录 DeepSeek-R1-0528-Qwen3-8B 介绍

DeepSeek-R1-0528

Paper Link👁️

  • Introduction

The DeepSeek R1 model has undergone a minor version upgrade, with the current version being DeepSeek-R1-0528. In the latest update, DeepSeek R1 has significantly improved its depth of reasoning and inference capabilities by leveraging increased computational resources and introducing algorithmic optimization mechanisms during post-training. The model has demonstrated outstanding performance across various benchmark evaluations, including mathematics, programming, and general logic. Its overall performance is now approaching that of leading models, such as O3 and Gemini 2.5 Pro.

Compared to the previous version, the upgraded model shows significant improvements in handling complex reasoning tasks. For instance, in the AIME 2025 test, the model’s accuracy has increased from 70% in the previous version to 87.5% in the current version. This advancement stems from enhanced thinking depth during the reasoning process: in the AIME test set, the previous model used an average of 12K tokens per question, whereas the new version averages 23K tokens per question.

Beyond its improved reasoning capabilities, this version also offers a reduced hallucination rate, enhanced support for function calling, and better experience for vibe coding.

  • Evaluation Results

DeepSeek-R1-0528

For all our models, the maximum generation length is set to 64K tokens. For benchmarks requiring sampling, we use a temperature of $0.6$, a top-p value of $0.95$, and generate 16 responses per query to estimate pass@1.

Category Benchmark (Metric) DeepSeek R1 DeepSeek R1 0528

  • --------- ---------------------------------- ----------------- ---

General

MMLU-Redux (EM) 92.9 93.4

MMLU-Pro (EM) 84.0 85.0

GPQA-Diamond (Pass@1) 71.5 81.0

SimpleQA (Correct) 30.1 27.8

FRAMES (Acc.) 82.5 83.0

Humanity's Last Exam (Pass@1) 8.5 17.7

Code

LiveCodeBench (2408-2505) (Pass@1) 63.5 73.3

Codeforces-Div1 (Rating) 1530 1930

SWE Verified (Resolved) 49.2 57.6

Aider-Polyglot (Acc.) 53.3 71.6

Math

AIME 2024 (Pass@1) 79.8 91.4

AIME 2025 (Pass@1) 70.0 87.5

HMMT 2025 (Pass@1) 41.7 79.4

CNMO 2024 (Pass@1) 78.8 86.9

Tools

BFCL v3 MultiTurn (Acc) - 37.0

Tau-Bench (Pass@1) - 53.5(Airline)/63.9(Retail)

Note: We use Agentless framework to evaluate model performance on SWE-Verified. We only evaluate text-only prompts in HLE testsets. GPT-4.1 is employed to act user role in Tau-bench evaluation.

DeepSeek-R1-0528-Qwen3-8B

Meanwhile, we distilled the chain-of-thought from DeepSeek-R1-0528 to post-train Qwen3 8B Base, obtaining DeepSeek-R1-0528-Qwen3-8B. This model achieves state-of-the-art (SOTA) performance among open-source models on the AIME 2024, surpassing Qwen3 8B by +10.0% and matching the performance of Qwen3-235B-thinking. We believe that the chain-of-thought from DeepSeek-R1-0528 will hold significant importance for both academic research on reasoning models and industrial development focused on small-scale models.

AIME 24 AIME 25 HMMT Feb 25 GPQA Diamond LiveCodeBench (2408-2505)

  • ------------------------------- --------- --------- ------------- -------------- ---------------------------

Qwen3-235B-A22B 85.7 81.5 62.5 71.1 66.5

Qwen3-32B 81.4 72.9 - 68.4 -

Qwen3-8B 76.0 67.3 - 62.0 -

Phi-4-Reasoning-Plus-14B 81.3 78.0 53.6 69.3 -

Gemini-2.5-Flash-Thinking-0520 82.3 72.0 64.2 82.8 62.3

o3-mini (medium) 79.6 76.7 53.3 76.8 65.9

DeepSeek-R1-0528-Qwen3-8B 86.0 76.3 61.5 61.1 60.5

  • Chat Website & API Platform

You can chat with DeepSeek-R1 on DeepSeek's official website: chat.deepseek.com, and switch on the button "DeepThink"

We also provide OpenAI-Compatible API at DeepSeek Platform: platform.deepseek.com

  • How to Run Locally

Please visit DeepSeek-R1 repository for more information about running DeepSeek-R1-0528 locally.

Compared to previous versions of DeepSeek-R1, the usage recommendations for DeepSeek-R1-0528 have the following changes:

  • System prompt is supported now.
  • It is not required to add "\\n" at the beginning of the output to force the model into thinking pattern.

The model architecture of DeepSeek-R1-0528-Qwen3-8B is identical to that of Qwen3-8B, but it shares the same tokenizer configuration as DeepSeek-R1-0528. This model can be run in the same manner as Qwen3-8B, but it is essential to ensure that all configuration files are sourced from our repository rather than the original Qwen3 project.

System Prompt

In the official DeepSeek web/app, we use the same system prompt with a specific date.

For example,

Temperature

In our web and application environments, the temperature parameter $T {model}$ is set to 0.6.

Prompts for File Uploading and Web Search

For file uploading, please follow the template to create prompts, where {file name}, {file content} and {question} are arguments.

For Web Search, {search results}, {cur date}, and {question} are arguments.

For Chinese query, we use the prompt:

For English query, we use the prompt:

优先采用原作者提供的中文模型卡片;仅有英文说明时由闲社AI翻译整理。重要参数、许可证和商用范围请以原文为准。

你将获得什么

完整仓库方式包含该固定版本中的权重、配置、分词器及说明文件;指定版本方式只下载所选文件或整组分片。多模态模型的投影文件、配置等依赖,请按作者说明配齐。

完整仓库可能包含多个权重格式,因此需要的磁盘空间可能大于单一格式的模型大小。下载文件不等于完成模型部署。

使用前须知

运行环境

运行模型需要的设备、工具与依赖,以原作者说明为准。

授权范围

是否允许商用、修改和分发,请查看模型许可证。

闲社提供资料与镜像下载方法,不代表已运行模型或完成安全审计。下载的代码应先检查,再执行。

返回顶部