Tongyi-DeepResearch-30B-A3B-Int4-W4A16

通义深度研究 DeepResearch-30B-A3B-Int4-W4A16 量化高精校准

原模型 iic/Tongyi-DeepResearch-30B-A3B

📖 关于量化损失方面的研究，可阅读公众号“觉察流”文章👇

《Reason Model 的“瘦身计划”：量化技术的得与失》

📖 关于通义DeepResearch方面的论文，可阅读公众号“觉察流”系列文章👇

文章合集：《【概述】通义DeepResearch开源发布：首个匹敌OpenAI的全栈Web Agent（七篇6.5万字）》

• 《通义 DeepResearch 开源概述》
• 《WebResearcher：从线性累积到迭代进化，重塑AI研究范式的三大支柱》
• 《ReSum：让AI智能体学会“记笔记”，突破长程推理瓶颈》
• 《WebWeaver：一个模仿人类思维的AI研究员是如何炼成的？》
• 《超越规模神话：WebSailor-V2 的数据-环境协同之道》
• 《从AgentFounder看未来：智能体基础模型的崛起与"持续预训练"新范式》
• 《环境即智能：从AgentScaler看下一代AI智能体的进化论》

作者在此 👇🏻 扫一扫

Download

SDK下载

Git下载

模型概述

Tongyi-DeepResearch-30B-A3B-Int4-W4A16 是一个基于 通义千问 Tongyi-DeepResearch-30B-A3B 的 INT4 量化并校准的模型。

• *模型名称:** Tongyi-DeepResearch-30B-A3B-Int4-W4A16
• *模型架构:** Qwen3 MoE
• *权重量化:** INT4

该模型通过将 iic/Tongyi-DeepResearch-30B-A3B 的权重量化为 INT4 数据类型而实现。

量化过程将每个参数从 16bit 减少到 4bit，将模型占用磁盘空间大小，以及推理时加载模型需要的GPU显存空间，减少到了大约为原模型的1/4。只有 transformer 中的 Linear 层权重是量化的，其他层均保持为原模型数据类型，采用了混合精度的计算方式，尽可能减少模型量化后所带来的精度损失。

量化本模型，使用了AutoGPTQ，采用 Symmetric group-wise 方式量化（group size : 128）。并在量化过程中，进行了数据校准，以提升模型生成精度。

注意：本模型需要 compute capability > 8.0（Ampere、Ada Lovelace、Hopper 架构）的 Nvidia GPU 来支持 INT4 **混合精度**计算。

部署推理

推荐使用 vLLM>=0.10.2 ( transformers>=4.56 )

Openai api 兼容模式

附录 Tongyi-DeepResearch-30B-A3B 介绍

We present **Tongyi DeepResearch**, an agentic large language model featuring 30 billion total parameters, with only 3 billion activated per token. Developed by Tongyi Lab, the model is specifically designed for **long-horizon, deep information-seeking** tasks. Tongyi-DeepResearch demonstrates state-of-the-art performance across a range of agentic search benchmarks, including BrowserComp-EN, BrowserComp-ZH, GAIA, Humanity's Last Exam, xbench-DeepSearch, and WebWalkerQA.

Key Features

⚙️ **Fully automated synthetic data generation pipeline**: We design a highly scalable data synthesis pipeline, which is fully automatic and empowers agentic pre-training, supervised fine-tuning, and reinforcement learning.

🔄 **Large-scale continual pre-training on agentic data**: Leveraging diverse, high-quality agentic interaction data to extend model capabilities, maintain freshness, and strengthen reasoning performance.

🔁 **End-to-end reinforcement learning**: We employ a strictly on-policy RL approach based on a customized Group Relative Policy Optimization framework, with token-level policy gradients, leave-one-out advantage estimation, and selective filtering of negative samples to stabilize training in a non‑stationary environment.

🤖 **Agent Inference Paradigm Compatibility**: At inference, Tongyi-DeepResearch is compatible with two inference paradigms: ReAct, for rigorously evaluating the model's core intrinsic abilities, and an IterResearch-based 'Heavy' mode, which uses a test-time scaling strategy to unlock the model's maximum performance ceiling.

Download

You can download the model then run the inference scipts in https://github.com/Alibaba-NLP/DeepResearch.