llama-3-chinese-8b-gguf

ChineseAlpacaGroup/llama-3-chinese-8b-gguf

PublisherChineseAlpacaGroup

Download access requirementsModelScope source

Llama-3-Chinese-8B-GGUF 提醒: GGUF文件已重新生成，由于llama.cpp仍然有可能对其作出修改，因此建议保留HF版模型。这个仓库包含了Llama-3-Chinese-8B-GGUF（兼容llama.cpp/ollama等），是Llama-3-Chinese-8B模型的量化版本。

Text Processing

View all download instructions Mirror repository documentation and license ↗

Model details

模型介绍

中文整理

Llama-3-Chinese-8B-GGUF

提醒: GGUF文件已重新生成，由于llama.cpp仍然有可能对其作出修改，因此建议保留HF版模型。

这个仓库包含了**Llama-3-Chinese-8B-GGUF**（兼容llama.cpp/ollama等），是Llama-3-Chinese-8B模型的量化版本。

注意：这是一个基座模型，不适用于对话、问答等任务。**

更多细节（性能、使用方法等）请参考GitHub项目页面：https://github.com/ymcui/Chinese-LLaMA-Alpaca-3

量化性能

评测指标：PPL，**越低越好**

Quant Size PPL (old model) 👍🏻 PPL (new model)

:---: -------: ------------------: ------------------:

Q2 K 2.96 GB 17.7212 +/- 0.59814 11.8595 +/- 0.20061

Q3 K 3.74 GB 8.6303 +/- 0.28481 5.7559 +/- 0.09152

Q4 0 4.34 GB 8.2513 +/- 0.27102 5.5495 +/- 0.08832

Q4 K 4.58 GB 7.8897 +/- 0.25830 5.3126 +/- 0.08500

Q5 0 5.21 GB 7.7975 +/- 0.25639 5.2222 +/- 0.08317

Q5 K 5.34 GB 7.7062 +/- 0.25218 5.1813 +/- 0.08264

Q6 K 6.14 GB 7.6600 +/- 0.25043 5.1481 +/- 0.08205

Q8 0 7.95 GB 7.6512 +/- 0.25064 5.1350 +/- 0.08190

F16 14.97 GB 7.6389 +/- 0.25001 5.1302 +/- 0.08184

其他

完整模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b

LoRA模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-lora

关于本模型的提问，请通过 https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 提交issue

This repository contains **Llama-3-Chinese-8B-GGUF** (llama.cpp/ollama/tgw, etc. compatible), which is the quantized version of Llama-3-Chinese-8B.

Note: this is a foundation model, which is not suitable for conversation, QA, etc.**

Further details (performance, usage, etc.) should refer to GitHub project page: https://github.com/ymcui/Chinese-LLaMA-Alpaca-3

Performance

Metric: PPL, **lower is better**

The model name with -im suffix is generated with important matrix, which has generally better performance.

Quant Size PPL (old model) 👍🏻 PPL (new model)

:---: -------: ------------------: ------------------:

Q2 K 2.96 GB 17.7212 +/- 0.59814 11.8595 +/- 0.20061

Q3 K 3.74 GB 8.6303 +/- 0.28481 5.7559 +/- 0.09152

Q4 0 4.34 GB 8.2513 +/- 0.27102 5.5495 +/- 0.08832

Q4 K 4.58 GB 7.8897 +/- 0.25830 5.3126 +/- 0.08500

Q5 0 5.21 GB 7.7975 +/- 0.25639 5.2222 +/- 0.08317

Q5 K 5.34 GB 7.7062 +/- 0.25218 5.1813 +/- 0.08264

Q6 K 6.14 GB 7.6600 +/- 0.25043 5.1481 +/- 0.08205

Q8 0 7.95 GB 7.6512 +/- 0.25064 5.1350 +/- 0.08190

F16 14.97 GB 7.6389 +/- 0.25001 5.1302 +/- 0.08184

Others

For full model, please see: https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b

For LoRA-only model, please see: https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-lora

If you have questions/issues regarding this model, please submit an issue through https://github.com/ymcui/Chinese-LLaMA-Alpaca-3

优先采用原作者提供的中文模型卡片；仅有英文说明时由闲社AI翻译整理。重要参数、许可证和商用范围请以原文为准。

你将获得什么

完整仓库方式包含该固定版本中的权重、配置、分词器及说明文件；指定版本方式只下载所选文件或整组分片。多模态模型的投影文件、配置等依赖，请按作者说明配齐。

完整仓库可能包含多个权重格式，因此需要的磁盘空间可能大于单一格式的模型大小。下载文件不等于完成模型部署。

使用前须知

运行环境

运行模型需要的设备、工具与依赖，以原作者说明为准。

授权范围

是否允许商用、修改和分发，请查看模型许可证。

闲社提供资料与镜像下载方法，不代表已运行模型或完成安全审计。下载的代码应先检查，再执行。

Problem with the information or download?

Pinned version · Multiple download options

llama-3-chinese-8b-gguf model download

Download the full repository or choose a version or file.

Download tools · More than GGUF

Use the Hugging Face download tool to obtain ChineseAlpacaGroup/llama-3-chinese-8b-gguf. Download Safetensors, GGUF, ONNX, configuration and other repository files. Downloading does not mean the tool can run this model.

Pinned-version file information is being completed. Existing downloads remain available, or load the file manifest below.

Choose a mirror or official source, your operating system and download scope below, then copy the command. Install the download tool first if needed.

Pinned version: abb5ad3ecc074c3ce2baac12590951df4d5c55d3. Downloads files only; does not start the model.

Choose a download source

Pinned ModelScope version: abb5ad3ecc074c3ce2baac12590951df4d5c55d3. Version identifiers can differ across sites. The alternative source was provided after checking weights and runtime configuration; this is not proof of a successful model run.

A checked, pinned ModelScope source is available for this model. Use the ModelScope download tool.

Choose what to download

Need just one GGUF precision? Load the file manifest and choose a version. Direct file links do not require Python; the command below can download multiple shards together.

Quantization / file

This reads the pinned version's file manifest; no model weights are downloaded.

Choose your operating system

For command-line downloads, follow these steps to get the full repository or selected files. Direct file links do not need these steps.

Open a terminal window

Create and open a folder for the model. In the folder's address bar, enter powershell and press Enter.

If Python is not installed, get it from the official Python website. In the terminal, enter python --version; continue when a version number appears.

First time? Install the download tool

Copy this command, paste it into the terminal, press Enter and wait for installation to finish.

python -m pip install --upgrade huggingface_hub

Already installed? Continue to step 3.

Run the download command

Select the button below, paste into the same terminal and press Enter. The tool downloads all files included in the current command.

Copying does not start a download. Paste and run the command on your device.

Where are the downloaded files?

Open the folder you chose. Look for the models folder. When progress stops and the terminal returns to its prompt without errors, check that all files are present. 'Copied' does not mean 'downloaded'.

No response or an error?

python not found: Install Python and reopen the terminal. On Windows, follow the installer's instructions to enable command-line access.

hf not found: Confirm that step 2 succeeded, then reopen the terminal. If needed, add Python's Scripts folder to your user PATH. Do not repeatedly download scripts.

Network interrupted: Stay in the same folder and run the same command again. The tool checks existing files.

Permission or disk error: Choose a writable folder with enough free space. Do not disable system security protection.

View the full download command

modelscope download --model "ChineseAlpacaGroup/llama-3-chinese-8b-gguf" --revision "abb5ad3ecc074c3ce2baac12590951df4d5c55d3" --local_dir "./models/ChineseAlpacaGroup--llama-3-chinese-8b-gguf--ms-abb5ad3ecc07"

Files transfer directly from the source you choose, not through Xianshe servers.

The version is pinned. Repeating the same command checks existing files. Change the source manually if it is slow or unavailable.

Public models do not require submitting keys to Xianshe. Restricted models still require the author's authorization. On Linux / macOS, use python3 instead of python when appropriate.