All systems operationalAI Agent Automation Platform
English
Language简体中文EnglishSelected

Choose your language

moss_nano_tts_onnx_100M_INT8

hzp2000itb/moss_nano_tts_onnx_100M_INT8

Publisherhzp2000itb

Download access requirementsModelScope source

MOSS-TTS-Nano ONNX 这是一个基于 MOSS-TTS-Nano 的本地 ONNX 推理仓库,面向 CPU 部署场景,支持: 基于参考声纹的 voice clone

Speech & Audio
Model details

模型介绍

中文整理

MOSS-TTS-Nano ONNX

中文 English

中文说明

简介

这是一个基于 MOSS-TTS-Nano 的本地 ONNX 推理仓库,面向 CPU 部署场景,支持:

文本转语音(TTS)

基于参考声纹的 voice clone

LLM token 流到句子级语音的流式合成

INT8 KV-Cache 推理

通道跳过与动态填充优化

本仓库模型基于 ModelScope 上的 openmoss/MOSS-TTS-Nano-100M 进行 INT8 量化,并导出为 ONNX 格式。

原始模型虽然仅有 0.1B 规模,但在 CPU 上即使经过常规推理加速,RTF 仍然难以稳定达到小于 1 的水平,因此在实时流式场景下仍难以达到可实用的工程标准。

本仓库中的量化与推理优化策略由作者独立设计实现,重点面向低资源 CPU 场景下的时延与吞吐优化,目标是进一步降低 CPU 推理成本并提升实时流式语音合成的可用性。

特性

主模型拆分为 global prefill / global step / local prefill / local step

音频 tokenizer 编码器与解码器采用 ONNX 部署

支持声纹缓存复用,减少重复编码开销

支持流式文本输入触发句子级增量语音生成

支持非关键通道跳过与中位数/动态填充两种优化策略

目录结构

环境要求

Python 3.10+

Linux / macOS / Windows

CPU 环境可运行

推荐使用 onnxruntime CPU provider

安装依赖

Quick start

  • 使用已有声纹缓存做单次合成
  • 当声纹缓存不存在时,用你自己的参考音频编码声纹
  • 模拟 LLM 流式输出并做流式语音生成
  • 保存每段流式音频分片

Python 调用示例

如果目标声纹缓存不存在,则必须显式提供 ref audio 。

核心文件说明

moss tts handler.py :面向应用集成的主处理器接口

simulate llm stream tts.py :模拟 LLM 文本流并做流式合成

tts inference with cache onnx optimized.py :单次推理与性能优化示例

onnx/tts nano/config.json :主模型结构与 token 配置

onnx/audio tokenizer/config.json :音频 tokenizer 配置

onnx/voice prints/*.npz :声纹缓存文件

量化与推理优化

本仓库包含以下优化方向:

INT8 ONNX 导出与部署

Global / Local 双层推理路径拆分

KV-Cache 推理缓存

非关键音频通道跳过

中位数填充与动态填充策略

可调 ONNX Runtime 线程参数

这些策略的目标是降低 CPU 推理耗时,并在音质与速度之间取得工程上可用的平衡。

优化动机 / 方法概述 / 效果目标

优化动机在于:原始 0.1B TTS 模型即使模型规模不大,在 CPU 场景下的主要瓶颈仍然集中在自回归 step 路径、多音频通道逐帧采样、局部 Transformer 的重复计算,以及未针对小核低并发推理负载细化的 ONNX Runtime 执行配置。这些问题会直接推高单句合成时延与流式输出间隔,使实时流式场景难以达到稳定可用的工程标准。

方法上,本仓库围绕推理链路做了面向 CPU 的结构化优化:首先将主模型导出为 INT8 ONNX,并拆分为 global prefill 、 global step 、 local prefill 、 local step 四条执行路径,以降低不同阶段的计算与缓存开销;其次在自回归生成阶段引入 KV-Cache 与局部 attention cache,减少重复状态构建;同时基于音频通道重要性分析,对非关键通道采用跳过生成策略,并结合中位数填充或动态填充,在尽量控制音质退化的前提下显著减少 local step 调用次数;此外还针对 prefill、step、decoder 分别暴露独立线程参数,使 ONNX Runtime 的线程模型更贴合 CPU 推理实际负载。

效果目标上,这套方法并不追求单纯的模型压缩,而是面向实时流式语音合成的整体工程可用性优化:在保持可接受音质与声纹一致性的前提下,尽可能压低 CPU 侧合成时延、改善吞吐与 RTF 表现,并推动系统更接近实时流式交互所需的实用部署区间。

使用限制与免责声明

本仓库仅用于研究、学习、评估与合规场景下的语音合成应用开发

使用 voice clone 功能时,你必须拥有参考音频的合法授权

禁止将本项目用于身份冒充、诈骗、侵权、伪造证据或其他违法违规用途

仓库中公开的声纹缓存不代表授权你克隆任意真实人物声音

使用者需自行承担合规与法律责任

License

本项目代码与仓库发布内容采用 Apache-2.0 。

请注意:如果你后续添加第三方样本、参考音频或外部权重,需要自行确认这些资产的再分发许可。

English

Overview

This repository provides a local ONNX inference setup for MOSS-TTS-Nano, optimized for CPU deployment scenarios. It supports:

text-to-speech (TTS)

voice cloning with reference voice prints

streaming synthesis from LLM text token streams

INT8 KV-cache inference

channel skipping and dynamic filling optimizations

The model in this repository is based on openmoss/MOSS-TTS-Nano-100M from ModelScope, quantized to INT8 precision, and exported to ONNX format.

Although the original model is only 0.1B in size, its real-time factor (RTF) on CPU is still difficult to keep consistently below 1 even with conventional inference acceleration, which makes it hard to reach a practical engineering level for real-time streaming scenarios.

The quantization and inference optimization strategy in this repository is independently designed and implemented by the author, focusing on low-resource CPU inference efficiency and aiming to further improve the practicality of real-time streaming speech synthesis.

Features

split inference path with global prefill , global step , local prefill , and local step

ONNX-based audio tokenizer encoder and decoder

reusable cached voice prints

sentence-level incremental synthesis for streaming text input

non-critical channel skipping with median or dynamic fill strategies

Requirements

Python 3.10+

Linux / macOS / Windows

CPU runtime supported

recommended runtime: onnxruntime CPU provider

Installation

Quick Start

  • Synthesize with an existing cached voice print
  • Encode your own reference audio when the cache does not exist
  • Simulate LLM streaming text and generate streaming audio

File Guide

moss tts handler.py : main application-facing TTS handler

simulate llm stream tts.py : streaming demo driven by simulated LLM text output

tts inference with cache onnx optimized.py : one-shot inference and optimization demo

onnx/tts nano/config.json : main model configuration

onnx/audio tokenizer/config.json : audio tokenizer configuration

onnx/voice prints/*.npz : cached voice print assets

Optimization Notes

This repository includes:

INT8 ONNX deployment

global/local dual-path inference

KV-cache acceleration

non-critical audio channel skipping

median fill and dynamic fill strategies

tunable ONNX Runtime threading parameters

These optimizations are intended to reduce CPU inference latency while preserving practical synthesis quality.

Motivation / Method / Goal

The optimization motivation is straightforward: although the original 0.1B TTS model is relatively small, the main CPU bottlenecks still come from the autoregressive step path, frame-by-frame multi-channel audio token sampling, repeated local Transformer computation, and generic ONNX Runtime settings that are not tuned for low-concurrency CPU inference workloads. In practice, these factors increase sentence-level latency and streaming gaps, making real-time deployment difficult.

Methodologically, this repository applies a structured CPU-oriented optimization pipeline. The model is first quantized to INT8 and exported to ONNX, then split into global prefill , global step , local prefill , and local step execution paths to reduce stage-specific compute and cache overhead. During autoregressive generation, KV-cache and local attention cache are used to avoid redundant state construction. In addition, non-critical audio channels are skipped based on channel importance analysis, with median-fill or dynamic-fill strategies used to trade a limited quality loss for a significant reduction in local step calls. The repository also exposes separate threading controls for prefill, step, and decoder stages so that ONNX Runtime can better match real CPU inference behavior.

The target is not merely model compression, but practical deployment quality for real-time streaming TTS. Under acceptable synthesis quality and voice consistency, the goal is to reduce CPU-side latency, improve throughput and RTF behavior, and move the system closer to a practically usable operating range for interactive streaming applications.

Distribution Notes

For Hugging Face and ModelScope publishing:

large files should be managed with Git LFS

  • .onnx , *.npz , and *.wav are covered by .gitattributes

only cached voice print files are intended for publication; raw reference audio is not included by default

Responsible Use

This repository is intended for research, evaluation, education, and compliant application development

You must have legal authorization for any reference audio used in voice cloning

Do not use this project for impersonation, fraud, infringement, fabricated evidence, or unlawful activity

Published cached voice prints do not grant permission to clone arbitrary real individuals

The user is responsible for legal and policy compliance

License

This repository is released under Apache-2.0 .

If you later add third-party samples, reference audio, or external weights, please verify their redistribution terms separately.

优先采用原作者提供的中文模型卡片;仅有英文说明时由闲社AI翻译整理。重要参数、许可证和商用范围请以原文为准。

你将获得什么

完整仓库方式包含该固定版本中的权重、配置、分词器及说明文件;指定版本方式只下载所选文件或整组分片。多模态模型的投影文件、配置等依赖,请按作者说明配齐。

完整仓库可能包含多个权重格式,因此需要的磁盘空间可能大于单一格式的模型大小。下载文件不等于完成模型部署。

使用前须知

运行环境

运行模型需要的设备、工具与依赖,以原作者说明为准。

授权范围

是否允许商用、修改和分发,请查看模型许可证。

闲社提供资料与镜像下载方法,不代表已运行模型或完成安全审计。下载的代码应先检查,再执行。

Back to top