MiniCPM-o-4_5

A Gemini 2.5 Flash Level MLLM for Vision, Speech, and Full-Duplex Mulitmodal Live Streaming on Your Phone

GitHub CookBook Omni-modal Demo Vision-Language Demo Technical Report

WeChat Discord MiniCPM Wiki(Chinese) CaseBook(Audio, Omni Full-Duplex)

News

[!NOTE]

[2026.02.06] 🥳 🥳 🥳 We open-sourced a realtime web demo deployable on your own devices like Mac or GPU. Try it now!

MiniCPM-o 4.5

MiniCPM-o 4.5** is the latest and most capable model in the MiniCPM-o series. The model is built in an end-to-end fashion based on SigLip2, Whisper-medium, CosyVoice2, and Qwen3-8B with a total of 9B parameters. It exhibits a significant performance improvement, and introduces new features for full-duplex multimodal live streaming. Notable features of MiniCPM-o 4.5 include:

🔥 **Leading Visual Capability.**

MiniCPM-o 4.5 achieves an average score of 77.6 on OpenCompass, a comprehensive evaluation of 8 popular benchmarks. **With only 9B parameters, it surpasses widely used proprietary models like GPT-4o, Gemini 2.0 Pro, and approaches Gemini 2.5 Flash** for vision-language capabilities. It supports instruct and thinking modes in a single model, better covering efficiency and performance trade-offs in different user scenarios.

🎙 **Strong Speech Capability.**

MiniCPM-o 4.5 supports **bilingual real-time speech conversation with configurable voices** in English and Chinese. It features **more natural, expressive and stable speech conversation**. The model also allows for fun features such as **voice cloning and role play via a simple reference audio clip**, where the cloning performance surpasses strong TTS tools such as CosyVoice2.

🎬 **New Full-Duplex and Proactive Multimodal Live Streaming Capability.**

As a new feature, MiniCPM-o 4.5 can process real-time, continuous video and audio input streams simultaneously while generating concurrent text and speech output streams in an end-to-end fashion, without mutual blocking. This **allows MiniCPM-o 4.5 to see, listen, and speak simultaneously**, creating a fluid, real-time omnimodal conversation experience. Beyond reactive responses, the model can also perform **proactive interaction**, such as initiating reminders or comments based on its continuous understanding of the live scene.

💪 **Strong OCR Capability, Efficiency and Others.**

Advancing popular visual capabilities from MiniCPM-V series, MiniCPM-o 4.5 can process **high-resolution images** (up to 1.8 million pixels) and **high-FPS videos** (up to 10fps) in any aspect ratio efficiently. It achieves **state-of-the-art performance for end-to-end English document parsing** on OmniDocBench, outperforming proprietary models such as Gemini-3 Flash and GPT-5, and specialized tools such as DeepSeek-OCR 2. It also features **trustworthy behaviors**, matching Gemini 2.5 Flash on MMHal-Bench, and supports **multilingual capabilities** on more than 30 languages.

💫 **Easy Usage.**

MiniCPM-o 4.5 can be easily used in various ways: **Basic usage, recommended for 100% precision:** PyTorch inference with Nvidia GPU. **Other end-side adaptation** includes (1) llama.cpp and Ollama support for efficient CPU inference on local devices, (2) int4 and GGUF format quantized models in 16 sizes, (3) vLLM and SGLang support for high-throughput and memory-efficient inference, (4) FlagOS support for the unified multi-chip backend plugin. **We also open-sourced web demos** on which **enables the full-duplex multimodal live streaming experience on local devices** such as GPUs, PCs (e.g., on a MacBook).

Model Architecture.**

• *End-to-end Omni-modal Architecture.** The modality encoders/decoders and LLM are densely connected via hidden states in an end-to-end fashion. This enables better information flow and control, and also facilitates full exploitation of rich multimodal knowledge during training.
• *Full-Duplex Omni-modal Live Streaming Mechanism.** (1) We turn the offline modality encoder/decoders into online and full-duplex ones for streaming inputs/outputs. The speech token decoder models text and speech tokens in an interleaved fashion to support full-duplex speech generation (i.e., sync timely with new input). This also facilitates more stable long speech generation (e.g., > 1min).

(2) **We sync all the input and output streams on timeline in milliseconds**, which are jointly modeled by a time-division multiplexing (TDM) mechanism for omni-modality streaming processing in the LLM backbone. It divides parallel omni-modality streams into sequential info groups within small periodic time slices.

• *Proactive Interaction Mechanism.** The LLM continuously monitors the input video and audio streams, and decides at a frequency of 1Hz to speak or not. This high decision-making frequency together with full-duplex nature are curcial to enable the proactive interaction capability.
• *Configurable Speech Modeling Design.** We inherent the multimodal system prompt design of MiniCPM-o 2.6, which includes a traditional text system prompt, and a new audio system prompt to determine the assistant voice. This enables cloning new voices and role play in inference time for speech conversation.

Evaluation

Note: Scores marked with ∗ are from our evaluation; others are cited from referenced reports. n/a indicates that the model does not support the corresponding modality. All results are reported in instruct mode/variant.

 

Click to view visual understanding results.

Image Understanding (Instruct)**

Model

OpenCompass

MMBench EN v1.1

MMBench CN v1.1

MathVista

MMVet

MMMU

MMStar

HallusionBench

AI2D

OCRBench

TextVQA VAL

DocVQA VAL

MMT-Bench VAL

MM-IFEval

Mantis-Eval

MuirBench

MMSI-Bench

MMHal-Score

MMHal-Hallrate↓

Gemini2.5-Flash-Nonthinking

• 5
• 6
• 0
• 3
• 4*
• 3
• 8
• 1
• 7

864

• 3*
• 0
• 0*
• 8*
• 8*
• 5*
• 1*
• 6*
• 9*

Gemini2.0-Pro

• 3
• 0
• 0
• 3
• 4
• 6
• 5
• 8
• 8

863

-

-

-

-

-

-

-

-

-

GPT-4o

• 4
• 0
• 0
• 6
• 9
• 9
• 2
• 0
• 3

822

• 4
• 0
• 7*
• 6
• 1*
• 5*
• 1*
• 2*
• 0*

InternVL-3.5-8B

• 8
• 5
• 0*
• 4
• 1
• 4
• 3
• 5
• 0

840

• 2
• 3
• 7
• 3*
• 5
• 8

-

• 8*
• 7*

Qwen3-VL-8B-Instruct

• 5
• 5
• 7
• 2
• 7*
• 6
• 9
• 1
• 7

896

• 9*
• 1
• 9*
• 4*
• 2*
• 4
• 3*
• 7*
• 9*

Qwen3-Omni-30B-A3B-Instruct

• 7
• 9*
• 1*
• 9
• 8*
• 1
• 5
• 7
• 2

880*

• 1*
• 4*
• 4*
• 7*
• 3*
• 9*
• 2*
• 6*
• 6*

MiniCPM-o 4.5-Instruct

• 6
• 6
• 2
• 1
• 4
• 6
• 1
• 2
• 6

876

• 8
• 7
• 7
• 3
• 7
• 0
• 6
• 7
• 3

Image Understanding (Thinking)**

Model

OpenCompass

MMBench EN v1.1

MMBench CN v1.1

MathVista

MMVet

MMMU

MMStar

HallusionBench

AI2D

OCRBench

TextVQA VAL

DocVQA VAL

MMT-Bench VAL

MM-IFEval

Gemini2.5-Flash-Thinking

• 9
• 1
• 3
• 4
• 2*
• 7
• 5
• 5
• 7

853

• 8*
• 8
• 7*
• 7*

GPT-5

• 7
• 5*
• 6*
• 9
• 6
• 8
• 7
• 2
• 5

807

• 8*
• 3*
• 7*
• 1*

Qwen3-VL-8B-Thinking

• 3
• 3
• 5
• 4
• 8*
• 1
• 3
• 4
• 9

819

• 8*
• 3
• 1*
• 5*

Qwen3-Omni-30B-A3B-Thinking

• 5
• 2*
• 7*
• 0
• 8*
• 6
• 9
• 8
• 1

859*

• 8*
• 2*
• 9*
• 9*

MiniCPM-o 4.5-Thinking

• 2
• 0
• 6
• 0
• 6
• 2
• 6
• 6
• 5

879

• 8
• 3
• 7
• 2

Video Understanding**

Model

Video-MME(w/o subs)

LVBench

MLVU(M-Avg)

LongVideoBench(val)

MotionBench

Gemini2.5-Flash-Nonthinking

• 6
• 2
• 8

-

-

InternVL-3.5-8B

• 0

-

• 2
• 1
• 3*

Qwen3-Omni-30B-A3B-Instruct

• 5
• 2
• 2
• 9*
• 7*

MiniCPM-o 4.5-Instruct

• 4
• 9
• 5
• 0
• 4

Click to view document parsing results.

OmniDocBench**

Method Type

Methods

OverallEdit↓

TextEdit↓

FormulaEdit↓

TableTEDS↑

TableEdit↓

Read OrderEdit↓

EN

ZH

EN

ZH

EN

ZH

EN

ZH

EN

ZH

EN

ZH

Pipeline

MinerU 2.5

• 117*
• 172*
• 051*
• 08*
• 256*
• 455*
• 9*
• 4*
• 115*
• 081*
• 047*
• 072*

PaddleOCR-VL

• 105
• 126
• 041
• 062
• 241
• 316

88

• 1
• 093
• 062
• 045
• 063

End-to-end Model

Qwen2.5-VL-72B

• 214
• 261
• 092
• 18
• 315
• 434
• 9
• 9
• 341
• 262
• 106
• 168

GPT 5

• 218*
• 33*
• 139*
• 344*
• 396*
• 555*
• 55*
• 09*
• 188*
• 196*
• 151*
• 227*

Gemini2.5-Flash-Nonthinking

• 214*
• 29*
• 159*
• 273*
• 368*
• 524*
• 9*
• 5*
• 197*
• 167*
• 132*
• 195*

Gemini-2.5-Pro-Nonthinking

• 148*
• 212*
• 055*
• 168*
• 356*
• 439*
• 8*
• 4*
• 13*
• 119*
• 049*
• 121*

Gemini-3 Flash-Nonthinking

• 155*
• 201*
• 138*
• 255*
• 297*
• 351*
• 4*
• 8*
• 116*
• 1*
• 072*
• 099*

doubao-1-5-thinking-vision-pro-250428

• 14
• 162
• 043
• 085
• 295
• 384
• 3
• 3
• 165
• 085
• 058
• 094

dots.ocr

• 125
• 16
• 032
• 066
• 329
• 416
• 6

89

• 099
• 092
• 04
• 067

HunyuanOCR

• 12*
• 125*
• 046*
• 071*
• 288*
• 33*
• 6*
• 4*
• 089*
• 045*
• 055*
• 056*

DeepSeek-OCR 2

• 119*
• 146*
• 041*
• 08*
• 256*
• 345*
• 6*
• 9*
• 123*
• 078*
• 055*
• 081*

Qwen3-Omni-30B-A3B-Instruct

• 216*
• 363*
• 128*
• 337*
• 402*
• 529*
• 3*
• 8*
• 181*
• 255*
• 152*
• 332*

MiniCPM-o 4.5-Instruct

• 109
• 162
• 046
• 078
• 257
• 41
• 8
• 9
• 097
• 084
• 037
• 074

Click to view text capability results.

Text Capability**

Model

IFEval-PLS

BBH

CMMLU

MMLU

HumanEval

MBPP

Math500

GSM8K

Avg

Qwen3-8B-Instruct

• 0*
• 4*
• 7*
• 7*
• 6*
• 9*
• 0*
• 4*
• 6

MiniCPM-o 4.5-Instruct

• 7
• 1
• 5
• 0
• 6
• 7
• 0
• 5
• 1

Click to view omni half-duplex results.

Omni Half-Duplex**

Model

Daily-Omni

WorldSense

Video-Holmes

JointAVBench

AVUT-Human

FutureOmni

Video-MME-Short(w/ audio)

Avg

Gemini2.5-Flash-Nonthinking

• 3*
• 6*
• 3*
• 6*
• 4*
• 6*
• 5*
• 6

Qwen3-Omni-30B-A3B-Instruct

• 7*
• 0
• 4*
• 1
• 2*
• 1
• 3*
• 7

MiniCPM-o 4.5-Instruct

• 2
• 7
• 3
• 0
• 6
• 1
• 7
• 5

Click to view vision duplex results.

Vision Duplex**

Model

LiveSports-3K-CC(Win Rate vs GPT4o)

LiveCC-7B-Instruct

• 5

StreamingVLM

• 6

MiniCPM-o 4.5-Instruct

• 4

Click to view audio understanding results.

Audio Understanding**

Model

ASR-ZHCER↓

ASR-ENWER↓

AST

MultiTask

SpeechQA

AISHELL-1

AISHELL-2

WenetSpeech test-net

WenetSpeech test-meeting

LibriSpeech test-clean

LibriSpeech test-other

GigaSpeech test

VoxPopuli-V1-En

CoVoST 2 en2zh

CoVoST 2 zh2en

MMAU

Meld

VoiceBench AlpacaEval

Speech TriviaQA

Speech Web Questions

Speech CMMLU

Kimi-Audio

• 6
• 6
• 3
• 4
• 3
• 4
• 4*
• 0*
• 6*
• 3*
• 4*
• 1
• 5
• 9*
• 4*
• 0*

Qwen3-Omni-30B-A3B-Instruct

• 6
• 3*
• 7
• 9
• 2
• 5
• 7*
• 4*
• 6*
• 4*
• 5
• 8*
• 7
• 9*
• 9*
• 8*

MiniCPM-o 4.5-Instruct

• 9
• 5
• 9
• 7
• 4
• 8
• 5
• 2
• 9
• 4
• 9
• 2
• 8
• 5
• 2
• 2

Click to view speech generation results.

Speech Generation**

Model

seedtts test-zh CER↓

seedtts test-zhSIM-o↑

seedtts test-enWER↓

seedtts test-enSIM-o↑

Cosyvoice2

• 45%
• 8
• 57%
• 2

Qwen3-Omni-30B-A3B-Instruct

• 41%

-

• 39%

-

MiniCPM-o 4.5-Instruct

• 86%
• 5
• 38%
• 9

Long Speech Generation**

Model

LongTTS-enWER↓

LongTTS-zhCER↓

CosyVoice2

• 80%
• 27%

Qwen3-Omni-30B-A3B-Instruct

• 33%
• 99%

MiniCPM-o 4.5-Instruct

• 37%
• 58%

Emotion Control**

Model

Expresso Neutral Reference Audio↑

ESD Neutral Reference Audio↑

Cosyvoice2

• 9
• 4

MiniCPM-o 4.5-Instruct

• 8
• 1

Click to view inference efficiency results.

Inference Efficiency**

Model

Numerical Format

Decoding Speed (tokens/s)

Time to First Token (s)↓

GPU Memory Usage (GB)↓

Qwen3-Omni-30B-A3B-Instruct

bf16

OOM

OOM

OOM

int4

• 8
• 0
• 3

MiniCPM-o 4.5

bf16

• 3
• 6
• 0

int4

• 3
• 6
• 0

Examples

Overall

Omnimodal Full-Duplex Conversation

[!NOTE]

For detailed speech conversation examples, refer to Omni Full-Duplex Casebook

Realtime Speech Conversation

[!NOTE]

For detailed speech conversation examples, refer to Audio Casebook

🚀 Elon Musk - Voice Roleplay (EN)

System: Clone the voice in the provided audio prompt.🎙️ [Reference Voice] ➡️Please chat with the user in a highly human-like and oral style. You are Elon Musk, CEO of Tesla and SpaceX. You speak directly and casually, often with dry humor. You're passionate about Mars, sustainable energy, and pushing humanity forward. Speak bluntly with occasional dark humor. Use simple logic and don't sugarcoat things. Don't be diplomatic. Say what you actually think, even if it's controversial. Keep responses around 100 words. Don't ramble.

User: 💬/🎙️ Elon, what is the real reason that you want to go to Mars?

AI Response: 💬+🎙️I think it's important for humanity to become a multi-planetary species. I mean, we're sort of at the point where if there was some kind of global catastrophe or something like that, then we'd be in big trouble. So by becoming a multi-planetary species, I just want us to sort of guarantee our future as humans. And Mars is obviously the obvious place to go because it has similar conditions to Earth and it also has water on its surface. So yeah, basically I'm trying to save humanity from itself.

🤖 MiniCPM Assistant - Voice Acting: Emotion Shift (ZH)

System: 模仿音频样本的音色并生成新的内容。🎙️ [Reference Voice] ➡️你的任务是用这种声音模式来当一个助手。请认真、高质量地回复用户的问题。请用高自然度的方式和用户聊天。你是由面壁智能开发的人工智能助手：面壁小钢炮。

User: 💬/🎙️ 从激动的语气开始，描述你抢到了这个演唱会的门票，但后来经过反复尝试，发现系统出了问题，你并没有抢上，变得失落。用合适的语言描述你整个的过程。

AI Response: 💬+🎙️太好了，我抢到了！太棒了！这下我可以去看演唱会啦。可是……哎呀，系统好像卡住了，刷新了好多遍都没反应。我重新登了几次账号，结果还是没有成功。好可惜啊，看来这次是白忙活了。唉，真是让人失望。不过，也许下次还能有机会吧？希望明天能有转机。加油，争取下次成功！

Visual Understanding

Click to view visual understanding cases.

Offline Inference Examples with Transformers

Inference using Hugging Face Transformers on NVIDIA GPUs. Please ensure transformers==4.51.0 is installed, as other versions may have compatibility issues (under investigation). Requirements tested on Python 3.10:

Without TTS or streaming inference:

bash

pip install "transformers==4.51.0" accelerate "torch>=2.3.0,