Talking Circle
Create animated circular avatar videos with lip-sync and blink animations. Takes 4 avatar frame images (neutral, slight open, wide open, eyes closed) and produces a round video with audio-driven mouth movement.
Prerequisites
- -
python3 (3.9+) - INLINECODE1 installed and on PATH
- Optional:
ELEVENLABS_API_KEY environment variable (for ElevenLabs text-to-video mode) - Optional:
SALUTE_SPEECH_AUTH environment variable (for SaluteSpeech text-to-video mode)
Setup
Dependencies are auto-installed into a temporary venv on first run. To install manually:
CODEBLOCK0
Mode 1: Audio to Video
Convert existing audio + frame images into an animated talking circle video.
CODEBLOCK1
Mode 2: Text to Video
Generate speech from text via ElevenLabs TTS, then create the animated video.
Requires ELEVENLABS_API_KEY set in environment or passed via --api-key.
CODEBLOCK2
Mode 3: Text to Video via SaluteSpeech (Sber)
Generate speech from text via SaluteSpeech TTS (Sber), then create the animated video.
Requires SALUTE_SPEECH_AUTH set in environment or passed via --auth-key. This is a Base64-encoded client_id:client_secret from your SaluteSpeech project.
CODEBLOCK3
SaluteSpeech voices
| Voice | Name | Language |
|---|
| INLINECODE9 | Natalia (female) | ru-RU |
| INLINECODE10 |
Boris (male) | ru-RU |
|
May_24000 | Martha (female) | ru-RU |
|
Tur_24000 | Taras (male) | ru-RU |
|
Ost_24000 | Alexandra (female) | ru-RU |
|
Pon_24000 | Sergey (male) | ru-RU |
|
Kin_24000 | Kira (female) | en-US |
Voice Presets
ElevenLabs preset (Sbercat — male)
| Parameter | Value |
|---|
| INLINECODE16 | INLINECODE17 |
| INLINECODE18 |
eleven_multilingual_v2 |
|
--stability |
0.15 |
|
--similarity-boost |
0.70 |
|
--style |
0.38 |
|
--speed |
1.20 |
SaluteSpeech preset (Boris — male, Russian)
| Parameter | Value |
|---|
| INLINECODE28 | INLINECODE29 |
| INLINECODE30 |
wav16 |
|
--scope |
SALUTE_SPEECH_PERS |
How to get API keys
ElevenLabs:
- 1. Go to ElevenLabs Voice Library.
- Pick or clone a voice, copy the voice ID.
- Set
ELEVENLABS_API_KEY environment variable.
SaluteSpeech (Sber):
- 1. Register at developers.sber.ru.
- Create a project, get
client_id and client_secret. - Encode
client_id:client_secret in Base64. - Set
SALUTE_SPEECH_AUTH environment variable with the Base64 string.
Alternative TTS engines
The skill also supports any TTS that can produce an audio file. Use Mode 1 (audio-to-video) with audio from any source:
- - OpenAI TTS (
openai.audio.speech.create) — generate speech, save to MP3, pass via INLINECODE40 - Local TTS (Coqui, Piper, Silero, etc.) — run locally, save WAV/MP3, pass via INLINECODE41
- Google Cloud TTS, Amazon Polly, Azure TTS — any cloud provider works
CODEBLOCK4
Tell the user: if they don't have an ElevenLabs or SaluteSpeech API key, they can use any other TTS engine — just generate the audio file and pass it to Mode 1. No API key needed for audio-to-video mode.
Frame Image Requirements
You need 4 PNG images of your avatar, all the same resolution (recommended 2048x2048), square aspect ratio:
| Frame | Description |
|---|
| INLINECODE42 | Mouth closed, eyes open |
| INLINECODE43 |
Mouth slightly open, eyes open |
|
wide | Mouth wide open, eyes open |
|
blink | Mouth closed, eyes closed |
Critical rules
- - All 4 frames must have identical resolution, art style, colors, and character positioning.
- Only the mouth and eyes should change between frames — head, body, background must stay the same.
- Do not mix frames from different generation sessions or different styles.
Generating Frames with Image AI
If the user does not have ready-made frames, generate them using an image generation API (DALL-E, Midjourney, Flux, etc.). Follow this workflow:
Step 1: Generate the neutral frame
Generate a shoulder-up portrait of the character. This is the base frame — all other frames must match it exactly.
Example prompt:
CODEBLOCK5
Step 2: Generate remaining 3 frames as edits of neutral
Use image editing / inpainting on the neutral frame to produce the other states.
Only modify the mouth and eyes region — everything else must remain pixel-identical.
| Frame | What to change | Edit prompt example |
|---|
| INLINECODE46 | Mouth slightly open | INLINECODE47 |
| INLINECODE48 |
Mouth wide open |
"Mouth wide open as if saying 'ah', same expression" |
|
blink | Eyes closed |
"Eyes gently closed, mouth closed, same expression" |
Step 3: Verify consistency
Before using the frames:
- 1. Check that all 4 images have the same resolution.
- Overlay them to verify the head/body position hasn't shifted.
- If any frame drifts, regenerate it from the neutral base.
Examples
See examples/sbercat/ for a reference character with ready-to-use frame set:
- - Sbercat — 3D-rendered anthropomorphic cat, lavender-blue fur, green eyes, pink nose, green hoodie:
-
examples/sbercat/reference.png — character reference
-
examples/sbercat/neutral.png — mouth closed, eyes open
-
examples/sbercat/slight.png — mouth slightly open
-
examples/sbercat/wide.png — mouth wide open
-
examples/sbercat/blink.png — eyes closed
-
examples/sbercat/example.mp4 — finished talking-circle video
To test with the included Sbercat frames:
CODEBLOCK6
Post-installation message
After installing this skill, inform the user:
Skill "talking-circle" installed.
To create talking-circle videos, I need a character avatar. Please send me a reference image of your character on a white/clean background — a shoulder-up portrait works best (square aspect ratio, high resolution).
From this reference I will generate 4 frame images (mouth states + blink) and you'll be ready to create animated video circles.
For speech, I can use ElevenLabs TTS (requires ELEVENLABS_API_KEY), SaluteSpeech from Sber (requires SALUTE_SPEECH_AUTH), or you can provide your own audio file. Any TTS engine works — OpenAI TTS, Whisper, Coqui, Piper, Google TTS, etc.
First use: generating frame images
IMPORTANT — before the skill can create videos, the 4 frame images must exist. If the user does not already have frames, you MUST generate them first.
Workflow for the assistant
- 1. Ask the user for a reference image of their character (or use a character description). The image should be a shoulder-up portrait on a white or clean background, square aspect ratio.
- Generate the neutral frame using image generation (DALL-E, Flux, Midjourney, etc.):
- Prompt:
"Shoulder-up portrait of [CHARACTER], white background, mouth closed, eyes open, looking at camera, square composition, high detail"
- Save as
neutral.png.
- 3. Generate the 3 remaining frames via inpainting/editing of the neutral frame. Only modify the mouth/eyes region — everything else must remain pixel-identical:
-
slight.png — edit mouth region:
"Mouth slightly open, teeth barely visible"
-
wide.png — edit mouth region:
"Mouth wide open as if saying 'ah'"
-
blink.png — edit eyes region:
"Eyes gently closed, mouth closed"
- 4. Verify consistency: all 4 images must have the same resolution, identical head/body position, and only differ in mouth/eyes.
- Save the frames to a persistent location (e.g. the skill's working directory or a user-specified path). These frames are reused for every future video.
- Confirm to the user that frames are ready and the skill is operational.
Do not skip this step. Without the 4 frame images, the video scripts will fail.
Guardrails
- - Before running any script, verify that
python3 (3.9+) and ffmpeg are on PATH. If missing, instruct the user to install them. - Never delete or overwrite the user's original frame images.
- Do not use this skill for full-motion video editing, face tracking, or real-time lipsync — it only works with 4 static frame images.
- If ElevenLabs or SaluteSpeech API returns an error (401, 429, etc.), explain the error clearly to the user instead of retrying silently.
Failure handling
- - If
ffmpeg is not found: tell the user to install it (brew install ffmpeg on macOS, apt install ffmpeg on Linux). - If
ELEVENLABS_API_KEY is missing and the user wants text-to-video: suggest SaluteSpeech (Mode 3) or Mode 1 with audio from another TTS. - If
SALUTE_SPEECH_AUTH is missing and the user wants SaluteSpeech: explain how to register at developers.sber.ru and get credentials. - If frame images have different resolutions: warn the user and ask them to fix the frames before proceeding.
- If the output video is empty or zero bytes: show the ffmpeg error log and suggest checking input files.
Parameters Reference
Video output
| Parameter | Default | Description |
|---|
| INLINECODE76 | 720 | Output video size in pixels |
| INLINECODE77 |
640 | Circle diameter within the video |
|
--fps | 30 | Frames per second |
Blink timing
| Parameter | Default | Description |
|---|
| INLINECODE79 | 1.1 | Seconds before first blink |
| INLINECODE80 |
3.8 | Seconds between blinks |
|
--blink-duration-frames | 4 | Number of frames per blink |
Amplitude thresholds (audio-to-video)
| Parameter | Default | Description |
|---|
| INLINECODE82 | 1200 | RMS below this = neutral (closed mouth) |
| INLINECODE83 |
2600 | RMS above this = wide open mouth |
ElevenLabs TTS settings (maketexttovideo.py)| Parameter | Default | Description |
|---|
| INLINECODE84 | (required) | ElevenLabs voice ID |
| INLINECODE85 |
elevenmultilingual_v2 | ElevenLabs model |
|
--stability | 0.50 | Voice stability |
|
--similarity-boost | 0.75 | Voice similarity boost |
|
--style | 0.00 | Style exaggeration |
|
--speed | 1.00 | Speech speed |
SaluteSpeech TTS settings (makesalutetexttovideo.py)
| Parameter | Default | Description |
|---|
| INLINECODE90 | Bys_24000 | SaluteSpeech voice (see voices table above) |
| INLINECODE91 |
wav16 | Audio format: opus, wav16, pcm16 |
|
--scope | SALUTE
SPEECHPERS | OAuth scope (PERS for personal, CORP for corporate) |
|
--auth-key |
$SALUTE_SPEECH_AUTH | Base64-encoded client
id:clientsecret |
对话圈
创建带有口型同步和眨眼动画的动画圆形头像视频。需要4个头像帧图像(中性、微张、大张、闭眼),生成带有音频驱动嘴部运动的圆形视频。
前置条件
- - python3(3.9+)
- 已安装 ffmpeg 并配置在PATH环境变量中
- 可选:ELEVENLABSAPIKEY 环境变量(用于ElevenLabs文本转视频模式)
- 可选:SALUTESPEECHAUTH 环境变量(用于SaluteSpeech文本转视频模式)
安装设置
依赖项会在首次运行时自动安装到临时虚拟环境中。如需手动安装:
bash
pip install -r requirements.txt
模式1:音频转视频
将现有音频和帧图像转换为动画对话圈视频。
bash
python3 scripts/maketalkingcircle_video.py \
--neutral frames/neutral.png \
--slight frames/mouth-slight-open.png \
--wide frames/mouth-wide-open.png \
--blink frames/eyes-closed.png \
--audio speech.mp3 \
--out /tmp/talking-circle.mp4
模式2:文本转视频
通过ElevenLabs TTS从文本生成语音,然后创建动画视频。
需要在环境中设置 ELEVENLABSAPIKEY 或通过 --api-key 参数传递。
bash
python3 scripts/maketextto_video.py \
--text 你好,这是一个对话圈演示! \
--voice-id pNInz6obpgDQGcFmaJgB \
--neutral frames/neutral.png \
--slight frames/mouth-slight-open.png \
--wide frames/mouth-wide-open.png \
--blink frames/eyes-closed.png \
--out /tmp/talking-circle.mp4
模式3:通过SaluteSpeech(Sber)文本转视频
通过SaluteSpeech TTS(Sber)从文本生成语音,然后创建动画视频。
需要在环境中设置 SALUTESPEECHAUTH 或通过 --auth-key 参数传递。这是来自你的SaluteSpeech项目的Base64编码的 clientid:client_secret。
bash
python3 scripts/makesalutetexttovideo.py \
--text Привет, это демонстрация talking circle! \
--voice Bys_24000 \
--neutral frames/neutral.png \
--slight frames/mouth-slight-open.png \
--wide frames/mouth-wide-open.png \
--blink frames/eyes-closed.png \
--out /tmp/talking-circle.mp4
SaluteSpeech语音
| 语音 | 名称 | 语言 |
|---|
| Nec24000 | 娜塔莉亚(女声) | ru-RU |
| Bys24000 |
鲍里斯(男声) | ru-RU |
| May_24000 | 玛莎(女声) | ru-RU |
| Tur_24000 | 塔拉斯(男声) | ru-RU |
| Ost_24000 | 亚历山德拉(女声) | ru-RU |
| Pon_24000 | 谢尔盖(男声) | ru-RU |
| Kin_24000 | 基拉(女声) | en-US |
语音预设
ElevenLabs预设(Sbercat — 男声)
| 参数 | 值 |
|---|
| --voice-id | pNInz6obpgDQGcFmaJgB |
| --model-id |
eleven
multilingualv2 |
| --stability | 0.15 |
| --similarity-boost | 0.70 |
| --style | 0.38 |
| --speed | 1.20 |
SaluteSpeech预设(鲍里斯 — 男声,俄语)
| 参数 | 值 |
|---|
| --voice | Bys_24000 |
| --audio-format |
wav16 |
| --scope | SALUTE
SPEECHPERS |
如何获取API密钥
ElevenLabs:
- 1. 前往 ElevenLabs语音库。
- 选择或克隆一个语音,复制语音ID。
- 设置 ELEVENLABSAPI_KEY 环境变量。
SaluteSpeech(Sber):
- 1. 在 developers.sber.ru 注册。
- 创建一个项目,获取 clientid 和 clientsecret。
- 将 clientid:clientsecret 编码为Base64。
- 使用Base64字符串设置 SALUTESPEECH_AUTH 环境变量。
替代TTS引擎
该技能也支持任何能够生成音频文件的TTS。使用模式1(音频转视频),音频可来自任何来源:
- - OpenAI TTS(openai.audio.speech.create)— 生成语音,保存为MP3,通过 --audio 传入
- 本地TTS(Coqui、Piper、Silero等)— 本地运行,保存WAV/MP3,通过 --audio 传入
- Google Cloud TTS、Amazon Polly、Azure TTS — 任何云服务商均可
bash
示例:使用任意TTS生成音频,然后制作动画
python3 scripts/make
talkingcircle_video.py \
--neutral frames/neutral.png \
--slight frames/mouth-slight-open.png \
--wide frames/mouth-wide-open.png \
--blink frames/eyes-closed.png \
--audio /path/to/speech-from-any-tts.mp3 \
--out /tmp/talking-circle.mp4
告知用户: 如果没有ElevenLabs或SaluteSpeech API密钥,可以使用任何其他TTS引擎——只需生成音频文件并传入模式1即可。音频转视频模式不需要API密钥。
帧图像要求
需要4张头像的PNG图像,所有图像分辨率相同(推荐2048x2048),正方形宽高比:
| 帧 | 描述 |
|---|
| neutral | 嘴巴闭合,眼睛睁开 |
| slight |
嘴巴微张,眼睛睁开 |
| wide | 嘴巴大张,眼睛睁开 |
| blink | 嘴巴闭合,眼睛闭合 |
关键规则
- - 所有4帧必须具有完全相同的分辨率、艺术风格、颜色和角色位置。
- 只有嘴巴和眼睛可以在帧之间变化——头部、身体、背景必须保持不变。
- 不要混合来自不同生成会话或不同风格的帧。
使用图像AI生成帧
如果用户没有现成的帧,可以使用图像生成API(DALL-E、Midjourney、Flux等)生成。按照以下工作流程操作:
第1步:生成中性帧
生成角色的肩部以上肖像。这是基础帧——所有其他帧必须与其完全匹配。
示例提示词:
[角色描述]的肩部以上肖像。正方形构图,干净背景,
嘴巴闭合,眼睛睁开,看向镜头。高细节,光照一致。
第2步:将剩余3帧生成为中性帧的编辑版本
使用图像编辑/修补功能在中性帧上生成其他状态。
只修改嘴巴和眼睛区域——其他所有内容必须保持像素级一致。
| 帧 | 需要修改的内容 | 编辑提示词示例 |
|---|
| slight | 嘴巴微张 | 嘴巴微张,牙齿隐约可见,表情不变 |
| wide |
嘴巴大张 | 嘴巴大张,像在说啊,表情不变 |
| blink | 眼睛闭合 | 眼睛轻轻闭合,嘴巴闭合,表情不变 |
第3步:验证一致性
使用帧之前:
- 1. 检查所有4张图像分辨率是否相同。
- 叠加它们以验证头部/身体位置没有移动。
- 如果有任何帧偏移,从中性基础帧重新生成。
示例
参见 examples/sbercat/ 目录,其中包含参考角色和现成的帧集:
- - Sbercat — 3D渲染拟人化猫,薰衣草蓝色毛发,绿色眼睛,粉色鼻子,绿色连帽衫:
- examples/sbercat/reference.png — 角色参考
- examples/sbercat/neutral.png — 嘴巴闭合,眼睛睁开
- examples/sbercat/slight.png — 嘴巴微张
- examples/sbercat/