Hum2Song

Turn a hummed melody into a complete song with local audio processing, without uploading sensitive recordings to third-party services.

Overview

This skill converts user humming or singing into complete songs using local AI models. The entire pipeline runs on your machine - no audio data is sent to external services.

Pipeline:

1. 1. 🎤 Audio Input → 2. 🎵 MIDI Extraction → 3. 🎼 Music Generation → 4. 🎧 Complete Song

Triggers

Use this skill when the user:

• - Hums or sings a melody and wants to turn it into a full song
• Has an audio recording of humming/singing
• Wants to create music from their own melodic ideas
• Asks to "turn my humming into a song"

Requirements

System Dependencies

CODEBLOCK0

Optional: ACE-Step for Music Generation (User Choice)

ACE-Step is an optional local AI. Users decide whether to install it.

CODEBLOCK1

Note: First use downloads ~4GB model weights to local cache. No automatic downloads.

Core Workflow

Step 1: Extract MIDI from Audio

Use Basic Pitch (Spotify's open source tool) to convert humming to MIDI:

CODEBLOCK2

Step 2: Enhance MIDI Structure

Clean and enhance the extracted MIDI:

CODEBLOCK3

Step 3: Generate Full Song

Option A: ACE-Step (Local AI, Optional)

Only if user has manually installed ACE-Step:

CODEBLOCK4

Option B: MIDI + SoundFont (No AI)

CODEBLOCK5

Usage

Quick Start

CODEBLOCK6

Parameters

Response Format

When User Provides Audio

CODEBLOCK7

When Complete

CODEBLOCK8

Example Interactions

Example 1: User Hums Audio

Example 2: User Specifies Style

Example 3: User Wants MIDI Output

User: Can I get the MIDI file to edit myself?

AI: 🎼 Absolutely!

I'll generate:
• Extracted MIDI (raw melody)
• Enhanced MIDI (quantized, cleaned)
• Full arrangement MIDI (all instruments)

All files will be in: ~/Music/hum2song_export/

Technical Details

Audio Processing

Input Formats: WAV, MP3, M4A, FLAC, OGG
Sample Rate: Automatically converted to 44.1kHz
Channels: Mono/Stereo → Mono for processing

MIDI Extraction

Model: Basic Pitch (Spotify, ICASSP 2022)
Pitch Range: C1 to C8
Note Detection: Polyphonic capable
Timing Resolution: 10ms

Music Generation

ACE-Step Model:

• - Size: 1B parameters (base), 3B (large)
• Training: Licensed music dataset
• Output: 44.1kHz stereo
• Latency: ~1s per second of audio on M1 Mac

SoundFont Synthesis:

• - No AI required
• Real-time synthesis
• High-quality instrument sounds
• Deterministic output

Limitations

• - Requires local Python environment setup
• ACE-Step needs ~4GB RAM for base model
• Processing time: 2-5 minutes for a 2-minute song
• Quality depends on humming clarity
• Complex harmonies may not be fully captured

Privacy & Security

✅ All processing is local - Your audio never leaves your machine
✅ No cloud services - No API keys or external uploads
✅ Open source tools - Basic Pitch, ACE-Step, Pretty MIDI
✅ No data collection - Nothing is logged or transmitted

References

• - basic-pitch.md - Audio to MIDI extraction
• INLINECODE6 - AI music generation
• INLINECODE7 - MIDI processing
• INLINECODE8 - Audio analysis utilities

Technical Information