Instagram Recipe Extractor

Extract recipes from Instagram reels using a multi-layered approach:

1. 1. Caption parsing — Instant, check description first
2. Audio transcription — Whisper (local, no API key)
3. Frame analysis — Vision model for on-screen text

No Instagram login required. Works on public reels.

When to Use

• - User sends an Instagram reel link
• User mentions "recipe from Instagram" or "save this reel"
• User wants to extract recipe details from a video post

How It Works (MANDATORY FLOW)

ALWAYS follow this complete flow — do not stop after caption if instructions are missing:

1. 1. User sends Instagram reel URL
2. Extract metadata using yt-dlp (--dump-json)
3. Parse the caption for recipe details
4. Check completeness: Does caption have BOTH ingredients AND instructions?

- ✅ YES: Present the recipe - ❌ NO (missing instructions or incomplete): Automatically proceed to audio transcription — do NOT stop or ask the user

1. 5. If audio transcription needed:

- Download video: yt-dlp -o "/tmp/reel.mp4" "URL" - Extract audio: ffmpeg -y -i /tmp/reel.mp4 -vn -acodec pcm_s16le -ar 16000 -ac 1 /tmp/reel.wav - Transcribe: whisper /tmp/reel.wav --model base --output_format txt --output_dir /tmp - Merge caption ingredients with audio instructions

1. 6. Present clean, formatted recipe (combining caption + audio as needed)
2. User decides what to do (save to notes, add to wishlist, etc.)

Completeness check heuristics:

• - Has ingredients = contains 3+ quantity+item patterns (e.g., "1 cup flour", "2 lbs chicken")
• Has instructions = contains action verbs (blend, cook, bake, mix, pour, add) + sequence OR numbered steps

Extraction Command

CODEBLOCK0

Key fields from JSON output:

• - description — The caption containing the recipe
• INLINECODE5 — Creator's name
• INLINECODE6 — Creator's handle
• INLINECODE7 — Original URL
• INLINECODE8 — Popularity indicator

Recipe Parsing

Look for these patterns in the caption:

Macros:

• - "X Calories | Xg P | Xg C | Xg F"
• "Macros per serving"
• "Cal/Protein/Carbs/Fat"

Ingredients:

• - Lines starting with quantities (1 cup, 2 tbsp, 24oz)
• Lines with measurement units
• Emoji bullet points (🥩 🌽 🧀 etc.)

Sections:

• - "For the [component]:"
• "Ingredients:"
• "Instructions:"
• "Directions:"

Output Format

Present extracted recipe cleanly:

CODEBLOCK1

User Actions After Extraction

Let the user decide what to do:

• - "Save to my recipes" → Save to Apple Notes (if meal-planner skill available)
• "Add to wishlist" → Save to INLINECODE9
• "Just show me" → Display only, no save
• "Plan this for next week" → Hand off to meal-planner skill

Wishlist Storage

Optional storage for recipes user wants to try later:

memory/recipe-wishlist.json:
CODEBLOCK2

Error Handling

If yt-dlp fails:

• - Check if URL is valid Instagram reel format
• May be a private account — inform user
• Suggest user paste caption text manually as fallback

If no recipe found in caption (IMPORTANT):

After extracting, scan the caption for recipe indicators:

• - Ingredient quantities (numbers + units like oz, cups, tbsp, lbs)
• Recipe sections ("For the...", "Ingredients:", "Instructions:")
• Cooking verbs (bake, cook, sauté, mix, combine)
• Macro information (calories, protein, carbs, fat)

If none found, tell the user clearly:

"I pulled the caption but it doesn't look like the recipe is there — it might just be a teaser or the recipe is only shown in the video itself. Here's what the caption says:

[show caption]

A few options:

1. 1. Check the comments — sometimes creators post recipes there
2. Check their bio link — might lead to the full recipe
3. Describe what you saw in the video and I can help find a similar recipe"

Recipe detection heuristics:
CODEBLOCK3

Integration with meal-planner

The meal-planner skill can reference this skill:

• - When planning meals, check wishlist for untried recipes
• Suggest wishlist recipes that match pantry items
• Mark recipes as "tried" after they're used in a meal plan

Audio Transcription (V2) — MANDATORY FALLBACK

When caption is missing instructions, ALWAYS transcribe the audio automatically. Do not stop and ask the user — just do it. This is the most common case since creators often put ingredients in captions but speak the instructions.

Step 1: Download video
CODEBLOCK4

Step 2: Extract audio
CODEBLOCK5

Step 3: Transcribe with Whisper
CODEBLOCK6

Step 4: Parse transcript for recipe
Look for cooking instructions, ingredients mentioned verbally.

Inference for Missing Measurements

ALWAYS infer quantities when not provided. Never present a recipe without amounts — estimate based on context and standard package sizes.

Vague Language → Specific Amounts

What they say	Infer
"some chicken"	~1 lb
"a bit of garlic"

2-3 cloves | | "handful of spinach" | ~2 cups | | "drizzle of oil" | 1-2 tbsp | | "season to taste" | ½ tsp salt, ¼ tsp pepper | | "splash of soy sauce" | 1-2 tbsp | | "a few tablespoons" | 2-3 tbsp | | "some rice" | 1 cup dry | | "cheese on top" | ½ - 1 cup shredded | | "diced onion" | 1 medium onion | | "bell peppers" | 2 peppers |

Standard Package Sizes (when item mentioned without amount)

Ingredient	Standard Package	Infer
Puff pastry	17oz sheet	1 sheet
Ground beef/turkey

1 lb pack | 1 lb | | Chicken breast | ~1.5 lb pack | 1.5 lbs | | Sausage links | 14oz / 4-5 links | 1 package | | Bacon | 12oz / 12 slices | ½ package (6 slices) | | Shredded cheese | 8oz bag | 1-2 cups | | Tortillas | 8-10 count | 1 package | | Canned beans | 15oz can | 1 can | | Broth/stock | 32oz carton | 1-2 cups | | Pasta | 16oz box | 8oz (half box) | | Rice | 2 lb bag | 1-2 cups dry |

Context-Aware Scaling

By recipe type:

• - Stir fry for 2 → 1 lb protein, 4 cups veggies
• Soup/stew → 1.5-2 lbs protein, 4 cups broth
• Sheet pan meal → 1.5 lbs protein, 3-4 cups veggies
• Appetizers → smaller portions, estimate ~12-15 pieces per batch

By servings mentioned:

• - "Serves 4" → Scale standard amounts for 4
• "Meal prep for the week" → Assume 5-8 servings
• No servings mentioned → Default to 4 servings

By protein target (if user has macro goals):

• - 40-50g protein per serving → ~6-8oz cooked meat per portion
• Scale recipe protein accordingly

Output Format

Always present inferred amounts clearly:
CODEBLOCK7

Mark inferred quantities with (estimated) so user knows what came from the source vs inference.

Combined Extraction Flow

CODEBLOCK8

Frame Analysis

Extract key frames and analyze with vision model.

Extract frames:
CODEBLOCK9

Send to vision model:
Use Claude's image analysis to read each frame:

• - Recipe cards / title screens
• Ingredient lists shown on screen
• Measurements in text overlays
• Step-by-step instructions displayed

Vision prompt:
CODEBLOCK10

Merge strategy:

• - Audio transcript = primary source (spoken instructions)
• Frame analysis = supplement (exact measurements, recipe cards)
• Combine both, prefer specific measurements from visual over inferred from audio

Pinned Comment Detection

Scan caption for these phrases (case-insensitive):

• - "recipe pinned"
• "pinned in comments"
• "check comments"
• "in the comments"
• "comment below"
• "recipe below"
• "full recipe in comments"

If detected, flag and notify user after extraction:

"Heads up — the creator said the recipe is pinned in the comments.
I got what I could from the audio, but yt-dlp can't access pinned comments
without login. If you want the exact recipe, copy the pinned comment and
send it to me — I'll format it properly."

Requirements

• - yt-dlp — INLINECODE11
• INLINECODE12 — INLINECODE13
• INLINECODE14 — pip3 install openai-whisper (runs locally, no API key)
• No Instagram login required for public reels