Decodo Scraper OpenClaw Skill
Use this skill to search Google, scrape any URL, or fetch YouTube subtitles via the Decodo Web Scraping API. Search outputs a JSON object of result sections; Scrape URL outputs plain markdown; Amazon and Amazon search output parsed product-page or search results (JSON). Amazon search uses --query. YouTube subtitles outputs transcript/subtitles. Reddit post and Reddit subreddit output post/listing content (JSON).
Authentication: Set DECODO_AUTH_TOKEN (Basic auth token from Decodo Dashboard → Scraping APIs) in your environment or in a .env file in the repo root.
Errors: On failure the script writes a JSON error to stderr and exits with code 1.
Tools
1. Search Google
Use this to find URLs, answers, or structured search results. The API returns a JSON object whose results key contains several sections (not all may be present for every query):
| Section | Description |
|---|
| INLINECODE4 | Main search results (titles, links, snippets). |
| INLINECODE5 |
AI-generated overviews or summaries when Google shows them. |
|
paid | Paid/sponsored results (ads). |
|
related_questions | “People also ask”–style questions and answers. |
|
related_searches | Suggested related search queries. |
|
discussions_and_forums | Forum or discussion results (e.g. Reddit, Stack Exchange). |
The script outputs only the inner results object (these sections); pagination info (page, last_visible_page, parse_status_code) is not included.
Command:
CODEBLOCK0
Examples:
CODEBLOCK1
Optional: --geo us or --locale en for location/language.
2. Scrape URL
Use this to get the content of a specific web page. By default the API returns content as Markdown (cleaner for LLMs and lower token usage).
Command:
CODEBLOCK2
Examples:
python3 tools/scrape.py --target universal --url "https://example.com"
python3 tools/scrape.py --target universal --url "https://news.ycombinator.com/"
3. Amazon product page
Use this to get parsed data from an Amazon product (or other Amazon) page. Pass the product page URL as --url. The script sends parse: true and outputs the inner results object (e.g. ads, product details, etc.).
Command:
CODEBLOCK4
Examples:
python3 tools/scrape.py --target amazon --url "https://www.amazon.com/dp/B09H74FXNW"
4. Amazon search
Use this to search Amazon and get parsed results (search results list, delivery_postcode, etc.). Pass the search query as --query.
Command:
CODEBLOCK6
Examples:
python3 tools/scrape.py --target amazon_search --query "laptop"
5. YouTube subtitles
Use this to get subtitles/transcript for a YouTube video. Pass the video ID (e.g. from youtube.com/watch?v=VIDEO_ID) as --query.
Command:
CODEBLOCK8
Examples:
python3 tools/scrape.py --target youtube_subtitles --query "dFu9aKJoqGg"
6. Reddit post
Use this to get the content of a Reddit post (thread). Pass the full post URL as --url.
Command:
CODEBLOCK10
Examples:
python3 tools/scrape.py --target reddit_post --url "https://www.reddit.com/r/nba/comments/17jrqc5/serious_next_day_thread_postgame_discussion/"
7. Reddit subreddit
Use this to get the listing (posts) of a Reddit subreddit. Pass the subreddit URL as --url.
Command:
CODEBLOCK12
Examples:
python3 tools/scrape.py --target reddit_subreddit --url "https://www.reddit.com/r/nba/"
Summary
| Action | Target | Argument | Example command |
|---|
| Search | INLINECODE24 | INLINECODE25 | INLINECODE26 |
| Scrape page |
universal |
--url |
python3 tools/scrape.py --target universal --url "https://example.com" |
| Amazon product |
amazon |
--url |
python3 tools/scrape.py --target amazon --url "https://www.amazon.com/dp/B09H74FXNW" |
| Amazon search |
amazon_search |
--query |
python3 tools/scrape.py --target amazon_search --query "laptop" |
| YouTube subtitles |
youtube_subtitles |
--query |
python3 tools/scrape.py --target youtube_subtitles --query "dFu9aKJoqGg" |
| Reddit post |
reddit_post |
--url |
python3 tools/scrape.py --target reddit_post --url "https://www.reddit.com/r/nba/comments/17jrqc5/..." |
| Reddit subreddit |
reddit_subreddit |
--url |
python3 tools/scrape.py --target reddit_subreddit --url "https://www.reddit.com/r/nba/" |
Output: Search → JSON (sections). Scrape URL → markdown. Amazon / Amazon search → JSON (results e.g. ads, product info, delivery_postcode). YouTube → transcript. Reddit → JSON (content).
Decodo Scraper OpenClaw 技能
使用此技能通过 Decodo 网页抓取 API 搜索 Google、抓取任意 URL 或获取 YouTube 字幕。搜索输出结果部分的 JSON 对象;抓取 URL输出纯 Markdown;Amazon和Amazon 搜索输出解析后的产品页面或搜索结果(JSON)。Amazon 搜索使用 --query。YouTube 字幕输出转录文本/字幕。Reddit 帖子和Reddit 子版块输出帖子/列表内容(JSON)。
身份验证:在环境变量或仓库根目录的 .env 文件中设置 DECODOAUTHTOKEN(来自 Decodo 仪表板 → 抓取 API 的基本身份验证令牌)。
错误处理:失败时脚本将 JSON 错误写入 stderr 并以退出码 1 退出。
工具
1. 搜索 Google
用于查找 URL、答案或结构化搜索结果。API 返回一个 JSON 对象,其 results 键包含多个部分(并非每个查询都会包含所有部分):
| 部分 | 描述 |
|---|
| organic | 主要搜索结果(标题、链接、摘要)。 |
| ai_overviews |
Google 显示的 AI 生成概览或摘要。 |
| paid | 付费/赞助结果(广告)。 |
| related_questions | 其他人还问了风格的问题和答案。 |
| related_searches | 建议的相关搜索查询。 |
| discussions
andforums | 论坛或讨论结果(例如 Reddit、Stack Exchange)。 |
脚本仅输出内部的 results 对象(这些部分);分页信息(page、lastvisiblepage、parsestatuscode)不包含在内。
命令:
bash
python3 tools/scrape.py --target google_search --query 你的搜索查询
示例:
bash
python3 tools/scrape.py --target google_search --query 2025年最佳笔记本电脑
python3 tools/scrape.py --target google_search --query python requests 教程
可选:--geo us 或 --locale en 用于指定位置/语言。
2. 抓取 URL
用于获取特定网页的内容。默认情况下,API 以 Markdown 格式返回内容(对 LLM 更清晰,令牌使用量更低)。
命令:
bash
python3 tools/scrape.py --target universal --url https://example.com
示例:
bash
python3 tools/scrape.py --target universal --url https://example.com
python3 tools/scrape.py --target universal --url https://news.ycombinator.com/
3. Amazon 产品页面
用于从 Amazon 产品(或其他 Amazon)页面获取解析后的数据。将产品页面 URL 作为 --url 传递。脚本发送 parse: true 并输出内部的 results 对象(例如 ads、产品详情等)。
命令:
bash
python3 tools/scrape.py --target amazon --url https://www.amazon.com/dp/PRODUCT_ID
示例:
bash
python3 tools/scrape.py --target amazon --url https://www.amazon.com/dp/B09H74FXNW
4. Amazon 搜索
用于搜索 Amazon 并获取解析后的结果(搜索结果列表、delivery_postcode 等)。将搜索查询作为 --query 传递。
命令:
bash
python3 tools/scrape.py --target amazon_search --query 你的搜索查询
示例:
bash
python3 tools/scrape.py --target amazon_search --query 笔记本电脑
5. YouTube 字幕
用于获取 YouTube 视频的字幕/转录文本。将 视频 ID(例如来自 youtube.com/watch?v=VIDEO_ID)作为 --query 传递。
命令:
bash
python3 tools/scrape.py --target youtubesubtitles --query VIDEOID
示例:
bash
python3 tools/scrape.py --target youtube_subtitles --query dFu9aKJoqGg
6. Reddit 帖子
用于获取 Reddit 帖子(主题)的内容。将完整的帖子 URL 作为 --url 传递。
命令:
bash
python3 tools/scrape.py --target reddit_post --url https://www.reddit.com/r/SUBREDDIT/comments/ID/...
示例:
bash
python3 tools/scrape.py --target redditpost --url https://www.reddit.com/r/nba/comments/17jrqc5/seriousnextdaythreadpostgamediscussion/
7. Reddit 子版块
用于获取 Reddit 子版块的列表(帖子)。将子版块 URL 作为 --url 传递。
命令:
bash
python3 tools/scrape.py --target reddit_subreddit --url https://www.reddit.com/r/SUBREDDIT/
示例:
bash
python3 tools/scrape.py --target reddit_subreddit --url https://www.reddit.com/r/nba/
总结
| 操作 | 目标 | 参数 | 示例命令 |
|---|
| 搜索 | googlesearch | --query | python3 tools/scrape.py --target googlesearch --query 笔记本电脑 |
| 抓取页面 |
universal | --url | python3 tools/scrape.py --target universal --url https://example.com |
| Amazon 产品 | amazon | --url | python3 tools/scrape.py --target amazon --url https://www.amazon.com/dp/B09H74FXNW |
| Amazon 搜索 | amazon
search | --query | python3 tools/scrape.py --target amazonsearch --query 笔记本电脑 |
| YouTube 字幕 | youtube
subtitles | --query | python3 tools/scrape.py --target youtubesubtitles --query dFu9aKJoqGg |
| Reddit 帖子 | reddit
post | --url | python3 tools/scrape.py --target redditpost --url https://www.reddit.com/r/nba/comments/17jrqc5/... |
| Reddit 子版块 | reddit
subreddit | --url | python3 tools/scrape.py --target redditsubreddit --url https://www.reddit.com/r/nba/ |
输出:搜索 → JSON(部分)。抓取 URL → Markdown。Amazon / Amazon 搜索 → JSON(结果,例如 ads、产品信息、delivery_postcode)。YouTube → 转录文本。Reddit → JSON(内容)。