Skill Veille - RSS Aggregator

RSS feed aggregator with URL deduplication and topic-based deduplication for OpenClaw agents.
Fetches articles from 20+ configured sources, filters already-seen URLs (TTL 14 days),
and deduplicates articles covering the same story using Jaccard similarity + named entities.

No external dependencies: stdlib Python only (urllib, xml.etree, email.utils).

---

Trigger phrases

• - "fais une veille"
• "quoi de neuf en securite / tech / crypto / IA ?"
• "donne-moi les news du jour"
• "articles recents sur [sujet]"
• "veille RSS"
• "digest du matin"
• "nouvelles non vues"

---

Quick Start

CODEBLOCK0

---

Setup

Requirements

• - Python 3.9+
• Network access to RSS feeds (public, no auth required)
• No pip installs needed

Installation

CODEBLOCK1

The wizard creates:

• - ~/.openclaw/config/veille/config.json (from config.example.json)
• INLINECODE2 (data directory)

Customizing sources

Edit ~/.openclaw/config/veille/config.json and add/remove entries in the "sources" dict:

CODEBLOCK2

---

Storage and credentials

Files written by this skill

Path	Written by	Purpose	Contains secrets
INLINECODE5	INLINECODE6	Sources, outputs, options	NO
INLINECODE7

veille.py | URL dedup store (TTL 14d) | NO | | ~/.openclaw/data/veille/topic_seen.json | veille.py | Topic dedup store (TTL 5d) | NO |

Files read from outside the skill

Path	Read by	Key accessed	When
INLINECODE11	INLINECODE12	INLINECODE13 (read-only)	Only when telegram_bot output is enabled and no bot_token is set in the output config

This is the only cross-config read. To avoid it entirely, set bot_token explicitly in your output config:

CODEBLOCK3

Output credentials (optional)

Credentials are only used if you enable the corresponding output. None are required for core functionality (RSS fetch + dedup).

Output	Credential source	What is used
INLINECODE17	INLINECODE18 or bot_token in output config	Bot token (read-only)
INLINECODE20

Delegated to mail-client skill (its own creds) | Nothing read directly |
| mail-client (SMTP fallback) | smtp_user / smtp_pass in output config | SMTP login |
| nextcloud | Delegated to nextcloud-files skill (its own creds) | Nothing read directly |

Cleanup on uninstall

CODEBLOCK4

---

Security model

Credential isolation

• - API keys are read from dedicated files (default ~/.openclaw/secrets/), never from config.json. The scorer warns at runtime if a key file has overly permissive filesystem permissions.
• SMTP credentials (fallback only) are stored in the output config block — use the mail-client skill delegation to avoid storing SMTP passwords.

Subprocess boundaries

• - Dispatch delegates to other OpenClaw skills via subprocess.run() (never shell=True). Script paths are validated to reside under ~/.openclaw/workspace/skills/ before execution, preventing path traversal.
• No credentials are passed as subprocess arguments — each skill manages its own authentication.

File output safety

• - The file output type validates the target path before writing: only ~/.openclaw/ is allowed by default. Additional directories can be whitelisted via config.security.allowed_output_dirs. Sensitive paths (.ssh, .gnupg, /etc/, .bashrc, etc.) are always blocked regardless of allowlist.
• Written content is checked for suspicious patterns (shell shebangs, SSH keys, PGP blocks, code injection) and size-limited to 1 MB.

Cross-config reads

• - The only cross-config file read is ~/.openclaw/openclaw.json for the Telegram bot token, and only when telegram_bot output is enabled without an explicit bot_token. This read is logged to stderr. Set bot_token in the output config to eliminate this read entirely.

Autonomous dispatch

• - When scheduled (cron), the skill can send messages/files to configured outputs without user interaction. All dispatch actions are logged to stderr with an audit summary. Use enabled: false on any output to disable it without removing its config.

---

CLI reference

fetch

CODEBLOCK5

Options:

• - --hours N : lookback window in hours (default: from config, usually 24)
• INLINECODE43 : filter already-seen URLs (uses seenurls.json TTL store)
• INLINECODE44 : deduplicate by topic (uses topicseen.json + Jaccard similarity)
• INLINECODE45 : path to custom JSON sources file

Output (JSON on stdout):
CODEBLOCK6

seen-stats

CODEBLOCK7

Shows URL seen store statistics (count, TTL, file path).

topic-stats

CODEBLOCK8

Shows topic deduplication store statistics.

mark-seen

CODEBLOCK9

Marks one or more URLs as already seen (prevents them from appearing in future fetches with --filter-seen).

score

CODEBLOCK10

Reads a digest JSON from stdin (output of fetch) and scores articles using an OpenAI-compatible LLM.
Returns enriched JSON with scored, ghost_picks, and per-article score/reason fields.

Options:

• - --dry-run : print summary on stderr without calling the LLM API

When llm.enabled is false (default), articles pass through unchanged ("scored": false).

Pipeline usage:
CODEBLOCK11

send

CODEBLOCK12

Reads a digest JSON from stdin and dispatches to all enabled outputs configured in config.json.
Accepts both raw fetch output (articles key) and LLM-processed digests (categories key).

Output types: telegram_bot, mail-client, nextcloud, file.

• - telegram_bot: bot token auto-read from OpenClaw config - no extra setup if Telegram already configured.
• INLINECODE69: delegates to mail-client skill if installed, falls back to raw SMTP config.
• INLINECODE70: delegates to nextcloud-files skill if installed (append mode by default with date separator).
• INLINECODE71: writes digest to a local file. Path must be under ~/.openclaw/ (default) or a directory listed in config.security.allowed_output_dirs. Sensitive paths and suspicious content are blocked (see Security model).

Configure outputs interactively:
CODEBLOCK13

config

CODEBLOCK14

Prints the active configuration (no secrets).

---

LLM scoring configuration

The llm key in config.json controls the optional LLM-based article scoring:

CODEBLOCK15

Key	Default	Description
INLINECODE77	INLINECODE78	Enable LLM scoring (requires API key)
INLINECODE79

https://api.openai.com/v1 | OpenAI-compatible API endpoint |
| api_key_file | ~/.openclaw/secrets/openai_api_key | Path to file containing the API key |
| model | gpt-4o-mini | Model to use for scoring |
| top_n | 10 | Max articles to send to LLM per batch |
| ghost_threshold | 5 | Score threshold for ghost_picks (blog-worthy articles) |

Scoring rules:

• - Only the first top_n articles are sent to the LLM. Articles beyond INLINECODE91

are excluded from the digest entirely. fetch returns articles sorted by date
desc, so top_n selects the most recent ones. Increase top_n to evaluate
more articles per run (higher token cost).

• - Score >= ghost_threshold : added to ghost_picks list
• Score >= 3 : kept in articles list
• Score <= 2 : excluded from output
• Articles are sorted by score (descending)

When disabled, the score subcommand passes data through unchanged.

Nextcloud output mode

The nextcloud output now defaults to append mode with a date separator. Each dispatch adds content below a ## YYYY-MM-DD HH:MM header, preserving previous entries.

Set "mode": "overwrite" in the output config to restore the old behavior:

CODEBLOCK16

File output configuration

The file output writes digests to the local filesystem. By default, only paths under ~/.openclaw/ are allowed. To authorize additional directories, use config.security.allowed_output_dirs:

CODEBLOCK17

Blocked paths (always rejected, even if inside an allowed directory):
.ssh, .gnupg, .config/systemd, crontab, /etc/, .bashrc, .profile, .bash_profile, .zshrc, INLINECODE113

Content validation — written content is rejected if it:

• - Exceeds 1 MB
• Contains shell shebangs (#!/), SSH keys, PGP blocks, or code injection patterns (eval(, exec(, __import__(, import os, import subprocess)

All blocked attempts are logged to stderr with the reason.

---

Templates (agent usage)

Basic digest

CODEBLOCK18

Prompt template

CODEBLOCK19

Agent workflow example

CODEBLOCK20

Pipeline (CLI)

CODEBLOCK21

Filtering by keyword (post-fetch)

CODEBLOCK22

---

Ideas

• - Add keyword-based filtering (--keywords security,cve,linux)
• Add per-source TTL override in config
• Export digest as HTML or Markdown
• Schedule with cron: INLINECODE121
• Weight articles by source tier for LLM prioritization
• Add OPML import/export for source list management
• Integrate with ntfy or Telegram for real-time alerts on high-priority articles

---

Combine with

• - mail-client : send the digest by email after fetching

CODEBLOCK23

• - nextcloud-files : archive the daily digest as a Markdown file

  veille fetch --filter-seen | jq .wrapped_listing -r > /tmp/digest.md
  nextcloud-files upload /tmp/digest.md /Digests/$(date +%Y-%m-%d).md
  

---

Troubleshooting

See references/troubleshooting.md for detailed troubleshooting steps.

Common issues:

• - No articles returned: check --hours value, verify feed URLs in config
• XML parse error on a feed: some feeds use non-standard XML; the skill skips broken items silently
• All articles filtered as seen: run seen-stats to check store size; reset with INLINECODE125
• Import error: ensure you run veille.py from its directory or via full path
• File output blocked: path is outside ~/.openclaw/ — add the target directory to config.security.allowed_output_dirs (see File output configuration)