Progressive Validator

Stop wasting 3 hours on a backtest that was doomed from the start. This skill implements a multi-stage validation pipeline that eliminates bad strategies in 15 minutes instead of 3 hours.

When to use

• - "Validate this strategy"
• "Run the progressive test pipeline"
• "Is this strategy worth a full backtest?"
• When planning the validation sequence for a new strategy variant

The Pipeline

CODEBLOCK0

Without progressive validation: Every failed strategy costs 3 hours.
With progressive validation: Most failures caught in 15-45 minutes.

Validation Stages

Stage 1: Smoke Test

• - Period: 2024-01-01 to 2024-03-31 (3 months)
• Time: ~15-20 minutes
• Threshold: Drawdown < 50%
• Purpose: Catch compilation errors, logic bugs, and catastrophic structural flaws
• What it covers: Q1 2024 (includes major tech rallies)

Stage 2: Stress Test

• - Period: 2024-02-01 to 2024-06-30 (5 months)
• Time: ~25-30 minutes
• Threshold: Drawdown < 45%
• Purpose: Test survival during the hardest market conditions
• What it covers: 2024 H1 — historically the worst "meat grinder" period for options strategies

Stage 3: Medium

• - Period: 2024-01-01 to 2025-06-30 (18 months)
• Time: ~45-60 minutes
• Threshold: Drawdown < 42%
• Purpose: Validate across bull/bear transitions and seasonal effects
• What it covers: Full 2024 volatility + 2025 early recovery

Stage 4: Full Period

• - Period: 2023-01-01 to 2026-01-31 (3 years)
• Time: ~2-3 hours
• Threshold: Drawdown < 40%, Sharpe >= 2.0, Profit >= 300%
• Purpose: Final acceptance test — benchmark against proven strategies
• What it covers: Complete market cycle including 2023 AI rally, 2024 correction, 2025 recovery

Usage

Configure windows

Define your validation windows in config:

CODEBLOCK1

Run each stage

Prerequisite: This skill coordinates validation stages. Actual backtest submission
is handled by the backtest-poller skill (cli.py). Ensure that skill is installed
and available on your path before running these commands.

CODEBLOCK2

Skip Rules

Not every change needs to start from Smoke:

Early-Stop Integration

This skill works alongside the backtest-poller skill (a separate package). The
backtest-poller's early-stop feature monitors drawdown in real time and deletes the
backtest run if the threshold is exceeded after 20% progress — no need to wait for
full completion of a doomed run. This validator tracks which stages passed or failed
locally, so you always know where to resume.

Dependency: Install the backtest-poller skill to enable submit/early-stop
functionality. This validator does not submit backtests itself.

Time Savings Example

Testing 5 strategy variants, 3 of which are bad:

Savings: ~70% of compute time.

Rules

• - Never skip stages without justification. The skip rules table above defines the only valid exceptions. If entry logic or survival structure changed, you must start from Smoke.
• A strategy must pass a stage before advancing. Do not promote a strategy to the next stage if the current stage resulted in early-stop or failure.
• Do not modify stage thresholds mid-validation. Changing max_dd between stages invalidates the progressive guarantee. Decide thresholds before starting.
• One strategy variant per validation run. Do not change the strategy code between stages — the point is to validate the same code across increasingly demanding windows.
• Record every result, even failures. Use python3 validator.py record <strategy> <stage> --status passed|failed to persist outcomes. Unrecorded results break the next and status commands.