Electricity Forecasting Framework

Overview

This skill provides end-to-end support for electricity load/demand forecasting projects, from data preprocessing to model deployment. It covers traditional statistical methods, modern machine learning approaches, and state-of-the-art deep learning architectures.

Quick Start

1. Define Your Forecasting Task

Horizon	Type	Typical Use
1-48 hours	Short-term (STLF)	Grid operations, unit commitment
1 week - 1 month

Medium-term | Maintenance scheduling, fuel planning | | 1-12 months | Long-term (LTLF) | Capacity planning, infrastructure investment |

2. Prepare Your Data

CODEBLOCK0

Required data columns:

• - timestamp: Datetime index (hourly or sub-hourly)
• INLINECODE1: Target variable (MW or kWh)
• INLINECODE2: Weather feature (°C)
• Optional: humidity, windspeed, solarradiation, holiday_flag

3. Select Your Model

See references/model-selection.md for detailed guidance.

Quick recommendation:

• - Baseline: Start with persistence or INLINECODE4
• Production STLF: Use XGBoost or LightGBM with weather features
• Research/SOTA: Try Temporal Fusion Transformer (TFT) or INLINECODE8

4. Train and Evaluate

CODEBLOCK1

Key metrics to track:

• - MAPE (%): Mean Absolute Percentage Error - business interpretability
• RMSE (MW): Root Mean Square Error - penalizes large errors
• MAE (MW): Mean Absolute Error - robust to outliers
• Coverage (%): Prediction interval coverage probability

Core Workflows

Data Preprocessing

1. 1. Load raw data with proper datetime parsing
2. Handle missing values: Forward-fill for short gaps, interpolate for longer
3. Feature engineering:

- Temporal: hour, dayofweek, month, isweekend, isholiday - Lag features: loadt-1, loadt-24, load_t-168 (weekly) - Rolling stats: rollingmean24h, rollingstd7d - Weather: temperature, humidity, apparent_temperature

1. 4. Normalization: RobustScaler or MinMaxScaler for deep learning models

See references/feature-engineering.md for complete feature list.

Model Training

CODEBLOCK2

Hyperparameter Tuning

Use scripts/hyperparameter_search.py for automated tuning:

CODEBLOCK3

Uncertainty Quantification

For risk-aware decision making:

• - Quantile Regression: Predict multiple quantiles (0.1, 0.5, 0.9)
• Conformal Prediction: Distribution-free uncertainty bounds
• Ensemble Methods: Model disagreement as uncertainty proxy
• Monte Carlo Dropout: For neural networks

See references/uncertainty.md for implementation details.

Model Reference

Statistical Models

Model	Best For	Pros	Cons
ARIMA	Stable series	Interpretable, fast	Assumes linearity
SARIMA

Strong seasonality | Captures daily/weekly patterns | Manual parameter tuning | | Prophet | Multiple seasonalities | Handles holidays well | Less accurate for STLF | | TBATS | Complex seasonality | Automatic parameter selection | Slower training |

Machine Learning Models

Model	Best For	Pros	Cons
XGBoost	Production STLF	Fast, accurate, handles missing	No native uncertainty
LightGBM

Large datasets | Faster than XGBoost, memory efficient | Sensitive to hyperparameters | | Random Forest | Baseline ML | Robust, easy to tune | Lower accuracy than boosting | | CatBoost | Categorical features | Handles categoricals natively | Slower training |

Deep Learning Models

Model	Best For	Pros	Cons
LSTM	Sequential patterns	Captures long-term dependencies	Slow training, hard to tune
GRU

Similar to LSTM | Faster convergence | Similar limitations | | Transformer | Long sequences | Parallel training, attention | Data-hungry, complex | | TFT | Multi-horizon | Interpretable attention, uncertainty | Complex implementation | | N-BEATS | Pure deep learning | Strong baseline, interpretable | Less flexible than TFT | | iTransformer | SOTA performance | Inverted transformer architecture | Recent, less battle-tested |

See references/deep-learning-models.md for architecture details and PyTorch implementations.

Evaluation Best Practices

Time Series Cross-Validation

Never use random k-fold! Use expanding or sliding window:

CODEBLOCK4

Backtesting Framework

CODEBLOCK5

Benchmark Comparison

Always compare against:

1. 1. Persistence: loadt = loadt-1
2. Seasonal Naive: loadt = loadt-24 (for hourly data)
3. Weekly Naive: loadt = loadt-168

Deployment

Production Pipeline

1. 1. Model serialization: Save with joblib or ONNX
2. Feature pipeline: Ensure identical preprocessing at inference
3. Scheduling: Cron or Airflow for automated forecasts
4. Monitoring: Track forecast drift and retrain triggers

See references/deployment.md for MLOps patterns.

Real-time Inference

CODEBLOCK6

Common Pitfalls

1. 1. Data leakage: Ensure no future information in features
2. Holiday handling: Special days need explicit modeling
3. Temperature nonlinearity: Use heating/cooling degree days
4. Concept drift: Retrain quarterly or when MAPE degrades >20%
5. Peak prediction: Models often under-predict peaks - consider quantile loss

Resources

• - Feature Engineering Guide
• Model Selection Guide
• Deep Learning Architectures
• Uncertainty Quantification
• Deployment Patterns
• Datasets Reference

Scripts

Script	Purpose
INLINECODE10	Data cleaning and feature engineering
INLINECODE11

Model training with validation | | scripts/hyperparameter_search.py | Automated hyperparameter optimization | | scripts/backtest.py | Time series cross-validation | | scripts/evaluate.py | Comprehensive metric calculation | | scripts/deploy_model.py | Export model for production |

Example Usage

CODEBLOCK7