- Python 54.5%
- TeX 45.5%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Replace placeholder README with comprehensive documentation covering uv setup, full pipeline walkthrough, map generation CLI, model architecture notes, and link to PVGIS scraper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
| documents | ||
| node_modules | ||
| plots | ||
| public | ||
| scripts | ||
| .gitignore | ||
| .python-version | ||
| bun.lock | ||
| package.json | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
| title |
|---|
| Global PV Power Forecasting |
Globally generalized photovoltaic power prediction using hourly simulation data from the European Commission's PVGIS database and three machine learning approaches: XGBoost tabular, XGBoost two-stage cascade, and a Spatio-Temporal Graph Neural Network (ST-Graph).
The project demonstrates that spatial architecture (graph convolutions over a geographic sensor network) substantially outperforms tabular approaches for solar forecasting, achieving 90.7% R² without any real-time weather data — surpassing the 78% single-location literature baseline while generalizing to coordinates never seen during training.
Results
| Model | MAE (W) | RMSE (W) | R² |
|---|---|---|---|
| Literature Baseline (single location) | — | — | 78.0% |
| XGBoost Tabular | 67.04 | 108.88 | 80.8% |
| XGBoost Two-Stage | 55.51 | 104.72 | 82.2% |
| ST-Graph | 37.07 | 75.00 | 90.7% |
| XGBoost Enriched (oracle) | 1.75 | 3.61 | 99.98% |
Data
~300 stations sampled globally from PVGIS (European Commission JRC). Each station provides hourly simulated PV output at 1 kWp with 14% system losses over 2005–2020, totaling ~42 million rows.
Data is acquired using a Rust scraper
that samples random land coordinates from a bitmap mask and queries the
PVGIS REST API. The CSV files are stored externally (not in this repository)
and referenced by the training scripts via --data-dir.
Project structure
├── scripts/
│ ├── eda.py # EDA figures (timeseries, scatter, correlation)
│ ├── train_xgboost.py # Train public, enriched, and two-stage XGBoost
│ ├── train_stgraph.py # Train ST-Graph GCN
│ ├── feature_importance.py # LightGBM feature importance plot
│ ├── export_plots.py # Render static PNGs for report and slides
│ └── generate_map.py # Interactive forecast map generation (CLI)
├── documents/
│ ├── report.tex # Academic report (LuaLaTeX)
│ ├── report.pdf # Compiled report
│ ├── slides.mkd # Slidev presentation (Spanish)
│ └── Proyecto_Final_Cálculo_Integral.pdf # Prior work (literature baseline)
├── plots/ # All PNGs referenced by report and slides
├── artifacts/ # Generated at runtime (.gitignored)
│ ├── models/ # Saved model files
│ └── *.csv / *.html # Metrics, predictions, interactive maps
├── public/plots # Symlink for Slidev image resolution
├── pyproject.toml # Python dependencies (managed with uv)
└── uv.lock
Setup
Install uv and sync the environment:
uv sync
All commands below assume the virtualenv is active (source .venv/bin/activate
or prefix with uv run).
Pipeline
Scripts are designed to run in order. Each caches its outputs and skips
re-computation on subsequent runs unless --force-retrain is passed.
Generate EDA plots
python scripts/eda.py
Reads PVGIS CSVs from --data-dir (default /mnt/aether/PVGIS/csv) and
produces plots/eda_timeseries.png, plots/eda_scatter.png, and
plots/eda_corr.png.
Train XGBoost models
python scripts/train_xgboost.py
Trains three XGBoost variants (public, enriched, two-stage) with GPU
acceleration (tree_method='hist', device='cuda'). Saves models to
artifacts/models/xgb_*.ubj and metrics to
artifacts/model_comparison_xgboost.csv.
Train ST-Graph
python scripts/train_stgraph.py
Builds a Haversine k-NN graph over ~300 stations and trains a 3-layer GCN
with residual connections using PyTorch Geometric. Saves model to
artifacts/models/stgraph.pt and graph metadata to
artifacts/models/stgraph_meta.npz.
Feature importance
python scripts/feature_importance.py
Trains a LightGBM enriched model and plots the top-10 features by split
count. Output: plots/feature_importance_top10.png.
Export static plots for report
python scripts/export_plots.py
Reads metrics CSVs and dense map prediction CSVs from artifacts/ and
renders plots/model_comparison.png and plots/map_*.png for embedding
in the LaTeX report and Slidev slides.
Generate forecast maps
python scripts/generate_map.py --model xgboost --timestamp "2025-06-21 12:00"
python scripts/generate_map.py --model stgraph --timestamp "2020-06-21 12:00" --n 10000
python scripts/generate_map.py --model xgboost --timestamp "2024-12-21 12:00" --export-png
Loads a saved model and predicts PV power at N random land points (default
25,000) for the given UTC timestamp. Outputs an interactive Plotly HTML map
and a prediction CSV to artifacts/. Pass --export-png for a static PNG.
Restrict to a geographic region with --bbox lat_min lat_max lon_min lon_max:
python scripts/generate_map.py \
--model xgboost \
--timestamp "2025-03-20 10:00" \
--n 50000 \
--bbox -10 50 -20 45
Increasing --n within a small bounding box raises point density for
high-resolution regional analysis without retraining.
Compile the report
Requires LuaLaTeX. Run twice so cross-references resolve:
lualatex documents/report.tex && lualatex documents/report.tex
mv report.pdf documents/report.pdf
Open the presentation
Requires Nix for Slidev:
nix shell 'github:NixOS/nixpkgs/nixos-unstable#slidev-cli' \
--command slidev documents/slides.mkd
Model architecture notes
XGBoost Tabular predicts power from coordinates and cyclical time features only (lat, lon, hour_sin, hour_cos, doy_sin, doy_cos, month). It learns the average solar climatology of 2005–2020 and can predict at any future coordinate and timestamp, but cannot distinguish clear from cloudy days.
XGBoost Two-Stage first predicts meteorological variables (irradiance, temperature, wind speed) from public features, then uses those predictions alongside public features to estimate power. The improvement over single-stage is modest because the first stage can only learn climatological averages.
ST-Graph represents the global station network as a graph with Haversine k-NN edges (k=5). Each node carries 24 hours of real power history plus cyclical time features. Three GCN layers with a residual connection aggregate neighborhood information, allowing the model to track weather fronts across the sensor network. It is a one-step-ahead forecaster: it requires live sensor readings and predicts only the next hour.