No description
  • Python 54.5%
  • TeX 45.5%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
yuuhikaze 58b99af9c8 docs: rewrite README with proper setup, pipeline, and architecture
Replace placeholder README with comprehensive documentation covering
uv setup, full pipeline walkthrough, map generation CLI, model
architecture notes, and link to PVGIS scraper.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-10 23:28:06 -05:00
documents docs: add references section and include prior work PDF 2026-05-10 23:18:54 -05:00
node_modules cmon 2026-05-05 07:02:23 -05:00
plots feat: reorganize project for submission 2026-05-10 23:09:07 -05:00
public fix: resolve Slidev image paths via public/ symlink 2026-05-05 07:04:53 -05:00
scripts feat: reorganize project for submission 2026-05-10 23:09:07 -05:00
.gitignore docs: add references section and include prior work PDF 2026-05-10 23:18:54 -05:00
.python-version feat: add global LightGBM training pipeline 2026-05-04 21:08:47 -05:00
bun.lock cmon 2026-05-05 07:02:23 -05:00
package.json cmon 2026-05-05 07:02:23 -05:00
pyproject.toml feat: vastly expand academic report with EDA plots and spatial visualization implications 2026-05-05 05:40:56 -05:00
README.md docs: rewrite README with proper setup, pipeline, and architecture 2026-05-10 23:28:06 -05:00
uv.lock feat: vastly expand academic report with EDA plots and spatial visualization implications 2026-05-05 05:40:56 -05:00

title
Global PV Power Forecasting

Globally generalized photovoltaic power prediction using hourly simulation data from the European Commission's PVGIS database and three machine learning approaches: XGBoost tabular, XGBoost two-stage cascade, and a Spatio-Temporal Graph Neural Network (ST-Graph).

The project demonstrates that spatial architecture (graph convolutions over a geographic sensor network) substantially outperforms tabular approaches for solar forecasting, achieving 90.7% R² without any real-time weather data — surpassing the 78% single-location literature baseline while generalizing to coordinates never seen during training.

Results

Model MAE (W) RMSE (W) R²
Literature Baseline (single location) — — 78.0%
XGBoost Tabular 67.04 108.88 80.8%
XGBoost Two-Stage 55.51 104.72 82.2%
ST-Graph 37.07 75.00 90.7%
XGBoost Enriched (oracle) 1.75 3.61 99.98%

Data

~300 stations sampled globally from PVGIS (European Commission JRC). Each station provides hourly simulated PV output at 1 kWp with 14% system losses over 2005–2020, totaling ~42 million rows.

Data is acquired using a Rust scraper that samples random land coordinates from a bitmap mask and queries the PVGIS REST API. The CSV files are stored externally (not in this repository) and referenced by the training scripts via --data-dir.

Project structure

├── scripts/
│   ├── eda.py                # EDA figures (timeseries, scatter, correlation)
│   ├── train_xgboost.py      # Train public, enriched, and two-stage XGBoost
│   ├── train_stgraph.py      # Train ST-Graph GCN
│   ├── feature_importance.py # LightGBM feature importance plot
│   ├── export_plots.py       # Render static PNGs for report and slides
│   └── generate_map.py       # Interactive forecast map generation (CLI)
├── documents/
│   ├── report.tex            # Academic report (LuaLaTeX)
│   ├── report.pdf            # Compiled report
│   ├── slides.mkd            # Slidev presentation (Spanish)
│   └── Proyecto_Final_Cálculo_Integral.pdf  # Prior work (literature baseline)
├── plots/                    # All PNGs referenced by report and slides
├── artifacts/                # Generated at runtime (.gitignored)
│   ├── models/               # Saved model files
│   └── *.csv / *.html        # Metrics, predictions, interactive maps
├── public/plots              # Symlink for Slidev image resolution
├── pyproject.toml            # Python dependencies (managed with uv)
└── uv.lock

Setup

Install uv and sync the environment:

uv sync

All commands below assume the virtualenv is active (source .venv/bin/activate or prefix with uv run).

Pipeline

Scripts are designed to run in order. Each caches its outputs and skips re-computation on subsequent runs unless --force-retrain is passed.

Generate EDA plots

python scripts/eda.py

Reads PVGIS CSVs from --data-dir (default /mnt/aether/PVGIS/csv) and produces plots/eda_timeseries.png, plots/eda_scatter.png, and plots/eda_corr.png.

Train XGBoost models

python scripts/train_xgboost.py

Trains three XGBoost variants (public, enriched, two-stage) with GPU acceleration (tree_method='hist', device='cuda'). Saves models to artifacts/models/xgb_*.ubj and metrics to artifacts/model_comparison_xgboost.csv.

Train ST-Graph

python scripts/train_stgraph.py

Builds a Haversine k-NN graph over ~300 stations and trains a 3-layer GCN with residual connections using PyTorch Geometric. Saves model to artifacts/models/stgraph.pt and graph metadata to artifacts/models/stgraph_meta.npz.

Feature importance

python scripts/feature_importance.py

Trains a LightGBM enriched model and plots the top-10 features by split count. Output: plots/feature_importance_top10.png.

Export static plots for report

python scripts/export_plots.py

Reads metrics CSVs and dense map prediction CSVs from artifacts/ and renders plots/model_comparison.png and plots/map_*.png for embedding in the LaTeX report and Slidev slides.

Generate forecast maps

python scripts/generate_map.py --model xgboost --timestamp "2025-06-21 12:00"
python scripts/generate_map.py --model stgraph --timestamp "2020-06-21 12:00" --n 10000
python scripts/generate_map.py --model xgboost --timestamp "2024-12-21 12:00" --export-png

Loads a saved model and predicts PV power at N random land points (default 25,000) for the given UTC timestamp. Outputs an interactive Plotly HTML map and a prediction CSV to artifacts/. Pass --export-png for a static PNG.

Restrict to a geographic region with --bbox lat_min lat_max lon_min lon_max:

python scripts/generate_map.py \
  --model xgboost \
  --timestamp "2025-03-20 10:00" \
  --n 50000 \
  --bbox -10 50 -20 45

Increasing --n within a small bounding box raises point density for high-resolution regional analysis without retraining.

Compile the report

Requires LuaLaTeX. Run twice so cross-references resolve:

lualatex documents/report.tex && lualatex documents/report.tex
mv report.pdf documents/report.pdf

Open the presentation

Requires Nix for Slidev:

nix shell 'github:NixOS/nixpkgs/nixos-unstable#slidev-cli' \
  --command slidev documents/slides.mkd

Model architecture notes

XGBoost Tabular predicts power from coordinates and cyclical time features only (lat, lon, hour_sin, hour_cos, doy_sin, doy_cos, month). It learns the average solar climatology of 2005–2020 and can predict at any future coordinate and timestamp, but cannot distinguish clear from cloudy days.

XGBoost Two-Stage first predicts meteorological variables (irradiance, temperature, wind speed) from public features, then uses those predictions alongside public features to estimate power. The improvement over single-stage is modest because the first stage can only learn climatological averages.

ST-Graph represents the global station network as a graph with Haversine k-NN edges (k=5). Each node carries 24 hours of real power history plus cyclical time features. Three GCN layers with a residual connection aggregate neighborhood information, allowing the model to track weather fronts across the sensor network. It is a one-step-ahead forecaster: it requires live sensor readings and predicts only the next hour.