Rebase Grid API speed benchmark

How fast can you fetch electricity consumption for a zone over a given time range, from the Rebase Grid API, from the ENTSO-E Transparency Platform and straight from the national TSOs' own APIs?

This repository holds the code, the raw timings and the charts behind the "Blazingly fast API built for scale" section on rebase.energy/grid.

Headline

Median time to fetch one year of consumption data, all samples pooled across eight European zones (run 25–26 June 2026):

Path	Median	Samples	What one fetch is
Rebase Grid API	0.36 s	370	one GET /v1/zone/{zone}/consumption?start&end
Native TSO APIs	3.49 s	160	each TSO's own endpoint, assembled in the fewest calls it allows
ENTSO-E	5.33 s	351	one A65 total-load request (large XML)

Per zone, median latency and median number of HTTP requests (results/summary.json):

Zone	Grid API, 30 d	ENTSO-E, 30 d	TSO, 30 d	Grid API, 1 y	ENTSO-E, 1 y	TSO, 1 y
EE (Elering)	171 ms	1 595 ms	280 ms	215 ms	5 489 ms	2 193 ms
DK-DK1 (Energinet)	153 ms	1 636 ms	226 ms	237 ms	3 139 ms	394 ms
DK-DK2 (Energinet)	186 ms	539 ms	280 ms	240 ms	4 964 ms	403 ms
FR (RTE)	243 ms	1 146 ms	1 713 ms, 2 req	215 ms	5 742 ms	3 961 ms, 2 req
FI (Fingrid)	238 ms	1 574 ms	1 345 ms, 2 req	523 ms	6 572 ms	3 476 ms, 3 req
BE (Elia)				453 ms	4 880 ms	7 423 ms
AT (APG)				478 ms	5 648 ms	25 981 ms, 366 req
IT (Terna)				554 ms	6 430 ms	54 052 ms, 41 req

The Grid API answers any range with one request in a few hundred milliseconds. ENTSO-E is one request too, but its latency grows with the size of the XML it has to build. TSOs with a bulk endpoint (Elering, Energinet, Elia) are quick; TSOs that only serve small windows (Fingrid pages, APG one day per call, Terna one month per call) have to be assembled from many calls.

Resolution is deliberately not normalised: the benchmark times the real user action, "get consumption for this range", whatever the source returns. Data agreement between the sources is out of scope.

Method

Full design in docs/design-spec.md. In short:

• Zones: EE, DK-DK1, DK-DK2, FR, FI for the full range sweep; BE, AT, IT added for the one-year pool.
• Ranges: 1, 2, 3, 7, 14, 21, 30, 60, 90, …, 330, 365 days, all ending at the same hour-aligned anchor two days before the run.
• Repeats: 10 timed repeats per (zone, range, source) plus one discarded warm-up. Repeats are interleaved across the three sources so transient network conditions hit all of them evenly, with a 1.5 s pause between timed calls that never enters a measurement. HTTP 429 is retried honouring Retry-After; a call that still fails is recorded with ok=false.
• ENTSO-E spread run: ENTSO-E latency depends on platform load, so the back-to-back repeats were replaced by a second run that visits every (zone, range) cell once per pass in shuffled order, 12 passes. Analysis uses these spread samples for ENTSO-E wherever they exist.
• Extra one-year samples: 40 more one-year repeats per source with the zone order shuffled per repeat, to thicken the one-year distributions. The TSO samples from this run are tagged final and are the only TSO samples in the headline pool (they use the final clients, including the bulk exports).
• Statistics: per cell median, mean, p95, standard deviation, IQR, success rate and a distribution-free 95 % confidence interval on the median.

Files

Path	What it is
results/calls.jsonl	every timed fetch of the main sweep, one JSON object per line
results/entsoe_spread.jsonl	the ENTSO-E spread run
results/extra_365_{our_api,entsoe,tso}.jsonl	the extra one-year repeats
results/summary.json	per-cell statistics, keyed zone → source → days
results/web.json	the numbers used on rebase.energy/grid
results/*.png	latency, request-count and payload-size curves; density strips; bars
media/	the two branded charts that were published, as rendered on 26 June 2026
docs/design-spec.md	the design spec the run followed

Each raw record has zone, range_days, source (our_api, entsoe, tso), rep, warmup, latency_ms, n_requests, n_bytes, n_rows, ok and error, plus the tags spread, pass, refetch and final described above.

Everything under results/ other than the .jsonl files is derived and can be rebuilt from the raw logs with python -m grid_api_benchmark.analysis (summary and curves), the plot_branded* modules (branded charts) and export_web (landing-page numbers). The committed media/ PNGs were rendered with the Cal Sans and Inter fonts; without them installed the plots fall back to DejaVu Sans.

Reproduce

Python 3.11 or newer.

git clone https://github.com/rebase-energy/grid-api-benchmark
cd grid-api-benchmark
pip install -e .            # or: uv sync

Set the credentials as environment variables (see .env.example). No key is ever written to the result files: the Grid API and Fingrid keys travel in headers, and the ENTSO-E token is redacted from error messages before they are recorded.

Variable	Needed for	Where to get it
REBASE_GRID_API_KEY	the Grid API path	free at https://grid.rebase.energy
ENTSOE_API_TOKEN	the ENTSO-E path	https://transparency.entsoe.eu/, request API access
FINGRID_API_KEY	the FI native client only	https://data.fingrid.fi/

Then run the stages in order. The main sweep takes several hours; every call is checkpointed as it completes and each stage resumes where it stopped.

python -m grid_api_benchmark.run                       # main sweep -> results/calls.jsonl, summary, curves
python -m grid_api_benchmark.run_entsoe_spread --passes 12
python -m grid_api_benchmark.collect_365 --source our_api --reps 40   # run the three in parallel
python -m grid_api_benchmark.collect_365 --source entsoe  --reps 40
python -m grid_api_benchmark.collect_365 --source tso     --reps 40
python -m grid_api_benchmark.analysis                  # rebuild summary.json + curves
python -m grid_api_benchmark.plot_branded              # density strip (media/density_strip_branded.png)
python -m grid_api_benchmark.plot_branded_bars         # bars (media/bars_branded.png)
python -m grid_api_benchmark.export_web                # results/web.json

refetch_tso.py re-measures the native TSO path for chosen zones after a client fix (it was used for FR after RTE's pagination was replaced by the bulk export); animate.py renders the bars-to-density animation to MP4 and needs ffmpeg.

To try a different set of zones or ranges, edit ZONES and RANGES_DAYS in grid_api_benchmark/config.py. Adding a zone means adding a native client in tso_clients.py and the zone's ENTSO-E area code in entsoe.py.

Absolute timings depend on where and when you run: live networks and live upstream load. The committed results are the record of the June 2026 run.

Notes on the native TSO clients

The clients talk to each TSO's public API directly and count every HTTP request they make:

• Elering (EE): dashboard.elering.ee/api/system/with-plan, one call for any range.
• Energinet (DK1, DK2): ProductionConsumptionSettlement dataset with limit=200000, one call.
• RTE (FR): OpenDataSoft bulk export of the consolidated and real-time éCO2mix datasets, two calls. The earlier offset-paged search endpoint is kept as rte for reference.
• Fingrid (FI): dataset 124, 10 000 rows per page.
• Elia (BE): OpenDataSoft bulk export of ods003, one call.
• Terna (IT) and APG (AT): no bulk endpoint. The client walks the range in 24-hour windows (Terna serves a month per call, APG one local day per call) and unions the timestamps. This is the honest cost of assembling a year from these APIs.

License

MIT. Copyright (c) 2026 Rebase Energy AB.