[mats-knmi]

First step to creating a dataset validator and infiller to allow usage of any xarray dataset with pysteps

[codecov]

Codecov Report

❌ Patch coverage is 82.35294% with 12 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@1911f27). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
pysteps/xarray_helpers.py	35.29%	11 Missing ⚠️
pysteps/io/importers.py	96.55%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #594   +/-   ##
=======================================
  Coverage        ?   84.98%           
=======================================
  Files           ?      173           
  Lines           ?    15459           
  Branches        ?        0           
=======================================
  Hits            ?    13138           
  Misses          ?     2321           
  Partials        ?        0           

Flag	Coverage Δ
unit_tests	84.98% <82.35%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ritvje]

Currently in the example for using MLCast datasets in pysteps, the following metadata is required by hand to use motion estimation (LK), extrapolation nowcast and geodata_from_dataset.

precip_var = "rain_rate"
window.attrs["precip_var"] = precip_var
# Data timestep in seconds
window["time"].attrs["stepsize"] = 300

# x, y intervals and units
window["x"].attrs["stepsize"] = window.x.values[1] - window.x.values[0]
window["y"].attrs["stepsize"] = window.y.values[1] - window.y.values[0]
window["x"].attrs["units"] = "m"
window["y"].attrs["units"] = "m"

# Grid projection
window.attrs["projection"] = window[window[precip_var].grid_mapping].proj4

I'm thinking that the stepsize attributes could be taken from the dimensions directly, and w.r.t to the projection we could try to be more compliant with CF conventions?

Also, the units of the data (kg m^-2 h^-1) are not accepted by plot_precip_field (it needs to be changed to mm/h)

[kmuehlbauer]

@mats-knmi Which of the commits in this branch are actually for the dataset validator?

I tried to rebase, but feel uncomfortable which changes should be kept from this branch. Anything I should be aware of?

[kmuehlbauer]

XRef: mlwp-tools https://mlwp-tools.github.io/ via Hackathon Talk on mlwp-tools.

This changes assumptions. It looks like there is prior art where PySTEPS should connect to.
Thanks @mpvginde and @leifdenby for the inspiring presentation!

[kmuehlbauer]

@mats-knmi Would it make sense to split out the readers/writers in a dedicated package?

(cc) @pySTEPS/core-devs

[kmuehlbauer]

I tried to rebase, but feel uncomfortable which changes should be kept from this branch. Anything I should be aware of?

@mats-knmi After merging #592 this increased the churn. I'll try merging in main locally.

[kmuehlbauer]

@mats-knmi I've managed to merge main manually, so fingers crossed I did not fail.

There are back-up branches in my fork.

[kmuehlbauer]

@izzyknmi Sorry for dragging your here, but maybe you can answer this from the top of your head. What do I need to do to proper install pygrib using uv? It still uses the conda incantation.

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details
pip/rioxarray	0.19.0	Unknown	Unknown
pip/rioxarray	0.23.0	Unknown	Unknown
pip/rioxarray	0.24.0	Unknown	Unknown

Scanned Files

• uv.lock