GitHub Actions CI: CPU compilers with ECT validation

[cenamiller]

Replaces #1469 (closed prematurely).

This PR adds some initial GitHub Actions continuous integration (CI) for MPAS-A. These workflows do automated build and correctness testing for Intel OneAPI, NVHPC, and GNU compilers on CPU.

This is a subset of the testing that's been developed on the github.com/NCAR/MPAS-Model-CI fork.

The goal of this testing is to catch build issues and validate that the model output remains within the bounds of internal variability using the Ensemble Consistency Test (ECT/PyCECT; Price-Broncucia et al. 2025, doi:10.5194/gmd-18-2349-2025).

These workflows:

• Build MPAS-A in double precision with SMIOL I/O inside NCAR hpcdev Docker containers on GitHub-hosted runners. (PIO testing is available on the MPAS-Model-CI fork and will be included in future PRs)
• Test three compiler families (GNU, Intel OneAPI, NVHPC) with MPICH. (OpenMPI testing is available on the MPAS-Model-CI fork and will be included in future PRs)
• Run on new PRs against develop (or when someone pushes a new commit to an open PR) and on push to master. Only the GNU + MPICH test also runs on push to develop.
• Validate correctness by running 3 perturbed ensemble members and comparing against a previously generated 200-member PyCECT ensemble summary

README CI status badges are not included and can be added as a future feature.

Test case data and ECT ensemble summaries are still hosted as GitHub release assets on NCAR/MPAS-Model-CI. The DATA_REPOSITORY variable in .github/ci-config.env controls where data is downloaded from.

File descriptions:

• .github/ci-config.env — central CI configuration
• .github/actions/ — 8 composite actions (build-mpas, download-testdata, resolve-container, run-perturb-mpas, validate-ect, mpas-version, print-mpas-logs, ect-summary)
• .github/workflows/_test-compiler.yml — reusable CPU build + ECT validation workflow
• .github/workflows/test-{gcc,intel,nvhpc}-mpich.yml — 3 caller workflows
• .github/data/ect_excluded_vars.txt — ECT variable exclusion list

Developed in NCAR/MPAS-Model-CI. Claude assistance.

[cenamiller]

Depending on what science changes have been merged into develop at the same time, we shouldn't expect the ect to pass until I generate a new summary file. But let's see!

[mgduda]

@cenamiller Thanks for this new PR!

[mgduda]

@cenamiller @abishekg7 Following up on the comments from PR #1469 (which this PR replaces), I agree with most of them, and I'm fine with addressing some of them (e.g., adding build tests for the init_atmosphere core, and adding a CMake build test) with a future PR.

Is it the case as of now that all of the tests will run with every push to a PR branch?

[abishekg7]

@cenamiller @abishekg7 Following up on the comments from PR #1469 (which this PR replaces), I agree with most of them, and I'm fine with addressing some of them (e.g., adding build tests for the init_atmosphere core, and adding a CMake build test) with a future PR.

Is it the case as of now that all of the tests will run with every push to a PR branch?

It is now a pared down set of tests, but yes, they will be triggered with every push to a PR branch.

[cenamiller]

Hi, yes, agreed with Abishek. I've been thinking about this. It may be nice to have the "runs on every push to a PR" branch feature for a bit, because it makes it easier to trigger tests on old PRs, with just a push. Otherwise, contributors would need to close/reopen the PR in order to trigger tests. But once we've run them on some old PRs, we can very easily turn that trigger off. And anyway, if the tests make it to the default branch, you'll be able to manually dispatch tests when you think it's appropriate and we won't be so beholden to an automatic trigger.

[mgduda]

@cenamiller @abishekg7 What do you all think about (1) removing the ECT tests from this PR, and (2) adding build tests for the init_atmosphere core to this PR? My thinking is that build tests for both the init_atmosphere and atmosphere cores are probably of greatest value right now, and omitting the ECT tests would help to reduce the amount of time all of the checks will typically take to complete.

[cenamiller]

@mgduda Yes, we can do that! It won't take very long to set up, but I don't have those tests ready as stand-alones. I can submit them before lunch tomorrow though. Would you like them to run on the same triggers as these? And which compilers for each? The options for 'just build' would be GNU, Intel OneAPI, NVHPC (for CPU) and NVHPC+OpenACC. And for sides we have PIO/SMIOL and MPICH/OpenMPI.

[mgduda]

@mgduda Yes, we can do that! It won't take very long to set up, but I don't have those tests ready as stand-alones. I can submit them before lunch tomorrow though. Would you like them to run on the same triggers as these? And which compilers for each? The options for 'just build' would be GNU, Intel OneAPI, NVHPC (for CPU) and NVHPC+OpenACC. And for sides we have PIO/SMIOL and MPICH/OpenMPI.

In the interest of keeping everything as simple as possible for now, would it be reasonable to add an extra step to the build workflows to compile the init_atmosphere core before or after we compile the atmosphere core? When building a second core within an MPAS-Model source tree that already has another core (along with the shared infrastructure) compiled, the incremental cost of building the second core is not so bad. Going this route would suggest we use the same build matrix as for the atmosphere core: GNU+MPICH for Intel oneAPI, NVHPC, and GNU.

[abishekg7]

I think keeping the existing build matrix and triggers, and just adding the build of init_atmosphere core as a step sounds good. Perhaps init_atmosphere core can be built before the build of atmosphere?

[mgduda]

I think keeping the existing build matrix and triggers, and just adding the build of init_atmosphere core as a step sounds good. Perhaps init_atmosphere core can be built before the build of atmosphere?

Adding a build of the init_atmosphere core before the build of the atmosphere core sounds fine to me. (Perhaps it's over-optimizing, but my suspicion is that more code is changed in the atmosphere core than in the init_atmosphere core, so building the atmosphere core first may help us to catch build errors introduced by a PR a bit sooner. Ultimately, though, the time savings either way may not be significant enough to influence our decision on which core to build first.)