Skip to content

pyop3: Only include petsc.h in kernels that need it - #5564

Open
pbrubeck wants to merge 1 commit into
mainfrom
pbrubeck/pyop3-petsc-include
Open

pbrubeck wants to merge 1 commit into
mainfrom
pbrubeck/pyop3-petsc-include

Conversation

@pbrubeck

Copy link
Copy Markdown
Contributor

Summary

Every loop generated by pyop3 included <petsc.h>, which took most of the time needed to compile a small kernel. Most loops use no PETSc: in the reynolds_robust_navier_stokes_hdiv demo, 78 of the 122 compiled loops have neither a matrix argument nor a local kernel. These are mostly the layout, map, sparsity and assignment loops.

The header is now only included for loops that need it:

 LoopyCodegenContext.finalize_kernel
   preambles = [stdio.h]
-  preambles += [petsc.h]
+  if subkernels or any argument is a Mat (OpaqueType)
+    preambles += [petsc.h]          # local kernels may use PetscScalar etc.

 with_attach_debugger(kernel)
+  preambles += [petscsys.h]         # like with_petsc_event and petsclog.h

Evidence

Compiling the generated loop idat_0[i] = i with the flags of pyop3 (mpicc, GCC 15):

wall time
with <petsc.h> 0.14 s
without <petsc.h> 0.04 s

tests/firedrake/demos/test_demos_run.py -k reynolds, with fresh caches, run side by side:

  • Before: 277 s cold, 234 s warm
    After: 268.5 s cold, 238 s warm

The warm difference is within the noise between runs.

Merge Danger

Door: two-way

Blast Radius: small

Only the preambles of generated loops change. A loop that uses a PETSc symbol without a matrix argument or a local kernel would now fail to compile. No such loop is generated in the demo. Cached kernels are keyed by their code, so they are recompiled once.


AI assistance: this PR was written with Claude Code (Claude Opus 5.5).

🤖 Generated with Claude Code

Every generated loop included <petsc.h>, which takes most of the time
needed to compile a small kernel (0.14 s against 0.04 s without it for a
trivial loop). Most loops (78 of 122 in the Reynolds demo) use no PETSc at
all. The header is now only included for loops that have matrix arguments
or call local kernels, and with_attach_debugger includes petscsys.h itself,
like with_petsc_event already does for petsclog.h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGJuwu7dPzKbSmATxWo9Qc
@pbrubeck pbrubeck added the LLM used An LLM was used in the production of this PR label Oct 26, 2026
@pbrubeck
pbrubeck marked this pull request as ready for review October 26, 2026 21:29
@pbrubeck pbrubeck added the pyop3 label Oct 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

LLM used An LLM was used in the production of this PR pyop3

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant