Skip to content

feat(templates): a pdf-check job in ci.yml that each site can switch off (#230) - #241

Merged
mmcky merged 5 commits into
mainfrom
feat/230-pdf-check
Nov 5, 2026
Merged

mmcky merged 5 commits into
mainfrom
feat/230-pdf-check

Conversation

@mmcky

@mmcky mmcky commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Malformed LaTeX can pass the HTML build and still break the PDF build, and a broken PDF build disrupts the publish. Previews are moving to HTML only (#148), so this PR gives templates/ci.yml a pdf-check job. It builds the PDF on every pull request and fails the PR's checks when the LaTeX build fails. The preview link never waits for it, and a site that publishes no PDF switches it off. This is decision 7 on #146.

Changes

File Change
templates/ci.yml New pdf-check job, which needs: preview. It downloads the preview job's execution cache and runs build-lectures with builder: pdflatex and the same -W --keep-going as cache.yml and publish.yml. On a failure it uploads the LaTeX .log as latex-log. The preview job gains one step at its end, after the deploy, that uploads _build/.jupyter_cache (include-hidden-files: true, kept for 1 day). It also gains an outputs: entry that carries the switch. The switch is PDF_CHECK: 'true' in a new top-level env: block. The header comment covers the switch and the required-status-check setup.
.github/workflows/test-actions.yml Two harness jobs, pdf-check-handoff and pdf-check-build, run the hand-off on the fixture in the lean image, through the template's own switch plumbing (below). Three small jobs, pdf-check-switch-*, run the PDF_CHECK: 'false' path. They sit next to env-container-update, in the file and in harness-summary's needs:. The workflow's env: gains PDF_CHECK: 'true', where the template sets it.
templates/README.md The ci.yml section and diagram mention the PDF check. A new "The PDF Check" section covers the switch, making pdf-check (and preview) required, how the job gets the executed notebooks, and re-running it once the one-day artifact has expired.
docs/user/actions/build-lectures.md One paragraph under "In the templates" on pdf-check, the switch and the required check, linking to the README section.
CHANGELOG.md Entry under Unreleased → Changed.

No action code changes.

Two choices, and why

Reusing the executed notebooks: needs: plus an artifact, not a parallel cache restore. A parallel job that restores main's build cache would start sooner. But it would execute every lecture the PR changed a second time, because those lectures are not in main's cache. On a GPU site that second execution needs a second GPU runner. With needs: preview, pdf-check takes the preview job's .jupyter_cache, which by then holds every lecture's outputs: the restored ones and the ones the PR's build executed. So it executes nothing and runs on ubuntu-latest wherever the preview job runs. The cost is that it starts only when the preview job has finished. The upload comes after the deploy, so it does not delay the link. An artifact rather than an Actions cache entry, because a cache entry per push would count against the repository's cache quota and could evict main's build caches. The artifact is kept for 1 day.

The switch: a workflow-level env: value, not a repository variable. The brief suggested vars.PDF_CHECK. But repository variables are not passed to a workflow run for a pull request from a fork. GitHub's 2023 docs said so in a callout, quoted in community discussion #44322, and replies there through 2026 still ask for that to change. The current docs no longer carry the callout, and say nothing either way. With a variable, a site that switched the check off would still run it on every fork PR, which are the contributions most likely to carry LaTeX mistakes in the first place. A value in the file applies to every PR. It is visible next to the jobs it controls, and changing it goes through review. A job-level if: cannot read env (the context table gives it github, needs, vars, inputs). So the preview job passes the value on as an output: if: needs.preview.outputs.pdf-check == 'true'. Off, the job is reported as skipped, and the preview job's upload step is skipped too.

One gotcha the template comments call out: upload-artifact skips hidden files by default, and the check applies to the path itself. @actions/glob drops any search-path item whose basename starts with .. So without include-hidden-files: true, uploading _build/.jupyter_cache would upload nothing at all.

What I checked, and how

  • The issue's "Done when", end to end, locally. I dry-ran the two harness jobs' steps with jupyter-book 1.0.4.post1 and TeX Live, in two separate directories standing in for the two jobs, with the cache copied between them.
    1. The HTML build (-W --keep-going) passes a page whose {math} block uses an undefined macro, and executes lecture1, whose marker cell leaves a file.
    2. The PDF build over the copied cache logs Using cached notebook: ID=1, leaves no marker, writes the PDF, and puts the cached output sum of squares: 285 in the .tex. The absolute paths differed between the two directories, so this is a stricter test of the hand-off than CI's identical paths.
    3. Adding the bad page to the toc makes jb exit 1 with Error: Failed to build pdf. The .log shows ! Undefined control sequence. with the macro on its context line.
  • The harness jobs assert exactly that in CI, inside quantecon-build, plus the artifact round trip: global.db must arrive under _build/.jupyter_cache after the upload and download. A missing include-hidden-files would fail there. As in build-fail-guard, the expected failure is attributed: the job checks for the LaTeX error in the log, not just a red outcome.
  • The switch, at runtime, both ways (added after review). pdf-check-handoff and pdf-check-build run the template's plumbing. PDF_CHECK in the workflow's env: is read into a job output, the upload is gated on env, and the PDF job is gated on the output. An empty output would skip pdf-check-build, which harness-summary fails. For the 'false' path, pdf-check-switch-off overrides the value at job level, as a site would. A stand-in upload step and a stand-in pdf-check job each fail if they run, and pdf-check-switch-verify checks that the output said false and that the stand-in job was skipped. Green in run 41821667551.
  • That jb fails on a LaTeX error. In jupyter-book 1.0.4, builder_specific_actions raises through _error when make all-pdf returns non-zero (jupyter_book/cli/main.py).
  • python scripts/generate-docs.py --check-examples and --check pass: the template's inputs match each action.yml, every job has permissions:, and the generated docs are current. I could not run actionlint or shellcheck locally: running a downloaded actionlint binary was blocked here. The gate runs both on this PR.
  • The pin check. actions/download-artifact@v8 was not used in any workflow, so the gate would have rejected it in a template. The harness jobs now use it, which also puts it under Dependabot.

What to watch after merge

  • Before the first adopter: the canary. The work plan (Work plan — QuantEcon/actions: v0.17.0 shipped; next the v0 move and wave 6's gate and workers #246) records a trial of this ci.yml on QuantEcon/test-actions-lecture-intro, with the default and with PDF_CHECK: 'false', to measure the artifact round trip on a real book first.
  • The first adopter, lecture-python.myst for lecture-python.myst: HTML-only PR previews; three-format build behind a full-preview label #148. Check the artifact's size and the upload and download times: its .jupyter_cache is the largest of the lecture repos. Also check that pdf-check passes on a PR that changes nothing LaTeX-related. A red there would mean its PDF build already fails on main's content.
  • "Re-run failed jobs" on a failed pdf-check. The re-run downloads the artifact the first attempt's preview job uploaded. Within the artifact's one day I expect that to work, since a partial re-run keeps the run's artifacts, but no test covers it. After that day it fails at the download, and the README says to use "Re-run all jobs".
  • A fork PR on a site that sets PDF_CHECK: 'false' should show pdf-check as skipped.

Not in this PR

Closes #230.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WT8bMXnw7kDFR66khoSGa9

…off (#230)

Malformed LaTeX can pass the HTML build and still break the PDF build,
and with it the publish. Previews are moving to HTML only (#148), so
ci.yml gains a pdf-check job that builds the PDF on every pull request
and fails its checks when the LaTeX build fails.

pdf-check needs the preview job, so the preview link never waits for
it, and builds from the _build/.jupyter_cache the preview job uploads
as an artifact after its deploy. It executes no notebook, so it needs
no GPU runner, and a re-run never doubles the execution cost. The
alternative, restoring main's build cache in a parallel job, would
execute every changed lecture twice. The upload sets
include-hidden-files: upload-artifact skips a hidden directory without
it, even when the path names that directory.

The switch is PDF_CHECK in the workflow's top-level env:, read by the
job through a preview-job output (a job-level if: cannot read env).
It is in the file rather than a repository variable because a run for
a fork's pull request does not receive repository variables. Off, the
job reports as skipped and the preview job uploads nothing.

Two harness jobs run the hand-off on the fixture in the lean image:
the HTML build passes math that pdflatex rejects; the cache arrives;
the PDF builds from it with no notebook executed (a marker cell proves
it); and adding the bad math fails the PDF build on that error. They
also pin actions/download-artifact@v8 in a workflow, as the gate's pin
check requires of every action a template uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMXnw7kDFR66khoSGa9
…ault shell

pdf-check-build's assertion used compgen, a bash builtin. A container
job's run: steps default to sh -e, so it failed with "compgen: not
found" although the PDF had been built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMXnw7kDFR66khoSGa9

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟡 Changes recommended

A critical issue remains in templates/ci.yml where a hyphenated job output key (pdf-check) will be misinterpreted in GitHub Actions expressions, preventing the dependent if: condition from evaluating as intended.

1 open issue
What changed in this PR

This pull request adds an optional (default-on) PDF build verification job to the CI workflow template so LaTeX/PDF-only failures are caught on pull requests even when the HTML build and preview succeed, while keeping preview deployment behavior unchanged and allowing sites to disable the check.

Changes:

  • Add a pdf-check job to templates/ci.yml that runs a pdflatex build using the executed-notebook cache handed off from the preview job via an artifact.
  • Extend .github/workflows/test-actions.yml with harness coverage to validate the preview → artifact → pdf-check handoff and expected LaTeX-only failure behavior.
  • Update template/user documentation and the changelog to describe the new job, the PDF_CHECK switch, and required-check guidance.
File Review Notes
templates/​README.md Nit: grammar—“executes no notebook” should be “executes no notebooks”.
templates/​ci.yml Critical: job output key uses a hyphenated name (pdf-check), which can be parsed as subtraction in expressions (e.g., needs.preview.outputs.pdf-check), so the if: condition may not work as intended. Use an underscore output name (or bracket notation) for reliable access.
docs/​user/​actions/​build-lectures.md Nit: grammar/readability—sentence is missing a word and should more clearly describe LaTeX errors that pass HTML but fail PDF; also pluralize “notebooks”.
CHANGELOG.md No additional issues noted.
.github/​workflows/​test-actions.yml No additional issues noted.

This pull request requires changes.

Review effort: Balanced

Comment thread templates/ci.yml
Comment on lines +66 to +67
outputs:
pdf-check: ${{ env.PDF_CHECK }}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No change here: the hyphen is safe. GitHub Actions expressions have no subtraction operator. The operator table lists only ( ), [ ], ., !, comparisons, && and ||. And the contexts reference says that "the property name must start with a letter or _ and contain only alphanumeric characters, -, or _". So needs.preview.outputs.pdf-check is a plain property dereference. This repository's actions rely on the same thing throughout, for example steps.restore.outputs.cache-matched-key in restore-jupyter-cache and inputs.save-cache. The gate's actionlint run, which type-checks these expressions, passes on this template.

The two wording nits in the review summary were fair, and are fixed in 3e21e0f: "executes no notebooks", and the build-lectures chapter's first sentence about pdf-check is now split in two so it parses.

Claude Code (on behalf of mmcky)

…d-lectures chapter

From the Copilot review on #241: "executes no notebooks", and the
build-lectures paragraph's first sentence split in two so it parses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMXnw7kDFR66khoSGa9

@mmcky mmcky left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the template, the two harness jobs and the docs against main. The hand-off is designed and proven the way the body says, the Copilot hyphen claim was rightly declined, and the include-hidden-files trap is caught by the harness's global.db assertion. Nothing to change in the code; one inline note on the part of the template that no run has yet exercised, with a cheap way to exercise it before the first adopter. For merge order: this branch conflicts with #245 in harness-summary's needs: list and the CHANGELOG, both one line, and is clean against the other three.

Claude Code (on behalf of mmcky)

Comment thread templates/ci.yml

# Tells pdf-check whether to run (job-level if: cannot read env).
outputs:
pdf-check: ${{ env.PDF_CHECK }}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the one part of the change that no run has evaluated. The two harness jobs prove the hand-off (the upload with the hidden directory, the download, the cached build, the LaTeX-only failure), and the gate's actionlint type-checks the expression, but nothing evaluates env in a job's outputs: at runtime, or needs.preview.outputs.pdf-check in the if:, or the PDF_CHECK: 'false' path (job reported skipped, upload step skipped). The context table does list env for jobs.<job_id>.outputs, so I expect it to work; it is just unexercised. Since lecture-python.myst is the first adopter and its .jupyter_cache is the largest, a cheaper first proof is the canary: give QuantEcon/test-actions-lecture-intro this ci.yml (its toc is five lectures, so the artifact is small), open one PR with the default and one with PDF_CHECK: 'false', and read the two pdf-check results. That also measures the artifact round trip before the big site does.

One more case for the README's re-run note: with retention-days: 1, a "Re-run failed jobs" of pdf-check the next day fails at the download, and "Re-run all jobs" is the answer, as the harness already says of itself.

Claude Code (on behalf of mmcky)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that this was the unexercised part. It's now covered in this repo's harness rather than only on the canary, in b2fd15a, and run 41821667551 is green.

  • The 'true' path, through the template's own plumbing. The harness now sets PDF_CHECK: 'true' in its workflow-level env:, where the template sets it. pdf-check-handoff reads it into outputs: pdf-check: ${{ env.PDF_CHECK }} and gates its upload on env.PDF_CHECK == 'true'. pdf-check-build runs only if: needs.pdf-check-handoff.outputs.pdf-check == 'true'. If the output came through empty, pdf-check-build would be skipped, and harness-summary fails a skipped job. It ran and passed.
  • The 'false' path. pdf-check-switch-off sets PDF_CHECK: 'false' at job level, overriding the workflow's value as a site's own env: block would. It has the same output and a stand-in upload step that fails if it runs: passed, so the step was skipped. A stand-in for pdf-check, gated like the template's job, fails if it runs: skipped. pdf-check-switch-verify checks that the output carried false and that the stand-in's result was skipped: passed. The stand-in is not in harness-summary's needs:, since it must be skipped; the verify job certifies it instead.

The canary trial is still worth doing for what the harness can't give: an artifact round trip measured on a real book before lecture-python.myst adopts it. The organiser has recorded it in the work plan (#246) as a step before #148, outside this PR.

The re-run note is in the templates README's "The PDF Check" section, with a pointer next to retention-days in the template: once the one-day artifact has expired, re-running pdf-check alone fails at the download, so use "Re-run all jobs".

Claude Code (on behalf of mmcky)

mmcky and others added 2 commits November 4, 2026 22:31
From the review on #241: nothing evaluated at runtime the template's
env-to-output switch, the job-level if: on the output, or the 'false'
path. actionlint type-checks them but does not run them.

pdf-check-handoff and pdf-check-build now use the template's plumbing:
PDF_CHECK in the workflow's env:, read into a job output, the upload
step gated on env and the PDF job gated on the output. An empty
output would skip pdf-check-build, which harness-summary fails. Three
small jobs run the 'false' path with a job-level override. A stand-in
upload step and a stand-in pdf-check job each fail if they run, and a
verify job checks the output said 'false' and the stand-in job was
skipped.

The templates README also says to re-run all jobs once the one-day
hand-off artifact has expired, since pdf-check alone then fails at the
download.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMXnw7kDFR66khoSGa9
The merge applied cleanly but left the pdf-check entry under
[0.13.0]; it moves back to [Unreleased] under a new ### Changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMXnw7kDFR66khoSGa9
@mmcky
mmcky merged commit 2c06c01 into main Nov 5, 2026
34 checks passed
@mmcky
mmcky deleted the feat/230-pdf-check branch November 5, 2026 00:48
mmcky added a commit that referenced this pull request Nov 5, 2026
Move [Unreleased] into a dated 0.14.0 section, carrying all of wave 1 (#246): #240, #241, #242, #244 and #245. Add an entry for Dependabot's quantecon-book-theme 0.22.1 bump in both images (#223), which had none. #240 changes every consumer's PR path, so the release is staged on the canary before v0 moves. This commit changes only CHANGELOG.md, and the harness has no @v0 workaround for this release to drop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WT8bMbMmeUUEj8EjiweB6S
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

templates/ci.yml: a parallel PDF check that each site can switch off

2 participants