Repository navigation
Description
A test-containers-lectures.yml run failed on lectures/hansen_singleton_1983.md in QuantEcon/lecture-python.myst — CellExecutionError / HTTP Error 429: Too Many Requests: a hidden cell fetches its dataset from raw GitHub at build time (lectures/hansen_singleton_1983.md:1432-1439).
The matrix builds several repos at once from GitHub's shared runner IPs, so raw GitHub rate-limits it — non-deterministically: the same notebook passed on the other container in the same run. It surfaced during the Anaconda 2026.06 migration (#95), but the cause is the fetch, not the migration or pandas.
Lecture-side fixes do not work: local _static/ paths break Colab, since QuantEcon/lecture-python.notebooks is a flat .ipynb directory with no _static/ (QuantEcon/lecture-python.myst#972, closed for that reason). It is not lecture-specific either — lectures/hansen_singleton_1982.md:1000, lectures/pandas_panel.md:68,188,505 and lectures/phillips_drifts_volatilities.md:103 fetch over raw GitHub too. Moving datasets to data-lectures does not help: pandas_panel.md already reads from that repo over the same github.com/.../raw/ host.
No retry, backoff or 429 handling exists here. The only mitigation, per-repo serialization at .github/workflows/test-containers-lectures.yml:46-48 (v0.7.0, re-scoped by run id in v0.9.0, #103), pairs the two containers for one repo; the three repos at :55-60 still build in parallel. Flakes also file tracker noise: on any non-workflow_dispatch failure (:138), notify-failure (:132-154) runs scripts/create-ci-failure-issue.sh → "🔴 CI failure: Test Container Builds".
Fix: retry the HTML build stage. With execute_notebooks: "cache" (lectures/_config.yml:21) and lecture-repo/_build/.jupyter_cache persisting across steps in the job — why stages 2 and 3 already skip execution — a retry re-executes only the failed notebooks, not the full 30-60 minute run, so no need to gate on the 429 string.
- Give the "Build HTML (Execute Notebooks)" step (
:95-102) anidandcontinue-on-error: true, then a conditional second attempt onsteps.<id>.outcome == 'failure' - Give the retry a distinct
failure-artifact-name—build-lecturesuploads execution reports under that name on failure (build-lectures/action.yml:38,179), so a repeated name collides with the first attempt's artifact - Validate with a maintainer
workflow_dispatchafter merge — it triggers only onworkflow_runandworkflow_dispatch(:22-27), so a PR cannot exercise it
Acceptance: a single transient 429 no longer fails the matrix; a persistent failure still does.
While in this workflow: restore the missing matrix repo
lecture-python-programming was dropped from the matrix (:54-64) in an unannounced swap in 81e96ff — unlike lecture-jax, which is commented out at :61-64 with a TODO and an issue link. containers/VALIDATION.md:25,42,46,50 and TESTING.md:11 still advertise the 4-repo matrix, so the only place lecture content executes on the images silently lost a quarter of its coverage while the docs kept claiming it.
It belongs in this PR rather than its own: it is four lines in the file you are already editing, and the repo has six build-time raw-GitHub fetches plus yfinance calls, so the retry above is the prerequisite for adding it without immediately reintroducing the flake. Note it is also the repo pinning anaconda=2026.07 against the containers' 2026.06.
- Add the
lecture-python-programmingmatrix row (current repo name, no.mystsuffix) - Correct the 4-repo claims in
containers/VALIDATION.mdandTESTING.mdto match
Activity
mmcky commentedon Jul 22, 2026
Transferring this issue to
QuantEcon/actions— on reflection this is only relevant to the container-validation workflow, where the test matrix makes multiple calls to GitHub from shared runner IPs while building several repos and hits raw-GitHub rate limits. The right fix is on the CI/container side (e.g. retries or caching), not in the lectures themselves.We won't be changing the data-fetch approach in the lectures: switching to local
_static/paths would break the Colab integration, since the published notebooks (lecture-python.notebooks) are a flat directory of.ipynbfiles with no_static/folder — every Colab user would hit aFileNotFoundError. That's why the lectures read vendored data over raw GitHub URLs (see also QuantEcon/lecture-python.myst#972, closed for the same reason). Longer term, a shareddata-lecturesrepository will give us a cleaner home for these datasets.[-]hansen_singleton_1983: build-time `pd.read_csv` of a vendored CSV over a raw-GitHub URL flakes on HTTP 429[/-][+]test-containers-lectures: absorb transient raw-GitHub 429s when building lecture repos[/+]docs(plan): record #203 as #102's fix
docs(plan): cite the lecture-side fix, not the CI alert, in the #102 row
docs(plan): record #102 as closed by #203, and the run that exercised…
Merge main into the #190 branch: keep #214's PLAN entry
docs(plan): #188 is closed; #282 and #291 join the open issues (#292)