You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Speed up the coverage job (about 25 minutes since coverage became real) #898
Since #897 made coverage real, the coverage job in ci.yml takes about 25 minutes (1,503 s on the run for main), where the other jobs take 10 to 15. It runs on every pull request and every push to main, since the job has no if:. It is not a required check, so it never blocks a merge, but it is the last job to finish on every run, and the one that tells the Codecov number.
Why it is slow
It runs the whole suite with coverage tracing, including the tests marked slow (the analytic IK tests that compile generated C and C++ code, about 110 s untraced), and it runs them under the default tracing core.
Ideas, cheapest first
COVERAGE_CORE=sysmon in the coverage step. Python 3.12 has a lighter tracing core. Try it on the job, and compare the totals with the current 85.98%, since a different core can change which lines count.
Do not trace the slow tests: run them in the matrix job on Linux with no coverage, and take coverage from the fast tests only. Check what the analytic IK tests contribute (they exercise about 4,000 lines).
Split the job with pytest-xdist and combine the data (coverage combine), as the tests are independent.
Run coverage on pushes to main and weekly, not on every PR. The PR would lose its Codecov patch status, which is informational now anyway.
Cache what is rebuilt on every run: the package install, robot_descriptions assets (already cached).
Measure first
The job log has the timing for each step. Note the compile-based tests in particular: on the Windows runner they were slow enough to hit the 50 s limit, so a Linux job that traces them may be paying the same cost.
I measured the first idea on GitHub's ubuntu-latest runners with Python 3.12, running the full suite (2,402 tests) twice per variant.
COVERAGE_CORE=sysmon counts exactly the same lines as the current setup (13,958 of 16,208, 86.118%) and cut the test time from 721 s to 521 s on one run and from 1,486 s to 483 s on another. The runners vary by about 2x between runs, so the saving was between roughly 30% and 65%.
One thing that may be useful: the slow tests add only 26 uniquely covered lines (0.16 points of coverage), far fewer than the estimate in the issue. They run nowhere else in CI (the matrix jobs use -m "not slow"), so I have left them alone.
Activity
ddivyanshsharmaa commentedon Nov 1, 2026
I measured the first idea on GitHub's
ubuntu-latestrunners with Python 3.12, running the full suite (2,402 tests) twice per variant.COVERAGE_CORE=sysmoncounts exactly the same lines as the current setup (13,958 of 16,208, 86.118%) and cut the test time from 721 s to 521 s on one run and from 1,486 s to 483 s on another. The runners vary by about 2x between runs, so the saving was between roughly 30% and 65%.One thing that may be useful: the
slowtests add only 26 uniquely covered lines (0.16 points of coverage), far fewer than the estimate in the issue. They run nowhere else in CI (the matrix jobs use-m "not slow"), so I have left them alone.I will open a one-line PR with the sysmon change.