Skip to content

Retain benchmark reports in workspace CI - #49

Merged
snkmcb merged 3 commits into
mainfrom
codex/benchmark-ci-reports
Nov 13, 2026
Merged

snkmcb merged 3 commits into
mainfrom
codex/benchmark-ci-reports

Conversation

@snkmcb

@snkmcb snkmcb commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Include the per-asset benchmark JSON in each workspace CI job's report-<cell> artifact. The workflow generator keeps the upload path in place when the OpenStrata matrix is regenerated.

Record the Windows measurements for all nine scale assets and nine real assets, including the full CC0 Leica Sofort SOG conversion. At 7,995,392 Gaussians, PLY, SPZ, and SOG each peaked at about 3.26 GiB.

Set the corpus limits to 100 ms for CanRead(), 150 ms for metadata-only reads, and 0.60 GiB plus 0.50 GiB per million Gaussians for peak resident memory.

Why?

The first hosted benchmark run passed, but its JSON reports stayed in the build tree. Saving those reports gives the v0.6.0 roadmap a cross-platform baseline for the resource limits.

Validation

  • Windows Release: all 18 CTest cases passed; all nine scale and nine real benchmark rows passed.
  • Workflow regeneration check, corpus manifest verification, and ost ci validate passed.
  • Source CI run 75 saved all 36 rows across Windows, macOS, and Linux.
  • Source CI run 76 measured a 0.985 GiB macOS peak for 1M PLY. The revised memory formula covers both observed macOS peaks and tightens the original limits at 4M and the import ceiling.
  • All 72 saved hosted rows and 18 saved local rows pass when evaluated against the revised limits. Both benchmark CTest cases passed on a fresh local Windows run.
  • Source CI run 77 passed all 12 jobs on the revised commit. All three uploaded JSON reports record the revised limits and contain 12 passing rows each.

Checklist

  • Linked the roadmap and explained the change.
  • Ran the relevant local checks and verified the hosted reports.
  • Updated the benchmark reference and build instructions.
  • Added no private or unlicensed assets.

Reviewer notes

The hosted jobs run the fixture, corpus, and 1M rows. The 4M, ceiling, and real rows were measured locally on Windows.

@snkmcb
snkmcb merged commit f78a991 into main Nov 13, 2026
13 checks passed
@snkmcb
snkmcb deleted the codex/benchmark-ci-reports branch November 13, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant