Repository navigation
Open
Feature
0 / 10 of 1 issue completed
Copy link
Labels
area: benchAffected areaAffected areaepicA large body of work that can be broken down into smaller tasks or stories.A large body of work that can be broken down into smaller tasks or stories.
Milestone
Description
Epic Overview 🎯
bench-action's PR comment doesn't scale past a couple of bench files. soroush-tech/umber-style#57 shows the problem with 7 files: 7 headings, 7 tables and 14 rows. The number the gate actually checks, speed vs baseline, never appears. The table shows "vs fastest", but which row is fastest changes from file to file, so you can't scan the comment for a regression. Case labels also repeat the file name and the case keys (core: merging five classes :: previous).
This epic replaces it with a single summary table, one row for each non-baseline case compared against the baseline. The per-file detail is folded below it. It also makes the comment configurable, so more than one bench job can post on the same PR.
Implementation Details 🛠️
- Data source (
@soroush.tech/bench2.3.0): a new--json-filewrites each case's raw numbers (ns, bytes) and itsratioPctagainst the baseline, in the same container-mount pattern as--md-file. The action reads this instead of parsing the formatted markdown, which has lost precision. - Summary layout (bench-action 1.2.0):
There is one row per non-baseline case, so a file comparing N packages gives N − 1 rows under the same benchmark name. Failed rows come first, then the rest from slowest to fastest.
## ✅ Benchmarks — 7 of 7 passed Baseline `previous` · each case must reach 80% of baseline speed · 1 round | | benchmark | case | time | baseline | speed | alloc | | :-: | ---------------------------- | ----- | --------: | --------: | ----: | -----: | | ✅ | a theme and its variables | local | 1.35 ms | 1.21 ms | 90% | +29.5% | | ✅ | extracting the CSS of a page | local | 123.31 µs | 139.50 µs | 113% | 0.0% | <details><summary>Full results (7 files)</summary> …per-file tables… </details> - New inputs:
comment-details(collapsedby default, oropenornone) controls the folded per-file tables.comment-titlesets the heading and is also part of the hidden sticky-comment marker, so two bench jobs on one PR keep separate comments. Leaving both inputs out gives the new layout with today's single sticky comment. Both the branded relay path ([Epic] Bench comment-relay worker at api.bench.soroush.tech #309) and the token path render the same body. - Rollout: release bench 2.3.0 → re-vendor it in bench-action → release bench-action 1.2.0 → bump umber-style to check the comment on a real PR.
Key Features ✨
| Feature | Description |
|---|---|
| JSON results | soroush-bench --json-file writes unformatted per-case numbers plus ratioPct against the baseline, for any number of cases. |
| Summary table | One table across all files and one row per non-baseline case against the baseline, showing the gated speed %. Failures come first. |
| Folded detail | Today's per-file tables move into a <details> block. comment-details controls it: collapsed, open or none. |
| Comment title | comment-title sets the heading and the sticky marker, so several bench jobs can comment on one PR without overwriting each other. |
Acceptance Criteria ✅
- On feat: 0.2.0 - the is-prop-valid package, siblings as ranges, cheaper types, and two copies of the engine that share their state - #56 umber-style#57's 7 files, the comment shows one summary table with 7 rows, plus a collapsed details block
- A bench file with 3 or more cases gives one summary row for each non-baseline case, all compared against the same baseline
- A breached case appears first with ❌, and its speed % matches the gate's failure message
- Two jobs with different
comment-titles on the same PR keep two separate sticky comments, and re-running either one updates its comment in place - bench 2.3.0 and bench-action 1.2.0 are released with release notes; both packages stay at 100% coverage
Dependencies
Metadata
Metadata
Assignees
Labels
area: benchAffected areaAffected areaepicA large body of work that can be broken down into smaller tasks or stories.A large body of work that can be broken down into smaller tasks or stories.
Activity