fix(runner): preview telemetry accuracy for Tier 1 and Tier 2 (DEV-3145) - #395
Merged
Merged
Conversation
- Tier 2 posts the reporter budget reset on every flush and re-arms the stderr relay budget - sandpack.compile_error gets its own per-page ceiling instead of sharing the runtime one - sandpack.compile_error carries a bucket attribute (registry, contract, emit, ingest test) - Tier 1 exposes a settled stale-preview state (data-preview-stale, status bar note) after a syntax error - regression test for a superseded bundler error between pushes - e2e stubShell and its inline copies also abort the versioned sandpack host Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- an older push's start no longer clears a newer transpile failure's stale state - compile_error reads the bucket when the error arrives, not when the collapse emits - share the compile key prefix between the collapse and sentry.ts - show the stale note only while the preview is ready Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Follow-ups from #371 on how the editor's live preview reports compile and runtime errors (ClickUp DEV-3145). After the Sentry trim,
sandpack.compile_errorandpreview.runtime_errorare the only record of handled preview errors, so their accuracy matters. This PR covers all six items on the ticket:ContainerRuntime.flush()now posts the same reset (sharedpostMonitorResethelper inmonitor.ts) before writing files, and re-arms the stderr relay budget.stderrSeenis not cleared, so old status-log lines are not re-relayed.demoEventCollapsenow countscompile:keys against their ownDEMO_COMPILE_CEILING, so neither kind can starve the other.bucketonsandpack.compile_error(fixed). Added to the registry row and contract §5, and emitted from a getter that is read when the error arrives (the collapse may emit up to 2 s later, after a bucket switch). Ingest already mappedbucket, covered by a new Faro normalise test. ADR-0041 still opens with "whether the last release raised compile errors on thenextbucket" and no doc or commit shows the omission was deliberate. Thenextquestion itself is unchanged: alerts and dashboards still group byht_majoronly.SandpackRuntimetracks whether the newest push failed to transpile and exposesonStaleChange. Once the edit burst settles and the preview isready, the pane getsdata-preview-stale="true"and the status bar reads "Showing last successful run". No new Sentry event and no error card. An older push's bundlerstartdoes not clear a newer failure.stubShell(fixed). NewabortSandpackHostsine2e/helpers.tsmatches the versioned*-sandpack.codesandbox.iohost.stubShelland the 20 inline or local copies of the two-glob abort across 18 specs now use it. TheE2E_LIVE-gated specs and the ready-state cases that rely on the live bundler are untouched.Types of changes
runner/) changeHow was this verified?
pnpm typecheckclean andpnpm testinrunner/: 2567 tests, 2567 pass, 0 fail (re-run by me, not only by the implementing agents).bucketblob, Faro ingest.App.tsx/EditorShellwiring ofdata-preview-staleand the status bar note.Checklist
runner/config/frameworks.json(see CONTRIBUTING.md); otherwise it won't appear on demos.handsontable.compnpm build(andpnpm dev) in the affected example/server-example locallyThe checklist items are for new or renamed examples; this PR changes only
runner/.Related issue(s):
Note
Medium Risk
Touches live-preview error reporting, Sentry/metric collapse, and Tier-2 monitor resets; behavior is heavily tested but affects observability and what users see during broken edits.
Overview
Improves preview error telemetry and UX after compile/runtime reporting became the main signal for handled preview faults.
Tier 1 (Sandpack): When the latest edit fails to transpile but the iframe still shows the last good run, the runtime exposes
onStaleChange; the authoring app debounces that with the same edit-settle window as other burst logic, then setsdata-preview-staleon the preview pane and “Showing last successful run” on the status bar (only while status isready). Stale clears on a successful rerun start or an unchanged push, without treating an older bundlerstartas fixing a newer failure.Telemetry:
sandpack.compile_errornow includes an optionalbucket(read at error time so collapsed emits stay correct). Compile vs runtime collapse uses a separate ceiling forcompile:keys so one stream cannot exhaust the other’s budget.Tier 2 (container): On each post-ready edit flush, the runtime re-arms the in-preview monitor (shared
postMonitorReset) before writing files and resets the stderr relay budget without re-relaying already-seen log lines—matching Tier 1’s per-run reset under Vite HMR.E2E:
abortSandpackHostscentralizes aborting versioned*-sandpack.codesandbox.iohosts so deterministic specs do not accidentally hit the live bundler.Docs/registry/tests updated for the new metric dimension and behaviors.
Reviewed by Cursor Bugbot for commit 5e74d28. Bugbot is set up for automated code reviews on this repo. Configure here.