COLDBOX-1457 Browser testing for ColdBox apps: @browser on BaseTestCase and named route helpers

[lmajano]

Description

Jira Issues

COLDBOX-1457. It needs TestBox's annotation-driven browser support, Ortus-Solutions/TestBox#222, which builds on TESTBOX-470 and Ortus-Solutions/TestBox#212.

This PR adds browser testing for ColdBox applications. It builds on TestBox browser support and the bx-playwright BoxLang module.

No BrowserTestCase: annotate BaseTestCase instead (BoxLang)

TestBox now attaches browser support to any spec whose class (or a parent class) has @browser, @browserProfile or @baseURL (Ortus-Solutions/TestBox#222). So a ColdBox browser test is just a BaseTestCase:

@appMapping( "/root" )
@browser
@baseURL( "http://127.0.0.1:8080" )
class extends="coldbox.system.testing.BaseTestCase" {
    function run() {
        it( "shows a user", () => {
            browse( ( page ) => {
                visitRoute( page, "users.show", { id : 5 } )
                assertRouteIs( page, "users.show" )
                expect( page ).toSee( "User 5" )
            } )
        } )
    }
}

• From TestBox, through the annotation:
  • browse(), this.playwright(), browserAvailable(), ensureBrowserInstalled();
  • the browser matchers;
  • screenshots, traces and videos attached on failure;
  • the bundle browser, closed by the runner after the bundle.
• BaseTestCase adds the named route helpers. They work on every engine, and only need the controller.
  • routeURL( name, params ) returns the path and query, built by ColdBox's own event.route().
  • visitRoute( page, name, params ) visits a named route.
  • assertRouteIs( page, name, params ) waits until the page is on that route. Without params it matches the route pattern, including optional placeholders. With params it matches the exact path.
• Logged-in tests use bx-playwright saved sessions, with no test-only login endpoints in ColdBox:

  this.playwright().session( "admin", ( page ) => visitRoute( page, "login" ).fill( "Email", "admin@site.com" ).fill( "Password", "secret" ).click( "Sign in" ) )
  browse( ( page ) => visitRoute( page, "admin.dashboard" ), { session : "admin" } )

• BaseModelTest, BaseInterceptorTest and any other spec can browse the same way.

Fixes

• event.route( "name@module" ) built links with no slash after the module entry point (bloglogin/3/). A unit test in RequestContextTest covers it.
• On Adobe, RequestContext.getMemento() included a this key, so RequestContextDecorator methods ran against the original context and missed its mocks. Regression specs in RequestContextDecoratorTest cover it.

Harness and tests

• Test-harness routes:
  • users.show;
  • browserPosts, with an optional placeholder;
  • a plain login form (browserTesting.login) and browserTesting.whoami, for the saved-session specs.
• tests/runner.cfm skips the BoxLang-only /browser folder on other engines.
• BrowserTestingSpec (a BaseTestCase with @browser) covers:
  • route paths, for app and module routes;
  • assertRouteIs(), with and without optional placeholders;
  • a saved session created through the login form;
  • per-page session isolation.
• The browser specs skip on TestBox releases without the annotation support. The route helper specs still run.

Verified locally, BoxLang 1.17.6, bx-playwright development + Chromium

• tests.specs.browser: 15/15 pass with TestBox#222.
• The same spec on the previous TestBox: 11 pass, 4 skip, 0 errors.
• tests.specs.integration: 97/97 pass with the new BaseTestCase.

Type of change

• Bug Fix
• New Feature
• This change requires a documentation update (ColdBox 8.3.0 release docs: What's New, browser testing guide, routing and WireBox updates ortus-docs/coldbox-docs#301, updated for @browser)

Checklist

• My code follows the style guidelines of this project
• I have commented my code (every new method has a docblock)
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works

🤖 Generated with Claude Code

https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

[github-actions]

Test Results

0 tests  ±0   0 ✅ ±0   0s ⏱️ ±0s
0 suites ±0   0 💤 ±0 
0 files   ±0   0 ❌ ±0 

Results for commit 9870dff. ± Comparison against base commit ccc9834.

♻️ This comment has been updated with latest results.

[lmajano]

CI status on 32e827b: the remaining red jobs are not caused by this PR.

• tests / Test Suites (adobe@2023, 11, true, 8)
  • Failure: one error in tests.specs.web.services.ModuleServiceTest > "Still resolves a handler-only URL for a module with only a mandatory-action route": "module requested 'test-module-conventions' is not loaded" (Router.cfc:323).
  • Origin: that spec and its test module came from COLDBOX-1454: fix interceptor buffer pool race between a request and its async announce thread #705, already on development. The same error appeared on COLDBOX-1454: fix interceptor buffer pool race between a request and its async announce thread #705's own CI (run 36101802801, adobe@2025).
  • Evidence it is intermittent: it passes on development and on adobe@2025 in this run, and passed 4 out of 4 local adobe@2023 runs.
  • This PR touches neither the spec, ModuleService nor Router. No fix exists yet; it belongs with COLDBOX-1454: fix interceptor buffer pool race between a request and its async announce thread #705.
• adobe@2025 and boxlang-cfml@be jobs: CommandBox setup failed with ForgeBox "408 Request Time-out" before any test ran.

These need a re-run of the failed jobs (runs 36863474898 and 36863469751). I don't have permission to re-run them (403), so a maintainer needs to. I'll keep watching the PR.

---

Generated by Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

Route helpers incorrectly skip without browser support and fail to match valid optional route variants.

Review effort: Balanced
Findings: 2

Open (2)

• Remove browser-support guard from visitRoute tests · New
• Match all same-named route variants in assertRouteIs · New

What changed in this PR

Adds BoxLang browser testing support to ColdBox and fixes request-context routing/decorator regressions.

Changes:

• Introduces BrowserTestCase with browser and named-route helpers.
• Adds browser harness routes, fixtures, and integration coverage.
• Fixes module route joining and decorator memento handling.

File	Description
AGENTS.md	Documents BoxLang annotation and optional-value conventions.
changelog.md	Records the feature and fixes.
system/​testing/​BrowserTestCase.bx	Implements browser and route-testing helpers.
system/​web/​context/​RequestContext.cfc	Fixes module links and memento copying.
test-harness/​config/​Router.cfc	Registers browser-test routes.
test-harness/​handlers/​browserTesting.cfc	Provides browser-test pages and login flow.
tests/​resources/​browser/​FakePage.bx	Adds a browser-independent page test double.
tests/​runner.cfm	Excludes BoxLang browser specs on other engines.
tests/​specs/​browser/​BrowserTestCaseSpec.bx	Covers browser and route helpers.
tests/​specs/​web/​context/​RequestContextDecoratorTest.cfc	Tests decorator scope isolation.
tests/​specs/​web/​context/​RequestContextTest.cfc	Tests module route slash handling.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[lmajano]

tests / Test Suites (boxlang-cfml@be, 21, true, 8) failed before any test ran, and the cause is not in this PR. The boxlang-cfml@be server failed to start because CommandBox could not resolve the engine:

No satisfying version found for [be]. ForgeBox can't find the package and you don't have a usable version in your local artifacts cache.

This is a ForgeBox/engine availability problem; the latest push (7146283) only changes a docblock. Its "TestBox Report boxlang-cfml@be-21" check passed only because no results were produced.

I could not re-run the job, because this integration has no permission to re-run workflows here (403). A maintainer can use "Re-run failed jobs" on run 37844858097 once the be engine resolves again.

The Adobe 2023 and 2025 jobs fail on one spec, ModuleServiceTest > Module Lifecycle ("test-module-conventions is not loaded"). This PR doesn't touch it, and it already fails the same way after the development merge.

---

Generated by Claude Code

[lmajano]

Shouldn't this extend from the TestBox BrowserSpec instead? I know we need the testbox stuff, but we need all the machinery in the new TestBox BrowserSpec?

[lmajano]

It can't extend both. BaseTestCase extends testbox.system.compat.framework.TestCase and BrowserSpec extends BaseSpec, so it's single inheritance either way.

• Extend BaseTestCase (current): we keep the ColdBox app loading that routeURL(), visitRoute() and assertRouteIs() rely on, plus execute(), getRequestContext() and the mapping annotations.
• Extend BrowserSpec: we'd have to copy all of that ColdBox loading code instead.

It already gets all of BrowserSpec's machinery. BrowserSpec is only the matcher registration plus five one-line methods that delegate to testbox.system.browser.BrowserSupport:

Method	What it does	BrowserTestCase
browse()	delegates to BrowserSupport	same delegation
playwright()	delegates to BrowserSupport	same delegation
browserAvailable()	delegates to BrowserSupport	same delegation
getBrowserSupport()	reads browserProfile and baseURL	same
closeBrowser()	marked @afterAll	same

So it shares the bundle browser, the run-ledger cleanup, the failure attachments and the inline report screenshots. It adds the route helpers, plus a guard that skips browser specs on TestBox releases older than 7.2, which have no browser support.

What's duplicated is about 60 lines of thin delegation. To remove it, TestBox could expose the browser methods as a mixin that any spec class can pull in, and both BrowserSpec and BrowserTestCase would use it. That means a TestBox change first and then a ColdBox follow-up, so I've left it out of this PR.

---

Generated by Claude Code

[lmajano]

CI status on 9870dff:

• Adobe compile error fixed. Earlier commits failed on Adobe 2023/2025 because ( route.regexPattern ?: "" ).reReplace(...) does not compile there. 9870dff uses reReplace() function calls. Adobe 2025 now runs 907 tests with 904 passing, and the integration specs pass locally on Adobe 2025 (97/97).
• Remaining Adobe 2025 failure is not from this PR. It is ModuleServiceTest > Module Lifecycle ("test-module-conventions is not loaded"). This PR does not touch module loading, and the same check is intermittent on development. The re-run could not be triggered from here (no permission to re-run workflows), so a maintainer re-run of that job should clear it.
• Format check: the route helper signatures are now formatted.
• Earlier BoxLang failures were infra. ForgeBox returned a 408 for cbstreams during box install, before any test ran.

---

Generated by Claude Code