feat(up): interactive up runs on a single plan by ndeloof · Pull Request #14305 · docker/compose

ndeloof · 2026-10-08T11:19:21Z

Stacked on #14302, #14300 and #14304 (not yet merged); the fix(pre_start) fix this switchover needs to show hook output has landed as #14353. Of the commits below, the switchover itself is feat(up): interactive up runs on a single plan; the two test(e2e) commits after it lock what it brings.

What this PR does, in one sentence

An interactive compose up now runs on the same single Create+Start plan as up -d, executing its Create phase, setting up the log session, then executing its Start phase, instead of chaining create() and start() over two daemon snapshots.

Context

Foreground up is the last consumer of the old create-then-start sequence (#14081): it creates the project from one snapshot, sets up attach, the log printer and the monitor, then calls start(), which re-observes the daemon and rebuilds its project from what the caller passed rather than from what the Create phase actually ran on. Everything else (up -d, scale, watch rebuilds, compose start) already goes through the plan engine.

What the PR brings

The plan is prepared once, with both phases. The Create phase runs under the up progress scope, as before. The session is then set up unchanged (signals, keyboard menu, watcher, monitor, attach), and the Start phase is dispatched last, outside the progress scope, on a context that survives Ctrl+C, with the printer as the hook-log listener, which start() no longer takes since nothing passes one. The guardrails of the old sequence are kept: the Start phase stays out of the progress scope (its refresh would repaint over the logs), signal.Notify still comes after the Create phase, a failed Create phase reaches neither the session nor the Start phase, a dry run only ever executes the Create phase (message unchanged), and compose start / compose run are untouched.

The monitor replays events from the daemon's time, which an interactive up reads right before the Start phase: a daemon whose time can't be read fails the command with an explicit error, with no container started.

One deliberate behavior change: the session, the watcher and the teardown now use the plan's canonical project instead of the caller's. That project is the one carrying what preparation resolved, in particular the #apisocket config of use_api_socket. The effect is a fix: a foreground up now gives a use_api_socket service its /run/secrets/docker/config.json, as up -d does since the detached switchover. The defect itself touches main in both modes (the file is absent in up foreground and in up -d there); this PR fixes the foreground half, and a scenario locks it (the file is injected whether or not credentials are configured).

A foreground up now shows the output of pre_start hooks, as it does post_start's. It never did, and not because of this series: the hook's log stream was opened while its container was still in created state, which the engine ends at once, empty. That also left the error of a failing hook without its output tail, in detached mode too. The stream is now opened once the hook is started; it landed on its own as #14353. A scenario asserts the output in a foreground up.

The monitor never sees a pre_start runner, whose exit would otherwise end the application under --abort-on-container-exit or win over the service's code under --exit-code-from: the runner lacks the one-off label the event filter requires. That protection was implicit, so a scenario locks it.

Known consequences, accepted: the snapshot is not refreshed between the Create and Start phases; and a library caller combining Attach with Wait no longer gets the post-start --wait verification (the CLI already makes --wait imply detached mode). This gap will be closed by validating such combinations at the API level, in a separate PR.

The seven characterization scenarios of #14300 pass unchanged against the new path, and unit tests pin the phase order against a mocked engine: create under the progress scope, then attach, then the first start; a container exiting before the event subscription is still seen, the replay starting at the daemon's time (set here an hour off the client's); a daemon whose time can't be read fails the up before any container starts; a failed create stops there; a dry run only creates.

Why this is the right next brick

It is the switchover the whole series builds toward, kept to one commit so it reverts trivially: the characterization scenarios (#14300), the phase handle (#14302) and the monitor replay (#14304) are all in place, and what is left of the old start path (the waitDependency helpers' placement and stale comments) can be cleaned up next: startService itself went with #14296.

Part of #14081.

🤖 Generated with Claude Code

Copilot

🟡 Changes recommended

The new monitor timestamp parser passes invalid arguments to assert.Check.

1 open finding

Pass boolean conditions to assert.Check · New

What changed in this PR

Moves foreground compose up onto the shared Create+Start plan while preserving attachment and monitoring behavior.

Changes:

Splits plan execution into independently driven Create and Start phases.
Migrates start, scale, and watch rebuild paths to the plan engine.
Adds monitor replay handling and extensive unit/E2E coverage.

File	Description
`pkg/e2e/up_interactive_test.go`	Adds foreground-up scenarios.
`pkg/e2e/testdata/TestUpInteractiveWaitsForHealthy/compose.yaml`	Health dependency fixture.
`pkg/e2e/testdata/TestUpInteractiveRestartsExitedContainer/compose.yaml`	Exited-container fixture.
`pkg/e2e/testdata/TestUpInteractiveRecreateStreamsNewContainer/compose.yaml`	Recreation fixture.
`pkg/e2e/testdata/TestUpInteractiveInjectsAPISocketConfig/compose.yaml`	API-socket fixture.
`pkg/e2e/testdata/TestUpInteractiveImmediateExit/compose.yaml`	Immediate-exit fixture.
`pkg/e2e/testdata/TestUpInteractiveHookLogs/compose.yaml`	Lifecycle-hook fixture.
`pkg/e2e/testdata/TestUpInteractiveDependencyTimeout/compose.yaml`	Dependency-timeout fixture.
`pkg/e2e/testdata/TestUpDryRunCreatesNothing/compose.yaml`	Dry-run fixture.
`pkg/compose/watch.go`	Uses one plan for rebuilds.
`pkg/compose/up.go`	Drives interactive plan phases around session setup.
`pkg/compose/up_phases_test.go`	Tests interactive phase ordering and failures.
`pkg/compose/start.go`	Migrates start to Start-only plans.
`pkg/compose/start_test.go`	Removes superseded imperative tests.
`pkg/compose/service_containers.go`	Refines dependency timeout handling.
`pkg/compose/service_containers_test.go`	Tests optional dependency deadlines.
`pkg/compose/scale.go`	Uses combined Create+Start planning.
`pkg/compose/restart_test.go`	Updates shared test helpers.
`pkg/compose/reconcile.go`	Expands start planning and provider handling.
`pkg/compose/reconcile_start_test.go`	Covers start-plan edge cases.
`pkg/compose/observed_state.go`	Makes observation phase-aware.
`pkg/compose/observed_state_test.go`	Tests Start-only observation behavior.
`pkg/compose/monitor.go`	Replays events across monitor startup.
`pkg/compose/monitor_test.go`	Tests replay and subscription timing.
`pkg/compose/executor.go`	Adds phased plan execution and wait bounds.
`pkg/compose/executor_start_test.go`	Tests phased execution and timeouts.
`pkg/compose/executor_ops.go`	Updates wait and rename operations.
`pkg/compose/down_test.go`	Updates shared container fixture usage.
`pkg/compose/create.go`	Extracts plan preparation and combined execution.
`pkg/compose/containers.go`	Removes obsolete filtering helper.

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

ndeloof · 2026-10-08T11:26:55Z

+	assert.Check(t, err, "since %q", since)
+	var n int64
+	if nsec != "" {
+		n, err = strconv.ParseInt(nsec, 10, 64)
+		assert.Check(t, err, "since %q", since)
+	}


Same finding as on #14304 (the monitor commit is shared), and I can't reproduce it: assert.Check (gotest.tools v3.5.2, internal/assert/assert.go) accepts nil (success) and error (failure, logged, non-fatal) as a comparison. A nil err is therefore a valid assertion, and a non-nil err produces a readable failure. Check is intentional (called from the monitor's goroutine). Code unchanged; thread left open for the maintainer.

codecov · 2026-10-08T11:31:41Z

Codecov Report

❌ Patch coverage is 93.05556% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/compose/executor.go	87.50%	2 Missing and 1 partial ⚠️
pkg/compose/up.go	92.30%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

docker-agent

Assessment: 🟢 APPROVE

Lower-confidence findings (not posted inline)

[low] pkg/compose/executor.go:474 — execBoundedWaitCondition compound error may leak "context deadline exceeded" text if translateWaitTimeout is not called (confidence: 🟠 weak 52/100)

fmt.Errorf("%w: %w", errWaitTimeout, err) produces a compound error that reads "timeout waiting for dependencies: context deadline exceeded". All current callers of run.runStart, upDetached, and start() correctly call translateWaitTimeout, which strips the chain. The residual risk is a future or library caller that calls into the plan executor without wrapping the result — they would see the raw compound message. Not an active bug, but a latent fragility.

Copilot

🟡 Changes recommended

Start can outrun monitor initialization, allowing an immediate container exit to be missed and foreground up to hang.

2 open findings

Capture monitor replay timestamp before starting container · New
Pass boolean conditions to assert.Check

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

docker-agent

Assessment: 🟢 APPROVE

docker-agent · 2026-10-08T12:09:40Z

+			// inherited deadline is a different, real failure the caller
+			// must still see.
 			if errors.Is(ctx.Err(), context.DeadlineExceeded) {
+				if !config.Required && origCtx.Err() == nil {


[low] Concurrent origCtx cancellation while the wait's own deadline fires can mask the cancellation as a timeout error

In waitDependency, the origCtx.Err() == nil guard (line 231) is used to distinguish "this call's own WithTimeout deadline fired" from "a deadline inherited from an ancestor context fired." The code comment covers the inherited-deadline case well.

There is a narrow but real race for optional dependencies: if origCtx is concurrently cancelled (i.e., origCtx.Err() becomes context.Canceled) at the same moment that ctx's own deadline fires, ctx.Done() closes first, ctx.Err() is context.DeadlineExceeded, and origCtx.Err() is now non-nil (context.Canceled). The guard fails, the function returns ctx.Err() (DeadlineExceeded), and the caller in waitDependencies translates it to:

if errors.Is(err, context.DeadlineExceeded) { return errors.New("timeout waiting for dependencies") }

The user sees "timeout waiting for dependencies" instead of a clean Ctrl-C exit.

The fix is to also allow origCtx having been cancelled (not deadline-expired) to keep the optional-dep skip, since a plain cancellation of the parent should be treated like the optional dependency's own timeout for an optional service:

Suggested change

if !config.Required && origCtx.Err() == nil {

if errors.Is(ctx.Err(), context.DeadlineExceeded) {

if !config.Required && (origCtx.Err() == nil || errors.Is(origCtx.Err(), context.Canceled)) {

This preserves the existing behaviour for all required dependencies and for the inherited-deadline case (where origCtx.Err() is context.DeadlineExceeded), while correctly treating a concurrent Ctrl-C cancellation of origCtx as a benign exit for optional dependencies.

Confidence Score

🟡 moderate 62/100

Copilot

🔵 Needs a closer look

The broad lifecycle, concurrency, event-replay, and dependency-ordering changes warrant final human validation.

1 open finding

Pass boolean conditions to assert.Check

1 resolved since last review

Capture monitor replay timestamp before starting container

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

docker-agent

Assessment: 🟢 APPROVE

…self run() walked both phases in one call, keeping the state they share local and unreachable. Split it into begin, which prepares the run without dispatching anything, and a planRun whose runCreate and runStart each dispatch their own phase, so a caller can act between the two -- the interactive up will set up its attach/printer/monitor session there. run() is now begin followed by both phases, so every existing caller behaves as before. The start phase takes its hook-log listener when it is entered, since an attached caller only has one once its session exists, and refuses to run before a successful create phase or twice, keeping the create-phase barrier intact for callers that drive the phases themselves. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Most e2e coverage of the start phase runs detached; the foreground `up`, whose create and start phases are driven by separate code paths, had none for hooks, health-gated dependencies, a container that exits as soon as it starts, restarts of exited containers, recreation, dependency timeouts or dry-run. Add scenarios that fix these observables before the machinery behind it changes. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

…reams The hook scenario matched "pre-start-hook-ran" in the output, but the service's own line "app-read:pre-start-hook-ran" contains it, so the assertion passed without any pre_start output reaching the session. A foreground up streams post_start's output but not pre_start's today (checked on main too). Lock what actually holds: pre_start's effect, read by the service through a value distinct from anything it logs. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Wait for the long-running service's final state after an abort instead of reading it the instant the command returns, document why the dependent is expected in the created state on a dependency timeout, and narrow the dry-run "no start" check to the service's own lines. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

monitor.Start lists the project's containers, then opens the event stream. Both are round trips: a container that starts and terminates in between has its die emitted to nobody, the monitor tracks it forever, and --abort-on-container-exit / --exit-code-from never fire. The window is narrow, but a plan running Create and Start back to back makes the first start follow the monitor's attachment closely. Ask the engine for events since a point taken before the listing, and take that point from the daemon (its system time, one extra round trip), not from the client: events are stamped by the daemon, so a client clock that differs from it, as against a remote daemon, would either miss part of the window or replay events from before the monitor was started. There is no falling back on the client's clock: a daemon whose time can't be read or parsed makes Start fail with an explicit error, and so the command. A caller that wants to know before it starts any container can read the time ahead of Start (readDaemonTime), which then doesn't ask again. Replayed events already reflected in the listing are harmless: the listing notifies nothing, so each is reported once, and the tracked set only gets an ID added or removed again. Unit tests drive the monitor against a fake event log that, like the engine, only delivers unsubscribed events to a subscription asking for them with since, and stamps events with a clock of its own. They pin that replay for a die, a start, a destroy and a restart policy, the daemon's time being used whatever its offset from the client's, the failure when it can't be read, and that `logs --follow` returns that failure. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Interactive up used to create() the project, then start() it: two daemon snapshots, the second built from the project as the caller gave it rather than the canonical one the Create phase ran on. It now prepares one Create+Start plan, like detached up. The Create phase runs under the "up" progress scope, then the attach/printer/ monitor session is set up exactly as before, and the Start phase is dispatched last, outside that scope, on a context that survives Ctrl+C, with the printer as the hook-log listener. A failed Create phase reaches neither the session nor the Start phase, and a dry run only ever runs the Create phase. The monitor's starting point is the daemon's time, which is read right before the Start phase: a daemon whose time can't be read fails the up with an explicit error, with no container started. The session, the watcher and the teardown now use the plan's own project, which carries what preparation resolved. That makes use_api_socket work in a foreground up: its config file lives only in that project. A scenario locks it. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

The scenario asserted the effect of pre_start only, because its output never reached the session. It does now that the hook's log stream is opened once the hook started. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

A pre_start runner carries the project and service labels but not the one-off label the monitor's event filter requires, so its exit never reaches the session. Nothing locked that: a runner exiting successfully would otherwise win over the service's own exit code under --exit-code-from, or abort the application under --abort-on-container-exit. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

ndeloof requested review from docker-agent and a balanced review from Copilot October 8, 2026 11:19

Copilot started reviewing on behalf of ndeloof October 8, 2026 11:20

View session

ndeloof mentioned this pull request Oct 8, 2026

Epic: converge the start phase into the plan engine #14081

Open

13 of 18 tasks

Copilot AI reviewed Oct 8, 2026

View reviewed changes

docker-agent reviewed Oct 8, 2026

View reviewed changes

ndeloof mentioned this pull request Oct 8, 2026

api: reject incompatible Up options with a typed error #14306

Open

ndeloof force-pushed the up-interactive-plan branch from 10cac17 to 6844922 Compare October 8, 2026 11:53

ndeloof requested review from docker-agent and a balanced review from Copilot October 8, 2026 11:53

Copilot started reviewing on behalf of ndeloof October 8, 2026 11:54

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

Comment thread pkg/compose/up.go

docker-agent reviewed Oct 8, 2026

View reviewed changes

ndeloof force-pushed the up-interactive-plan branch from 6844922 to 66baf1b Compare October 8, 2026 12:14

ndeloof requested review from docker-agent and a balanced review from Copilot October 8, 2026 12:14

Copilot started reviewing on behalf of ndeloof October 8, 2026 12:15

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

docker-agent reviewed Oct 8, 2026

View reviewed changes

ndeloof mentioned this pull request Oct 8, 2026

chore: epic #14081 integration branch (validation only, do not merge) #14308

Draft

ndeloof force-pushed the up-interactive-plan branch from 66baf1b to 1ac85a6 Compare October 16, 2026 09:28

ndeloof mentioned this pull request Oct 16, 2026

fix(pre_start): capture a hook's output by opening its log stream once started #14353

Merged

ndeloof force-pushed the up-interactive-plan branch from 1ac85a6 to 8361db1 Compare October 16, 2026 12:13

ndeloof added 6 commits October 16, 2026 15:13

ndeloof added 2 commits October 16, 2026 15:13

ndeloof force-pushed the up-interactive-plan branch from 8361db1 to 7c2b70b Compare October 16, 2026 13:14

ndeloof mentioned this pull request Oct 16, 2026

fix(monitor): subscribe to events since before the initial listing #14304

Open

Navigation Menu

Repository navigation

feat(up): interactive up runs on a single plan - #14305

feat(up): interactive up runs on a single plan#14305
ndeloof wants to merge 8 commits into
docker:mainfrom
ndeloof:up-interactive-plan

ndeloof commented Oct 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

ndeloof Oct 8, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Oct 8, 2026 •

edited

Loading

Uh oh!

docker-agent left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

docker-agent left a comment

Uh oh!

docker-agent Oct 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

docker-agent left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	if !config.Required && origCtx.Err() == nil {
	if errors.Is(ctx.Err(), context.DeadlineExceeded) {
	if !config.Required && (origCtx.Err() == nil \|\| errors.Is(origCtx.Err(), context.Canceled)) {

Conversation

ndeloof commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does, in one sentence

Context

What the PR brings

Why this is the right next brick

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

ndeloof Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

docker-agent left a comment

Choose a reason for hiding this comment

Assessment: 🟢 APPROVE

Lower-confidence findings (not posted inline)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

docker-agent left a comment

Choose a reason for hiding this comment

Assessment: 🟢 APPROVE

Uh oh!

docker-agent Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

docker-agent left a comment

Choose a reason for hiding this comment

Assessment: 🟢 APPROVE

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ndeloof commented Oct 8, 2026 •

edited

Loading

ndeloof Oct 8, 2026 •

edited

Loading

codecov Bot commented Oct 8, 2026 •

edited

Loading