Add generated Minimal Todo container benchmarks by LoopedBard3 · Pull Request #2215 · aspnet/Benchmarks

LoopedBard3 · 2026-10-06T21:35:46Z

Summary

Adds a generated Minimal Todo benchmark family for C#, Go, and Rust. Each application implements the same small, read-only HTTP API, serializes the fixed dataset on each successful Todo request, and ships with a Dockerfile for container builds.

This repository carries the runtime application source, Crank scenarios, and lab scheduling integration only, a build/run/operate projection, not the full generated candidate. The generated test suite, fixtures, and validation/admission/comparison history for this cohort are owned by the separate dotnet-performance-tools workflow and are not mirrored into this repository.

Execution paths and cadence

Language	Scheduled scenario	Build method	Cadence per pod
C#	`csharp-project`	Direct Crank build of the `.csproj`, targeting `net11.0` with `channel: latest`	Every regular slot, currently twice daily
Go	`go`	Docker	Weekly
Rust	`rust`	Docker	Weekly

Scheduled runs target gold-lin, azure-arm64, and cobalt-cloud-lin.
Go and Rust share the existing Math.round(Date.now() / 43200000) % 14 == 0 gate, combined with each pod's condition. The current 12-hour pipeline schedules give one eligible slot per week per pod; this is not a global weekly run or a deduplication mechanism for manual reruns.
The Docker scenarios (csharp, go, rust) in this repository are a test-stripped runtime projection, available here for manual resource comparisons and container smoke testing, not the full generated candidate. The canonical full source, tests, and build recipes used for generation, admission, and pre-promotion comparison live in the separate dotnet-performance-tools workflow. The direct C# scenario does not replace that admission or comparison process.

Workload and reporting

Setting	Scheduled value
Application limits	1 CPU, 512 MiB
Load	HTTP/1.1 `GET /todos`, 25,200 offered requests/second, 128 connections
Timing	30-second warmup, 120-second measurement, 2-second request timeout
Application/load port	8080

The scenarios use the official Crank Bombardier job and the shared CI/Azure deployment profiles. Reporting uses stock request/error/latency/throughput metrics and the shared steadystate.profile.yml CPU/working-set P90 results. Raw Bombardier JSON, measurement arrays, and metadata are excluded from scheduled persistence.

Application settings use normal typed job properties and --application.* overrides, with no scenario-specific onConfigure or calibration scripts. Manual source selection defaults to main; scheduled runs pin the triggering commit using --application.sources.app.branchOrCommit "#$(Build.SourceVersion)". Readiness uses Application started.; full contract verification belongs to the generation/admission flow rather than every scheduled run.

Validation

GitHub build/test and CLA checks pass.
All 43 pod-scheduler tests pass; regenerated pipeline YAML remains consistent with its source configuration.
All four scenarios resolve with Crank --debug, including default settings and CPU/memory/port/commit/local-source overrides, using test-only endpoints rather than live agents.
The C# application was built and exercised on Linux ARM64 through both Docker and direct Crank execution; native source/runtime identity, resource limits, HTTP results, and owned-resource cleanup were checked. This evidence predates this round's runtime-only publication cleanup (test/fixture removal); Docker remains unavailable in this checkout to re-verify the cleaned-up recipe.
Separate workflow-level validation (generation/admission/comparison integration checks) is owned by the dotnet-performance-tools workflow and is not part of this PR's CI.

Limitations and operational notes

The 25,200-RPS operating rate was calibrated against an earlier pinned Docker cohort. It is retained as a starting policy, not a recalibration of the current floating images or direct C# build.
The exploratory C# Docker/direct comparison showed similar throughput and latency and approximately 3% lower mean quota-relative CPU for the direct build. It is not a fully accepted paired result: the direct run's controller exit status was lost to a local supervisor bug, and the two builds used different .NET versions and memory measurement definitions.
Docker images float on .NET 11 nightly, Go 1.27, and Rust 1.96; Docker runtime images use Ubuntu 26.04. Direct C# runs use the agent's host OS and Crank's coherent latest toolchain. These choices, plus the Bombardier import tracking main, intentionally trade exact cross-run reproducibility for tracking current builds.
Current floating Go/Rust images have not received a new matched-rate calibration in this effort. The first scheduled results should be treated as the operational baseline for those configurations.
Scheduling changes take effect after merge and a subsequent scheduled pipeline run.

Adds a frozen, reviewed Minimal Todo benchmark cohort (C#, Go, Rust) with a versioned HTTP contract and one native Crank scenario file. Source generation, admission review, pre-promotion comparison, and quota calibration tooling for this cohort -- including the HTTP contract fixture and the one-time state/body verification that uses it -- are owned by the separate dotnet-performance-tools repository; this change is the shipped, day-to-day Crank runtime surface only. - One scenario/minimal-todo/minimal-todo.benchmarks.yml defines all three language scenarios with two roles each (application, load; no separate contract-check/before/after roles and no checks/ fixture fetch). Regular runs validate: application readiness (the startup gate), then the measured Bombardier result strictly (exact URL/method/client/rate/duration/ connections/timeout/request headers, response counters all 2xx with zero other status classes, zero transport errors, delivered rate/duration within 5% of offered), plus CPU/memory tail coverage and sample-gap checks. No requiredArchitecture/architecture gating: Docker images are multi-arch, so the same scenario runs unmodified on x64 or ARM64 agents. Manual connected runs use the shared build/ci.profile.yml / build/azure.profile.yml profiles like every other scenario; there is no scenario-specific profile file. - Each app's own Dockerfile builds directly from floating, multi-arch base images: .NET 11 nightly SDK/ASP.NET (no stable .NET 12 tag is published yet), golang:1.27-bookworm, and rust:1.96-bookworm builders, all finishing on a shared ubuntu:26.04 runtime image (verified against the live .NET 11 nightly manifest). No pinned digests, no Dockerfile.daily/global.json SDK pin, no build-context receipt tooling. - Scheduler template (build/minimal-todo-scenarios.yml) is enabled with the previously accepted matched-contract rate (25200 req/s, 1 CPU, 512 MiB, 128 connections, 30s warmup, 120s duration, 2s request timeout, port 8080), dispatched to the gold-lin, azure-arm64, and cobalt-cloud-lin pods. Validation: offline Crank config resolution (--profile offline) succeeds for all three scenarios (now two roles each); the full scripts/pod-scheduler test suite (43 tests, including the committed-YAML snapshot check) passes; no dead references to any removed file remain. Docker and the floating .NET 11/ Go 1.27/Rust 1.96 toolchains were not available in this environment, so the Dockerfiles could not be build-verified here. This round changed the Docker base images and the C# project's target framework; the previously accepted 25200 req/s rate was calibrated against the prior pinned images and has not been re-run against these new images. The first post-merge scheduled run is the first operational confirmation under the new images, not a re-validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

… scenario Follow-up nits on the generated Minimal Todo scenario, no history rewrite: - Remove the custom offline variable/profile entirely (no other scenario in this repository uses one). The application/contract-check jobs no longer clear endpoints conditionally; the bombardier job's onConfigure simply validates that application/load each resolve to exactly one agent endpoint, matching the connected-run assumption every other scenario makes. - Remove job-level options that only restated Crank's own defaults (reuseBuild/reuseSource/collectCounters/downloadOutput/ downloadOutputOutput/displayBuild/displayOutput/noClean: false) and that have no prior art anywhere else in scenarios/*.benchmarks.yml; options blocks now contain only requiredOperatingSystem: linux, matching the repository-wide convention. - Remove the per-app job variables language/framework/sdkImage/runtimeImage and the onResultsCreated block that copied them into todo/* properties: they duplicated the todo/language and todo/framework --property values the scheduler template already passes, and were not read by the build itself. The calibration summary (todo/calibration/*, todo/cpu/*, todo/memory/*) is unchanged. - README: removed the offline-resolution section describing the now-removed profile; no other content changed. Validation: static config assertions confirm the offline variable/profile, the removed options, and the removed job variables are all absent. The full scripts/pod-scheduler test suite (43 tests) passes; regenerating both CI pipelines produces no diff. Structural resolution was re-verified with crank --debug against a test-only sanitized temp copy of the scenario file (outside this repository, never committed) with a non-listening loopback placeholder endpoint per role -- --debug only resolves and serializes the job graph, so no real agent was contacted. Docker and the floating .NET 11/ Go 1.27/Rust 1.96 toolchains remain unavailable in this environment. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

Follow-up nit, no history rewrite: - Import the official, unmodified Bombardier job via https://github.com/@raw/dotnet/crank/main/src/Microsoft.Crank.Jobs.Bombardier/bombardier.yml instead of redefining its source/framework/SDK/runtime/arguments/ onConfigure/options locally. Deleted the entire local jobs.bombardier definition, including its workload-variable range checks and the application/load endpoint-count/URL-shape validation (no custom endpoint validation needed: the shared ci/azure profile always supplies a real endpoint, matching every other scenario in this repository). - Renamed the top-level requestTimeout variable to timeout (default unchanged at 2) to match the imported job's own variable name, and updated build/minimal-todo-scenarios.yml's --variable requestTimeout=2 to --variable timeout=2. - Each scenario's load role now references the imported bombardier job with explicit variables path: /todos, transport: http1, presetHeaders: none, and customHeaders for Accept: application/json, Connection: keep-alive, and Accept-Encoding: identity. rate/connections/warmup/duration/timeout/ serverAddress/serverPort are intentionally not set per-role: they are supplied the same way as before, via the scheduler template's explicit --variable flags (or an operator's own CLI variables for a manual run), not a scenario-file override. - README: describes the imported Bombardier job and its explicit variables instead of the old pinned-commit wrapper description. Validation: resolved a test-only sanitized temp copy of the scenario file (outside this repository, never committed) with crank --debug, using a non-listening loopback placeholder endpoint -- confirmed the import resolves and merges correctly for all three languages, both with no CLI variables (imported job's own defaults apply, as expected) and with the same --variable overrides the scheduler template passes (connections, warmup, duration, rate, timeout, serverPort all bind correctly into the imported job, and the resulting Bombardier arguments match exactly what was previously hardcoded). The only network access during this validation was crank's own normal fetch of the imported YAML's raw GitHub URL (the same mechanism every scenario that imports wrk.yml/bombardier.yml already relies on) plus a refused local loopback connection; no real agent was contacted. The full scripts/pod-scheduler test suite (43 tests) passes; regenerating both CI pipelines produces no diff beyond the already-staged timeout rename in build/minimal-todo-scenarios.yml. Docker and the floating .NET 11/Go 1.27/Rust 1.96 toolchains remain unavailable in this environment. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

Copilot

🟡 Changes recommended

The C# container restore omits its lock file before invoking locked mode, preventing the image from building.

5 open findings

Copy packages.lock.json before locked restore · New
Correct validation record for the .NET 11 target · New
Correct README claim about digest-pinned builder image · New
Clarify build architecture support and cross-compilation behavior · New
Update deployment description to identify Ubuntu runtime · New

What changed in this PR

Adds scheduled Minimal Todo container benchmarks across C#, Go, and Rust.

Changes:

Adds equivalent read-only Todo services with contract and lifecycle tests.
Adds Docker builds and a shared Crank Bombardier scenario.
Enables scheduled execution across Linux benchmark pods.

File	Description
`src/BenchmarksApps/MinimalTodo/rust/tests/lifecycle.rs`	Tests Rust process lifecycle.
`src/BenchmarksApps/MinimalTodo/rust/tests/fixtures/minimal-todo-v1.expected.json`	Defines Rust golden response.
`src/BenchmarksApps/MinimalTodo/rust/tests/contract.rs`	Tests Rust HTTP contract.
`src/BenchmarksApps/MinimalTodo/rust/src/main.rs`	Hosts the Rust service.
`src/BenchmarksApps/MinimalTodo/rust/src/lib.rs`	Implements Rust endpoints.
`src/BenchmarksApps/MinimalTodo/rust/rust-toolchain.toml`	Selects Rust 1.96.
`src/BenchmarksApps/MinimalTodo/rust/README.md`	Documents Rust operation.
`src/BenchmarksApps/MinimalTodo/rust/Dockerfile`	Builds the Rust image.
`src/BenchmarksApps/MinimalTodo/rust/Cargo.toml`	Declares Rust dependencies.
`src/BenchmarksApps/MinimalTodo/rust/Cargo.lock`	Locks Rust dependencies.
`src/BenchmarksApps/MinimalTodo/rust/.gitignore`	Excludes Rust build output.
`src/BenchmarksApps/MinimalTodo/rust/.dockerignore`	Limits Rust build context.
`src/BenchmarksApps/MinimalTodo/go/testdata/minimal-todo-v1.expected.json`	Defines Go golden response.
`src/BenchmarksApps/MinimalTodo/go/server.go`	Implements Go endpoints.
`src/BenchmarksApps/MinimalTodo/go/server_test.go`	Tests Go HTTP behavior.
`src/BenchmarksApps/MinimalTodo/go/README.md`	Documents Go operation.
`src/BenchmarksApps/MinimalTodo/go/main.go`	Hosts the Go service.
`src/BenchmarksApps/MinimalTodo/go/lifecycle_test.go`	Tests Go lifecycle behavior.
`src/BenchmarksApps/MinimalTodo/go/go.mod`	Declares the Go module.
`src/BenchmarksApps/MinimalTodo/go/Dockerfile`	Builds the Go image.
`src/BenchmarksApps/MinimalTodo/go/.dockerignore`	Limits Go build context.
`src/BenchmarksApps/MinimalTodo/Directory.Build.targets`	Isolates MSBuild targets.
`src/BenchmarksApps/MinimalTodo/csharp/TodoApi.slnx`	Defines the C# solution.
`src/BenchmarksApps/MinimalTodo/csharp/tests/TodoApi.Tests/TodoApi.Tests.csproj`	Configures C# tests.
`src/BenchmarksApps/MinimalTodo/csharp/tests/TodoApi.Tests/packages.lock.json`	Locks test dependencies.
`src/BenchmarksApps/MinimalTodo/csharp/tests/TodoApi.Tests/minimal-todo-v1.expected.json`	Defines C# golden response.
`src/BenchmarksApps/MinimalTodo/csharp/tests/TodoApi.Tests/LifecycleTests.cs`	Tests C# lifecycle behavior.
`src/BenchmarksApps/MinimalTodo/csharp/tests/TodoApi.Tests/ContractTests.cs`	Tests C# HTTP contract.
`src/BenchmarksApps/MinimalTodo/csharp/tests/TodoApi.Tests/AppProcess.cs`	Manages test subprocesses.
`src/BenchmarksApps/MinimalTodo/csharp/tests/TodoApi.Tests/ApiFixture.cs`	Hosts C# test instances.
`src/BenchmarksApps/MinimalTodo/csharp/src/TodoApi/TodoApi.csproj`	Configures the C# app.
`src/BenchmarksApps/MinimalTodo/csharp/src/TodoApi/Todo.cs`	Defines the Todo model.
`src/BenchmarksApps/MinimalTodo/csharp/src/TodoApi/Program.cs`	Implements C# endpoints.
`src/BenchmarksApps/MinimalTodo/csharp/src/TodoApi/packages.lock.json`	Locks application restore.
`src/BenchmarksApps/MinimalTodo/csharp/README.md`	Documents C# operation.
`src/BenchmarksApps/MinimalTodo/csharp/NuGet.Config`	Configures NuGet sources.
`src/BenchmarksApps/MinimalTodo/csharp/Dockerfile`	Builds the C# image.
`src/BenchmarksApps/MinimalTodo/csharp/Directory.Build.props`	Sets shared C# properties.
`src/BenchmarksApps/MinimalTodo/csharp/.gitignore`	Excludes C# outputs.
`src/BenchmarksApps/MinimalTodo/csharp/.dockerignore`	Limits C# build context.
`src/BenchmarksApps/MinimalTodo/.gitattributes`	Preserves generated bytes.
`scenarios/README.md`	Links the new benchmark.
`scenarios/minimal-todo/README.md`	Documents scenario operation.
`scenarios/minimal-todo/minimal-todo.benchmarks.yml`	Defines Crank scenarios.
`scenarios/minimal-todo/.gitattributes`	Fixes scenario line endings.
`build/README.md`	Documents scheduling.
`build/minimal-todo-scenarios.yml`	Defines scheduled dispatches.
`build/job-variables.yml`	Registers the scenario config.
`build/benchmarks-ci-azure.yml`	Schedules Azure pod runs.
`build/benchmarks-ci-02.yml`	Rebalances generated scheduling.
`build/benchmarks-ci-01.yml`	Schedules gold-lin runs.
`build/benchmarks_ci_pods.json`	Adds the local pod cohort.
`build/benchmarks_ci_azure_pods.json`	Adds the Azure pod cohort.

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Follow-up, no history rewrite. Adds a side-by-side comparison point alongside the existing Docker csharp scenario, which remains unchanged (verified byte-identical in the diff). - New job todo-csharp-project builds src/TodoApi/TodoApi.csproj directly with Crank (project: + framework: net11.0 + channel: latest, confirmed real Crank Job model fields via the actual dotnet/crank source), no Docker fields at all. Reuses the exact same sources/sourceRevision main-or-40hex validation and CPU/memory/port onConfigure logic as todo-csharp. - New scenario csharp-project pairs todo-csharp-project with the same imported Bombardier load job and variables (path: /todos, transport: http1, presetHeaders: none, customHeaders for Accept/Connection/Accept-Encoding) as the csharp scenario. - Not added to build/minimal-todo-scenarios.yml or any pod/CI configuration: the scheduled csharp scenario stays the Docker baseline; csharp-project is only ever run manually via an explicit --scenario csharp-project. - README: documents the optional scenario, why channel: latest is used (coherent latest SDK/runtime/ASP.NET Core, not a floating Docker digest), and that Crank's own CPU/memory limits (cgroups) apply identically to both jobs so a comparison isolates packaging/runtime/host-OS effects rather than a resource-quota difference. Validation: resolved a test-only sanitized temp copy of the scenario file (outside this repository, never committed) with crank --debug for all four scenarios (csharp, go, rust, csharp-project) using a non-listening loopback placeholder endpoint -- all resolve cleanly; confirmed the csharp-project application job resolves Project/Framework/Channel/Port correctly and has no DockerFile. The full scripts/pod-scheduler test suite (43 tests) passes; regenerating both CI pipelines produces no diff (this scenario is not referenced by any pod/CI configuration). No real agent was contacted. Docker and the floating .NET 11/Go 1.27/Rust 1.96 toolchains remain unavailable in this environment. Live/lab execution (the requested one-time paired Docker-vs-direct comparison on real agents) is out of scope for this commit and this session: this checkout has no credentials or network path to real lab agents, consistent with this effort's established division of responsibility where live dispatch is performed by the parent/native executor, never by this candidate-integration checkout. Reported this limitation back to the requester; the comparison itself has not been run. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

Addresses five PR review comments on aspnet#2215 (normal follow-up commit, no history rewrite, no scenario/scheduler/workload change): 1. (csharp/Dockerfile, review 4213810222) The pre-restore COPY only included TodoApi.csproj, so `dotnet restore --locked-mode` ran with no packages.lock.json on disk; the real lock file only arrived in the later `COPY src/TodoApi/ src/TodoApi/` step, after restore had already run. Added packages.lock.json to the pre-restore COPY alongside the csproj (not excluded by .dockerignore). Locked-mode and the lock file itself are unchanged -- this only fixes when the lock file becomes visible to restore. 2. (csharp/README.md, review 4213810258) The "Verification in the supplied environment" section (Windows SDK 10.0.401, runtimes 10.0.12) is now explicitly labeled as historical/original .NET 10 evidence, predating this round's floating .NET 11 images; states plainly that the current .NET 11 configuration was not build-verified in this checkout. Also corrected two stale present-tense claims: "strict runtime selection" (the RuntimeFrameworkVersion/roll-forward pin it described was removed from Directory.Build.props this round) and "the pinned runtimes" for framework-dependent RID publishes (now "an installed, compatible runtime"), plus the package-lock description (the app does have its own packages.lock.json, just with an empty recorded dependency set, not "no package lock"). 3/4. (go/README.md, reviews 4213810293 and 4213810316) Corrected the top-of-file claim that "the Docker builder pins that version and its image digest" -- the builder floats on `golang:1.27-bookworm` with no digest pin, and `go.mod`'s `go 1.27` is a minimum-version directive, not an exact pin. Corrected the Container section and Files list, which claimed "native builder to cross-compile for the selected target" and "pinned multi-stage Linux container build": the Dockerfile has no BUILDPLATFORM/TARGETARCH, so `docker build --platform ...` selects a single target platform for the whole build (native for that platform, via emulation when needed) rather than cross-compiling from one builder architecture to another; that's a different mechanism from the standalone GOOS/GOARCH cross-compilation commands already documented separately (left unchanged). 5. (rust/README.md, review 4213810360) Corrected "a minimal Debian runtime" in the current "Linux containers" section to `ubuntu:26.04` (the actual current runtime base). The Validation section's historical "Runtime image" row (Debian glibc 2.36-9+deb12u14) is preserved verbatim but its heading/intro now explicitly label it as the original Debian-based build from before this round's switch to Ubuntu, so it can't be misread as evidence the current image was tested. Validation: for item 1, Docker is unavailable in this environment, so I built the smallest possible local reproduction instead -- a byte-for-byte copy of the real build context, staged exactly like each Dockerfile COPY step, restored/published with the local SDK (TargetFramework temporarily overridden net11.0->net10.0 in that throwaway copy only, since net11.0 isn't installed locally; no change to the real project). Both the old and the fixed COPY order restore and publish successfully (exit 0) for this specific project today: TodoApi.csproj has zero package references, and neither the csproj nor Directory.Build.props sets RestorePackagesWithLockFile, so no lock file is auto-generated and --locked-mode has nothing to validate against either way right now -- reproducing NU1004 was not possible today, and I am not claiming it was observed. The fix is still correct and makes --locked-mode meaningful going forward (e.g. once a real dependency is added, restore will then actually see and validate against the committed lock file before building, instead of silently not reading it). All four touched files: git diff --check is clean (one pre-existing CRLF/cr-at-eol cosmetic flag per file, consistent with each file's existing line-ending convention, not a real defect, same pattern as every prior round in this PR). No scenario, scheduler, pod, or workload file was touched. Per-comment coverage: all five identified review comments (4213810222, 4213810258, 4213810293, 4213810316, 4213810360) are addressed by this commit. Threads are intentionally left unresolved pending confirmation this fix is accepted. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

…ocker Normal follow-up commit, no history rewrite/scheduler-architecture change/ main-branch change. Implements a new scheduling policy while keeping the Docker generation/comparison path fully intact: - build/minimal-todo-scenarios.yml: the C# scenario entry now submits `--scenario csharp-project` (the existing direct, non-Docker build) instead of `--scenario csharp`, unconditionally on every regular scheduled run (condition: 'true', same cadence as before). todo/language=csharp and todo/framework=net11.0 properties are unchanged. The Go and Rust entries keep their existing Docker `go`/`rust` scenarios but add `condition: Math.round(Date.now() / 43200000) % 14 == 0` (true once every 14 half-day scheduled slots, i.e. once per ~7 days for each pipeline's own 12-hour cron), sharing the same slot value so both languages fall on the same calendar day within a given pipeline. The published Service Bus message's condition now combines both: `(parameters.condition) && (scenario.condition)`, matching the existing idiom in build/containers-scenarios.yml, rather than bypassing the pod-level condition. - The Docker todo-csharp/csharp, todo-go/go, and todo-rust/rust jobs and scenarios in scenarios/minimal-todo/minimal-todo.benchmarks.yml are completely unchanged (not renamed, not removed) and remain the canonical generation/update/admission/pre-promotion comparison path. Rate (25200), 1 CPU, 512 MiB, 128 connections, 30s/120s warmup/duration, 2s timeout, port 8080, and the shared steadystate config are all unchanged. - scenarios/minimal-todo/README.md: replaced the old "optional, not scheduled" description of csharp-project with a new "Execution paths" section mapping which scenario is scheduled where and why, including an explicit, hedged disclosure of the preliminary Docker-vs-direct comparison (~3% lower CPU quota for direct, similar latency; process-exit verification had an unresolved local-reader-bug caveat) -- stated plainly as not a fully accepted paired test or recalibration; the retained 25200 rate is the prior operational policy value, not re-derived from this comparison. - build/README.md: updated the Minimal Todo section to describe the new per-language cadence and point at the scenario README for the full mapping. Validation: regenerating both CI pipelines produces no diff (the cadence condition lives entirely inside the scenario-dispatch template, which is expanded at actual pipeline-run time, not at pod-scheduler generation time). The full scripts/pod-scheduler test suite (43 tests) passes. Confirmed no other file overrides minimal-todo-scenarios.yml's `scenarios` parameter default (the only two pipeline call sites pass only connection/serviceBusQueueName/serviceBusNamespace/arguments), so adding a `condition` key to every default scenario item cannot cause an Azure template "object does not contain property" expansion error elsewhere. Verified the weekly-condition formula numerically against both pipelines' actual cron schedules (ci01/ci02 "0 3/12 * * *", Azure "0 2/12 * * *") across 28 consecutive triggers (14 days) from several arbitrary start dates: exactly 2 of 28 (i.e. exactly 1 per 7 days) evaluate true for each cadence, every time -- a direct consequence of the triggers being an exact-12-hour arithmetic sequence, mod 14. Re-verified all four scenarios (csharp, go, rust, csharp-project) still resolve cleanly via `crank --debug` against a test-only sanitized temp copy (outside this repository, never committed, non-listening loopback placeholder endpoint; no real agent contacted). git diff --check is clean (pre-existing per-file CRLF/cr-at-eol cosmetic flag, same pattern as every prior round, not a real defect). No app code, build mode, or toolchain was changed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

…ions Normal follow-up commit, no history rewrite/scheduler-architecture change/ live jobs. Replaces the bespoke per-job onConfigure hooks (resource-variable -> typed-property mapping, main-or-SHA format validation, forced LocalFolder=null) with plain Crank job properties, matching the established pattern in build/containers-scenarios.yml (--application.cpuLimitRatio / --application.memoryLimitInBytes). - Removed all four app onConfigure blocks (todo-csharp, todo-csharp-project, todo-go, todo-rust): no replacement runtime JS, no forced LocalFolder clear, no main-or-40hex script guard. Normal Crank job/profile/CLI semantics now apply, including local-source overrides (previously impossible: the removed hook unconditionally reset LocalFolder to null on every run regardless of any --application.sources.app.localFolder override). - Each app job now sets typed defaults directly: cpuLimitRatio: 1, memoryLimitInBytes: 536870912, port: 8080 -- preserving the existing 1 CPU / 512 MiB manual default (deleting the hook with no replacement would have silently made these 0/unlimited). All three are overridable via --application.cpuLimitRatio / --application.memoryLimitInBytes / --application.port, same as any other scenario. - The "app" source keeps its name/destination/remote-repository/cache policy; branchOrCommit now defaults to the literal main (a normal branch clone) instead of a templated "#{{sourceRevision}}" marker. The scenario's sourceRevision variable is removed; pinning an exact commit is now a normal CLI source override, --application.sources.app.branchOrCommit "#<sha>" (the nested sources dictionary form, not the obsolete --application.source.* alias -- "app" is a named source, not the default unnamed one). The obsolete cpu/memoryBytes/sourceRevision scenario variables are removed now that every consumer has been rewired; the load/workload variables (serverAddress/serverPort/rate/warmup/duration/ connections/timeout) and the imported Bombardier headers are unchanged. - Each app's PORT environment variable now derives from the job's own resolved port ("{{job.Port}}") instead of the removed serverPort-reading hook, so --application.port overrides correctly flow into the container's listening port. The imported Bombardier load job's own default port (5000) no longer matches the app's default (8080); each scenario's load role now sets variables.serverPort: 8080 (a plain per-role variable default, no JS, the same mechanism already used for path/transport/ presetHeaders/customHeaders) so the two match by default, while a CLI --variable serverPort=<port> still overrides both app and load together when passed explicitly. - build/minimal-todo-scenarios.yml: replaced --variable cpu=1 --variable memoryBytes=536870912 --variable sourceRevision=$(Build.SourceVersion) with --application.cpuLimitRatio 1 --application.memoryLimitInBytes 536870912 --application.port 8080 --application.sources.app.branchOrCommit \"#$(Build.SourceVersion)\" (JSON- escaped double quotes around the commit value, matching this same messageBody's existing quoting convention for other values, so the leading "#" cannot be read as a shell comment start by whatever splits this string downstream). --variable serverPort=8080 and the other workload variables are unchanged; result properties (cpu/memoryBytes/ todo/*) are unchanged and remain truthful. - Docs: updated scenarios/minimal-todo/README.md's "Scenario" section to describe the new typed-property/CLI-override shape instead of sourceRevision/onConfigure, and fixed the two stale runtime YAML comment blocks (one job comment, one scenario comment) that still described csharp-project as "optional"/"not part of any scheduled lane" -- it has been the scheduled day-to-day C# lane since commit 3a802b0. Also corrected a "ci01/ci02" imprecision to "ci01" (ci02's cron and contents are unrelated to Minimal Todo). Validation: all four scenarios (csharp, go, rust, csharp-project) resolve cleanly via crank --debug against a test-only sanitized temp copy (outside this repository, never committed, non-listening loopback placeholder endpoint; no real agent contacted), confirmed for every combination the exact requirement calls for: - No CLI overrides: CpuLimitRatio=1.0, MemoryLimitInBytes=536870912, Port=8080, BranchOrCommit="main", EnvironmentVariables.PORT="8080", and the load job's resolved URL is http://.../8080/todos (serverPort default now matches) -- for all four scenarios. - With --application.cpuLimitRatio 0.5 --application.memoryLimitInBytes 268435456 --application.port 8091 --application.sources.app.branchOrCommit "#<40-hex-sha>" --variable serverPort=8091: CpuLimitRatio=0.5, MemoryLimitInBytes=268435456, Port=8091, BranchOrCommit="#<sha>" (not truncated at the "#"), EnvironmentVariables.PORT="8091" (matches Job.Port, not left stale), and the load job's resolved URL is http://.../8091/todos. - --application.sources.app.localFolder override is honored (previously impossible due to the removed forced-clear hook). - The imported official Bombardier job is still used and its arguments (headers, --http1, path) are unchanged in both cases. Separately simulated the exact build/minimal-todo-scenarios.yml messageBody JSON with placeholder values substituted for every ${{ }}/$() token: the result parses as valid JSON and the resolved args string contains --application.sources.app.branchOrCommit "#<sha>" with the quotes intact around the "#", matching this exact quoting convention already used successfully elsewhere in the same messageBody for other special-character values. The full scripts/pod-scheduler test suite (43 tests) passes; regenerating both CI pipelines produces no diff beyond the minimal-todo-scenarios.yml edit itself. git diff --check is clean (the one remaining flag is the same pre-existing per-file CRLF/cr-at-eol cosmetic pattern as every prior round, confirmed not a real defect). No app code, Dockerfile, job/scenario rename, SelfContained/framework/channel, or generation/comparison Docker path was changed; the Go/Rust weekly and C# regular-cadence scheduling from the prior commit is unchanged. No live agents or Docker commands were used. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

DrewScoggins

Left comments, but once addressed LGTM.

Normal follow-up commit, no history rewrite/scheduler-architecture change/ merge/live jobs. The PR was already marked ready for review (draft: false); that state is preserved. RUNTIME CONFIG (no workload/value change): - Added a `# 512 MiB` human comment to each of the four app jobs' memoryLimitInBytes: 536870912 (todo-csharp, todo-csharp-project, todo-go, todo-rust). - Added an explicit `rate: 100 # Override with --variable rate=...` to each of the four scenarios' load role (csharp, go, rust, csharp-project). The imported Bombardier job's own default rate is 0 (uncapped); the scenario's former root-level `rate: 100` variable never actually reached the load job (the same non-propagation behavior already documented for serverPort), so a manual run with no --variable override was silently uncapped rather than the intended low smoke-test default. Removed the now-redundant root `rate` variable (nothing in this file templates `{{rate}}` directly; only the per-role default has any effect). A CLI `--variable rate=...` override (e.g. the scheduled 25200) still wins over this default, confirmed empirically. TEST/FIXTURE/EVIDENCE REMOVAL (runtime repo only -- tools-side canonical archive is untouched, already independently preserved per 248's prior confirmation): - Deleted csharp/tests/TodoApi.Tests/ entirely (ApiFixture.cs, AppProcess.cs, ContractTests.cs, LifecycleTests.cs, TodoApi.Tests.csproj, minimal-todo-v1.expected.json, packages.lock.json). - Deleted go/lifecycle_test.go, go/server_test.go, go/testdata/ (the golden fixture). - Deleted rust/tests/ entirely (contract.rs, lifecycle.rs, fixtures/minimal-todo-v1.expected.json). - Zero production .cs/.go/.rs files were touched (confirmed via `git diff --cached --stat` filtered to those paths: no output). The fixed five-Todo dataset, Program.cs/Todo.cs, main.go/server.go, main.rs/lib.rs (including its embedded `#[cfg(test)]` unit tests, which are part of the production file and were left untouched), Cargo.toml/Cargo.lock/go.mod, and all Dockerfiles needed for regular runs are all unchanged. WIRED BUILD SURFACES: - rust/Dockerfile: removed `COPY tests ./tests` and the `FROM build AS test` / `RUN cargo test --release --locked` stage; kept the production `cargo build --release --locked`, the ubuntu:26.04 runtime stage, nonroot user, and entrypoint unchanged. - csharp/TodoApi.slnx: removed the `/tests/` folder and TodoApi.Tests.csproj reference, now a single-project solution (src/TodoApi/TodoApi.csproj). - csharp/src/TodoApi/TodoApi.csproj: removed the dead `<InternalsVisibleTo Include="TodoApi.Tests" />` (csproj metadata only, no .cs edit). - Dead dockerignore/gitignore patterns removed: csharp/.dockerignore's `tests` line, csharp/.gitignore's now-unreachable `TestResults/` line, go/.dockerignore's `*_test.go`/`testdata/` lines, rust/.dockerignore's `!tests/`/`!tests/**` allow-list lines (its "ignore everything, allow-list specific paths" pattern no longer needs to admit a path that doesn't exist). README CLEANUP (operator-facing only; full generated test suite, fixture, and validation/campaign history remain owned by the separate dotnet-performance-tools canonical archive): - csharp/README.md, go/README.md, rust/README.md: rewritten concisely to app/framework description, build, run, contract, PORT, readiness, shutdown, and container build/run instructions only. Removed the historical Windows-SDK-10/original-Debian-image validation sections, test run commands/counts, fixture SHA-256 hashes, local artifact/bin/log/ evidence directory listings, and the Files sections' now-dead test-file entries. - scenarios/minimal-todo/README.md: documents the new rate/memory/serverPort per-role defaults; states plainly that production code only ships here (no test projects/fixtures); adds an explicit operational note that the 25200 req/s rate is a uniform policy value applied to all three languages and both the Docker and direct C# builds, not a per-language or per-resource-point capacity guarantee; reinforces that the full generated test suite/fixture/checker/review/campaign history is owned by dotnet-performance-tools. The C#-direct-regular/Go-Rust-weekly cadence and Docker-comparison-availability design from the prior two commits is unchanged and explicitly preserved. Validation: `git diff --cached --stat` filtered to *.cs/*.go/*.rs shows zero production file changes (only test-file deletions). Grepped the full repository for references to every deleted file/directory name: zero remaining matches (no dead links in any README or build/*.yml). All four scenarios (csharp, go, rust, csharp-project) resolve cleanly via `crank --debug` against a test-only sanitized temp copy (outside this repository, never committed, non-listening loopback placeholder endpoint; no real agent or Docker command used), confirming: default load rate is now 100 (previously an uncapped 0), memoryLimitInBytes is unchanged at 536870912, and a `--variable rate=25200` CLI override still wins. The full scripts/pod-scheduler test suite (43 tests) passes; regenerating both CI pipelines produces zero diff. git diff --check is clean (the only flagged lines are the same pre-existing per-file CRLF/cr-at-eol cosmetic pattern seen in every prior round for a fully-rewritten file -- confirmed zero genuine trailing whitespace). Docker and the floating toolchains remain unavailable in this environment, so the Dockerfiles could not be build-verified here; this is an honest, disclosed limitation, not a passed build. No shared agent, Docker daemon, install, or remote probe was started. No live/native/lab/SQL/workflow dispatch was performed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d9d74a4-18fa-4e63-a55d-792c3aabe5f8

DrewScoggins

LGTM

LoopedBard3 force-pushed the loopedbard3-generated-cohort-integration branch 2 times, most recently from e4601bc to 9fed829 Compare October 7, 2026 17:50

LoopedBard3 force-pushed the loopedbard3-generated-cohort-integration branch from 9fed829 to 6e75ce9 Compare October 7, 2026 20:01

LoopedBard3 and others added 2 commits October 7, 2026 13:36

LoopedBard3 self-assigned this Oct 8, 2026

LoopedBard3 requested a balanced review from Copilot October 8, 2026 01:45

Copilot started reviewing on behalf of LoopedBard3 October 8, 2026 01:45

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

LoopedBard3 and others added 4 commits October 7, 2026 19:49

LoopedBard3 requested review from DrewScoggins and sebastienros October 12, 2026 18:13

LoopedBard3 marked this pull request as ready for review October 12, 2026 18:13