Skip to content

build: only warn about stale licenses.yaml versions in PR CI - #20774

Merged
FrankChen021 merged 3 commits into
apache:masterfrom
amaechler:license-check-warn-version-drift
Nov 1, 2026
Merged

FrankChen021 merged 3 commits into
apache:masterfrom
amaechler:license-check-warn-version-drift

Conversation

@amaechler

@amaechler amaechler commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Part of #20732.

Description

Every bump of a bundled dependency fails PR CI until someone edits its version in licenses.yaml by hand. As proposed on dev@, PR CI now only warns about these versions, and the release build stays strict. The release-guide step that refreshes the versions before each release branch follows in a separate PR, also part of #20732.

Warning mode in check-licenses.py

With --stale-versions=warn (the default is fail), a reported dependency becomes a warning instead of an error when:

  • the reported version is in no licenses.yaml entry,
  • its groupId and artifactId are registered under other versions, all with the reported license, and
  • one of those registered versions is no longer in any report, i.e. the dependency was bumped.

The warning lists the registered versions and the commands to refresh licenses.yaml. With the default fail, the release build lists the same dependencies as errors with the same commands, rather than as missing licenses.

Contributor docs

dev/license.md now says what PR CI does when a contributor leaves a version as it is.

How it was checked
  • Unit tests in test_check_licenses.py, which the apache-release build runs:
    • a bumped version warns, including when another registered version still ships;
    • each of these fails: an unregistered dependency, a changed license, an unknown reported license, registered versions with mixed licenses, a registered version that still ships, and an old version registered only under an excluded module while the new one ships in the release;
    • one end-to-end check_licenses() run per mode.
  • Against real dependency reports from a local build, with licenses.yaml made stale by hand (OSHI 7.7.1 to 7.7.0, snappy-java 1.1.10.12 to 1.1.10.11):
    • fail exits 1 and lists the 3 as stale versions with the refresh commands, and warn exits 0 with the same 3 as warnings;
    • with the OSHI license also changed, warn exits 1 on OSHI and still warns on snappy-java;
    • with snappy-java not registered at all, warn exits 1;
    • refresh-license-versions.py then updates exactly those 2 entries, and fail passes again.
  • ./mvnw -pl distribution -Papache-release exec:exec@check-licenses on a stale version fails by default and passes with -Ddruid.distribution.licenses.staleVersions=warn.
  • Replay of build(deps): bump the maven-minor-patch group with 9 updates #20765, the first grouped Dependabot PR, without its licenses.yaml edits:

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The current head correctly keeps stale-version failures strict by default and only downgrades same-license version drift in the PR CI paths. The custom YAML loader preserves dependency versions as strings, while unknown and mismatched license checks remain blocking; I found no actionable correctness or regression issue.

Reviewed 11 of 11 changed files, including the CI scripts, distribution Maven wiring, license checker and parser, YAML loader, refresh/generation callers, registry changes, documentation, and the expanded tests.

Validation: GitHub reported the PR as MERGEABLE/CLEAN, the prepared worktree merged cleanly against origin/master with git merge-tree, git diff --check passed, and bash -n passed for both changed shell scripts. Maven and Python unit tests were not run; this was a static review.


This is an automated review by Codex GPT-5.6 Luna(Max)

@FrankChen021 FrankChen021 added the Approval Recommended Used when AI completes a review and reports no findings label Oct 29, 2026
check-licenses.py gets --stale-versions=fail|warn, defaulting to fail. With
warn, a reported dependency becomes a warning when its groupId and artifactId
are registered under other versions, all with the reported license, and one
of those versions is no longer in any report. The warning gives the commands
to refresh licenses.yaml. Unregistered dependencies, changed or unknown
licenses, license_file_path problems and unknown module tags still fail, and
so does a module that starts bundling another version of a library while the
registered version still ships. If the dependency ships in the release,
registrations under excluded modules do not count.

distribution/pom.xml passes druid.distribution.licenses.staleVersions (default
fail) to the check, and packaging-check.sh sets it to warn, so release builds
stay strict.
@amaechler
amaechler force-pushed the license-check-warn-version-drift branch from 4315404 to 0351daa Compare October 29, 2026 22:02
check-licenses.py now separates stale versions from missing licenses in both
modes, so the release build lists them with the refresh hint instead of as
missing licenses. --stale-versions only decides whether they fail the check.

Drop the --stale-versions flag from license_checks_script.sh: nothing calls
that script.
@amaechler
amaechler marked this pull request as ready for review October 29, 2026 22:11

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The stale-version classifier can downgrade a genuinely missing shipped dependency when its current GAV is registered only under an excluded module. PR CI may then pass while the release build later fails due to the missing license registration.

Reviewed 5 of 5 changed files; finding in 1 file.

Validation: git diff --check and bash -n .github/scripts/packaging-check.sh passed. The targeted Python test was blocked by missing yaml/PyYAML.

Severity Findings
P0 0
P1 1
P2 0
P3 0
Total 1

This is an automated review by Codex GPT-5.6-Luna(max)

After addressing the findings or replying to the comments, you can request another review from me to trigger a new automated review.

Comment thread distribution/bin/check-licenses.py Resolved
@FrankChen021 FrankChen021 removed the Approval Recommended Used when AI completes a review and reports no findings label Oct 31, 2026
…missing

split_stale_versions treated a reported version that is registered only
under an excluded module, with an older version registered under a shipped
module, as stale. When an extension ships that version, the release has no
registration for the jar and refresh-license-versions.py neither updates nor
flags the entry. Only treat a missing version as stale when it is in no
licenses.yaml entry at all.

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The prior P1 is resolved in the current head: stale-version classification now requires the current GAV to be absent from licenses.yaml entirely, so an excluded-only current registration remains a missing license and cannot be downgraded to a warning. The added regression test covers this scenario, and no actionable issues remain in the reviewed changes.

Reviewed 5 of 5 changed files.

Validation: git diff --check passed and bash -n passed for the changed packaging script. The focused Python test was attempted but could not start because the yaml/PyYAML module is unavailable (ModuleNotFoundError: No module named 'yaml'). No Maven or broader tests were run.


This is an automated review by Codex GPT-5.6-Luna(max)

@FrankChen021 FrankChen021 added the Approval Recommended Used when AI completes a review and reports no findings label Oct 31, 2026
@FrankChen021
FrankChen021 merged commit 46d2de3 into apache:master Nov 1, 2026
28 checks passed
@github-actions github-actions Bot added this to the 40.0.0 milestone Nov 1, 2026
@amaechler
amaechler deleted the license-check-warn-version-drift branch November 1, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Approval Recommended Used when AI completes a review and reports no findings Area - Build Area - Dependencies GHA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants