Skip to content

[GH-3407] Use community MinIO fork image for S3 reader tests - #3408

Merged
jiayuasu merged 1 commit into
apache:masterfrom
james-willis:jw/minio-image-fork
Sep 26, 2026
Merged

jiayuasu merged 1 commit into
apache:masterfrom
james-willis:jw/minio-image-fork

Conversation

@james-willis

@james-willis james-willis commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Did you read the Contributor Guide?

Is this PR related to a ticket?

What changes were proposed in this PR?

MinIO has withdrawn its public container images. quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z no longer resolves, and Docker Hub minio/minio denies anonymous pulls. As a result, the two S3 tests in GeoPackageReaderTest and OsmReaderTest fail on every java.yml run.

This PR switches both tests to pgsty/minio:RELEASE.2026-08-04T00-00-00Z, a community-maintained fork of minio/minio published on Docker Hub. It is still marked .asCompatibleSubstituteFor("minio/minio"). The fork keeps the upstream image layout: minio entrypoint, port 9000, MINIO_ROOT_USER/MINIO_ROOT_PASSWORD, and the /minio/health/live health endpoint. So MinIOContainer, the io.minio client, and the S3A path-style settings all stay as they are, and the change is two lines.

Alternatives considered:

  • Adobe S3Mock or LocalStack would mean replacing MinIOContainer and the test helpers in every Spark module. Recent LocalStack images also need an auth token.
  • bitnamilegacy/minio is frozen and receives no updates.
  • Chainguard's free tier only publishes latest, so the version can't be pinned.
  • [GH-3398] Keep extra predicates on broadcast and nested KNN joins #3399 takes another route: it builds an image from the upstream GitHub release binary at test time. That keeps the exact MinIO version but adds a Dockerfile, a commons-compress workaround, and a GitHub download on every run. This PR is a smaller standalone fix that unblocks CI for all open PRs.

Other uses: I found no other references to MinIO images in python/, R/, docker-compose files, or docs.

How was this patch tested?

  • docker pull pgsty/minio:RELEASE.2026-08-04T00-00-00Z succeeds anonymously. The tag is multi-arch (amd64/arm64).
  • Locally with JDK 17: mvn -pl spark/common test -Dtest=none -DfailIfNoTests=false -DwildcardSuites=org.apache.sedona.sql.GeoPackageReaderTest,org.apache.sedona.sql.OsmReaderTest reported Tests: succeeded 34, failed 0, including both S3 tests.

License

  • pgsty/minio is licensed AGPL-3.0 (repo). That is the same license as the quay.io/minio/minio image these tests already used, so the license posture doesn't change.
  • The image is only pulled at test time by Testcontainers and runs as a separate process. It is not a Maven dependency, it is not bundled in any Sedona artifact, and no AGPL code is included in or linked into Sedona source or binary releases.
  • The Java-side test dependencies (org.testcontainers:testcontainers-minio, io.minio:minio) are Apache-2.0 and unchanged.

Did this PR include necessary documentation updates?

  • No, this PR does not affect any public API so no need to change the documentation.

@james-willis
james-willis marked this pull request as ready for review September 25, 2026 22:09
@jiayuasu
jiayuasu merged commit 07ce415 into apache:master Sep 26, 2026
41 checks passed
jiayuasu added a commit to kalayciburak/sedona that referenced this pull request Sep 26, 2026
Keep the MinIO S3 test image from upstream apache#3408 and remove the branch-only image builder and Dockerfile. The remaining PR diff contains only the KNN join fix, its tests, and documentation.
@james-willis
james-willis deleted the jw/minio-image-fork branch October 1, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

java.yml S3 reader tests fail: MinIO test image no longer publicly pullable

2 participants