Skip to content

fix(artifacts): reject hierarchical name collisions in FsspecArtifactService - #8175

Open
tang-ontheway wants to merge 1 commit into
google:mainfrom
tang-ontheway:fix/fsspec-hierarchical-name-collision
Open

tang-ontheway wants to merge 1 commit into
google:mainfrom
tang-ontheway:fix/fsspec-hierarchical-name-collision

Conversation

@tang-ontheway

Copy link
Copy Markdown

Fixes #8173

Summary

On hierarchical filesystems (such as file:// and memory://), FsspecArtifactService stores versions at {artifact_path}/{version}, so an artifact (e.g. report) and a nested artifact whose next segment is a numeric version (e.g. report/0 or report/0/sub) map the same path to both a file and a directory:

  1. Saving report then report/0 (or report/0/sub) raised a raw FileExistsError / NotADirectoryError from makedirs or retried 10 times before raising FileExistsError.
  2. Saving report/0 (or report/0/sub) then report hit FileExistsError on the existing report/0 directory, treated it as a concurrent save, and silently skipped to version 1.

This change:

  • Detects directory collisions in the version-write retry loop on non-GCS backends and raises InputValidationError instead of advancing the version counter.
  • Detects file-ancestor collisions in makedirs and _write_version on non-GCS backends and raises InputValidationError instead of leaking raw filesystem errors or retrying.
  • Documents this hierarchical filesystem limitation in the FsspecArtifactService class docstring.
  • Leaves flat object stores (gs://) unaffected.

Testing plan

  • Added parametrized regression tests in tests/unittests/integrations/fsspec/test_fsspec_artifact_service.py covering report -> report/0, report -> report/0/sub, report/0 -> report, report/0/sub -> report, and later-version collisions (report v0 -> report/1 -> report v1) on both memory:// and file://, plus a gs:// test verifying flat object stores still allow both.
  • Ran uv run pytest tests/unittests/integrations/fsspec/test_fsspec_artifact_service.py tests/unittests/artifacts/test_artifact_service.py: 1682 passed, 26 xfailed (with 1 unrelated google.auth FutureWarning).
  • Ran uv run ruff check src/google/adk/integrations/fsspec/_fsspec_artifact_service.py tests/unittests/integrations/fsspec/test_fsspec_artifact_service.py: All checks passed!.
  • Ran the issue FsspecArtifactService: on file:// and memory://, an artifact and a nested artifact named '<it>/<number>' collide #8173 reproduction script against memory:// and file://: both collision directions now raise InputValidationError without advancing versions.

@google-cla

google-cla Bot commented Nov 16, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@tang-ontheway
tang-ontheway force-pushed the fix/fsspec-hierarchical-name-collision branch from 9275253 to c52efe9 Compare November 16, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FsspecArtifactService: on file:// and memory://, an artifact and a nested artifact named '<it>/<number>' collide

2 participants