Skip to content

FsspecArtifactService: on file:// and memory://, an artifact and a nested artifact named '<it>/<number>' collide #8173

Description

@AkshayShah03

🔴 Required Information

Describe the Bug:

On file:// and memory://, FsspecArtifactService (added with write support in df9f9d0) can't keep an artifact apart from a nested artifact whose last segment is a number, such as report and report/0. Versions are stored at {artifact_path}/{version}, so .../report/0 has to be both the file holding version 0 of report and the directory holding the versions of report/0. A hierarchical filesystem can't have both, while GCS can, because its keys are flat.

The result depends on which one is saved first:

  1. report, then report/0: the second save raises a raw FileExistsError. On file:// it comes from makedirs, which runs before the retry loop. On memory:// the error is a FileExistsError too, so the version retry loop tries 10 times on a key that can never succeed and then raises.
  2. report/0, then report: report/0 exists as a directory, so the exclusive create of report/0 raises FileExistsError. The loop treats that as a concurrent save and moves on to version 1. The save succeeds, but report has no version 0: save_artifact returns 1 and load_artifact(version=0) returns None. In the other services the first save of a new artifact is version 0.

The services support nested names (test_nested_artifact_does_not_leak_versions_into_parent and the other nested tests in tests/unittests/artifacts/test_artifact_service.py cover doc and doc/nested). The FsspecArtifactService docstring doesn't mention this limitation.

Steps to Reproduce:

  1. pip install "google-adk[fsspec]" (or a checkout of main)
  2. Run the script below with ADK_ENABLE_FSSPEC_ARTIFACT_SERVICE=1

Expected Behavior:

Either the same results as InMemoryArtifactService (both saves work, and each artifact starts at version 0), or, if the layout can't support it, a clear InputValidationError naming the conflicting artifact. A save should not quietly give the artifact a different version number.

Observed Behavior:

Case 1: save 'report' then 'report/0'
  InMemory           save('report/0') -> 0
  fsspec memory://   save('report/0') -> FileExistsError: /c1/app/u/s/report/0
  fsspec file://     save('report/0') -> FileExistsError: [Errno 17] File exists: '/tmp/.../c1/app/u/s/report/0'
Case 2: save 'report/0' then 'report'
  InMemory           returned version=0 list_versions=[0] load(version=0)=b'r'
  fsspec memory://   returned version=1 list_versions=[1] load(version=0)=None
  fsspec file://     returned version=1 list_versions=[1] load(version=0)=None

Environment Details:

  • ADK Library Version: main at d05e482
  • Desktop OS: macOS
  • Python Version: 3.12.13 (fsspec 2026.9.0)

Model Information:

  • Are you using LiteLLM: N/A
  • Which model is being used: N/A (artifact service only)

🟡 Optional Information

Regression:

No. Writes on fsspec are new in df9f9d0.

Minimal Reproduction Code:

import asyncio, tempfile
from google.genai import types
from google.adk.artifacts.in_memory_artifact_service import InMemoryArtifactService
from google.adk.integrations.fsspec import FsspecArtifactService

tmp = tempfile.mkdtemp()
A = dict(app_name="app", user_id="u", session_id="s")

def blob(b):
  return types.Part(inline_data=types.Blob(data=b, mime_type="application/pdf"))

def services(tag):
  return [
      ("InMemory", InMemoryArtifactService()),
      ("fsspec memory://", FsspecArtifactService(f"memory://{tag}")),
      ("fsspec file://", FsspecArtifactService(f"file://{tmp}/{tag}")),
  ]

async def main():
  print("Case 1: save 'report' then 'report/0'")
  for name, svc in services("c1"):
    await svc.save_artifact(**A, filename="report", artifact=blob(b"r"))
    try:
      result = await svc.save_artifact(**A, filename="report/0", artifact=blob(b"n"))
    except Exception as e:
      result = f"{type(e).__name__}: {e}"
    print(f"  {name:18s} save('report/0') -> {result}")

  print("Case 2: save 'report/0' then 'report'")
  for name, svc in services("c2"):
    await svc.save_artifact(**A, filename="report/0", artifact=blob(b"n"))
    v = await svc.save_artifact(**A, filename="report", artifact=blob(b"r"))
    versions = await svc.list_versions(**A, filename="report")
    v0 = await svc.load_artifact(**A, filename="report", version=0)
    print(f"  {name:18s} returned version={v} list_versions={versions}"
          f" load(version=0)={v0 and v0.inline_data.data}")

asyncio.run(main())

Additional Context:

Since the layout matches GcsArtifactService on purpose, I don't think changing it is the answer. A smaller fix could be, on non-GCS backends:

  • in the retry loop, check whether the key that raised FileExistsError is a directory. If it is, raise an InputValidationError saying the name conflicts with a nested artifact, instead of moving to the next version;
  • catch the makedirs / parent-is-a-file failure on the nested side and raise the same error;
  • add a line to the class docstring saying that on hierarchical filesystems an artifact can't have a nested artifact whose next segment is a number.

I'm happy to send a PR along these lines, or something different if you'd rather handle it another way.

How often has this issue occurred?:

  • Always (100%)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions