zarr compression for all data_vars in Dataset #5798
Unanswered
raybellwaves
asked this question in
Ideas
Replies: 1 comment
-
A syntactic sugar for what you wrote is: ds.to_zarr(
*args,
encoding={
dv: {"compressor": compressor}
for dv in ds.data_vars
},
) |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Looking at http://xarray.pydata.org/en/stable/user-guide/io.html#zarr-compressors-and-filters compression is applied through the encoding arg which expects a dictionary with keys as data_vars. If a user just has a dataset of numerical DataArray's (float32) it would be nice to able to specify a general basis as opposed to per data_var. This could be a compression arg but maybe can't be used with the encoding arg with the assumption the user doesn't want to encode other things e.g. mask and scale.
ds.to_zarr("ds.zarr", compression=compression)
Here's a util which does this.
Not sure how this would would with filters as the current encoding arg gives the best flexibility to use compression and filters. The encoding arg is also across other API e.g. to_netcdf.
Beta Was this translation helpful? Give feedback.
All reactions