Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable DataTree.to_zarr(compute=False) #9958

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ New Features
~~~~~~~~~~~~
- Relax nanosecond datetime restriction in CF time decoding (:issue:`7493`, :pull:`9618`).
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_ and `Spencer Clark <https://github.com/spencerkclark>`_.
- Enable the ``compute=False`` option in :py:meth:`DataTree.to_zarr`. (:pull:`9958`).
By `Sam Levang <https://github.com/slevang>`_.

Breaking changes
~~~~~~~~~~~~~~~~
Expand Down
6 changes: 2 additions & 4 deletions xarray/core/datatree_io.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ def _datatree_to_zarr(
consolidated: bool = True,
group: str | None = None,
write_inherited_coords: bool = False,
compute: Literal[True] = True,
compute: bool = True,
**kwargs,
):
"""This function creates an appropriate datastore for writing a datatree
Expand All @@ -100,9 +100,6 @@ def _datatree_to_zarr(
"specifying a root group for the tree has not been implemented"
)

if not compute:
raise NotImplementedError("compute=False has not been implemented yet")

if encoding is None:
encoding = {}

Expand All @@ -124,6 +121,7 @@ def _datatree_to_zarr(
mode=mode,
encoding=encoding.get(node.path),
consolidated=False,
compute=compute,
**kwargs,
)
if "w" in mode:
Expand Down
19 changes: 19 additions & 0 deletions xarray/tests/test_backends_datatree.py
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,25 @@ def test_to_zarr_default_write_mode(self, tmpdir, simple_datatree):
with pytest.raises(zarr.errors.ContainsGroupError):
simple_datatree.to_zarr(tmpdir)

@requires_dask
def test_to_zarr_compute_false(self, tmpdir, simple_datatree):
import dask.array as da

filepath = tmpdir / "test.zarr"
original_dt = simple_datatree.chunk()
original_dt.to_zarr(filepath, compute=False)

for node in original_dt.subtree:
for name, variable in node.dataset.variables.items():
var_dir = filepath / node.path / name
var_files = var_dir.listdir()
assert var_dir / ".zarray" in var_files
assert var_dir / ".zattrs" in var_files
if isinstance(variable.data, da.Array):
assert var_dir / "0" not in var_files
else:
assert var_dir / "0" in var_files

def test_to_zarr_inherited_coords(self, tmpdir):
original_dt = DataTree.from_dict(
{
Expand Down
Loading