API docs

Core data model

Key([offsets, vars])

A key for keeping track of chunks of a distributed xarray.Dataset.

Reading and writing data

open_zarr(store, **kwargs)

Returns a lazily indexable xarray.Dataset and chunks from a Zarr store.

DatasetToChunks(dataset[, chunks, ...])

Split one or more xarray.Datasets into keyed chunks.

ChunksToZarr(store[, template, zarr_chunks, ...])

Write keyed chunks to a Zarr store in parallel.

DatasetToZarr(dataset, store[, zarr_chunks])

Write an entire xarray.Dataset to a Zarr store.

make_template(dataset[, lazy_vars])

Make a lazy Dask xarray.Dataset for use only as a template.


Mean(dim[, skipna, dtype, fanout])

Calculate the mean over one or more distributed dataset dimensions.

Mean.Globally([dim, skipna, dtype, fanout])

Calculate global mean over a pcollection of xarray.Dataset objects.

Mean.PerKey([dim, skipna, dtype, fanout])

Calculate per-key mean over a pcollection of (hashable, Dataset).

MeanCombineFn([dim, skipna, dtype])

CombineFn for computing an arithmetic mean of xarray.Dataset objects.



Consolidate existing chunks across offsets into bigger chunks.


Consolidate existing chunks across variables into bigger chunks.


Split existing chunks into smaller chunks.


Split existing chunks into a separate chunk per data variable.

Rechunk(dim_sizes, source_chunks, ...[, ...])

Rechunk to an arbitrary new chunking scheme with bounded memory usage.

Utility transforms


Check that keys match the dataset for each key, dataset tuple.

Utility functions

offsets_to_slices(offsets, sizes[, base])

Convert offsets into slices with an optional base offset.

validate_chunk(key, datasets)

Verify that keys correspond to Dataset properties.

consolidate_chunks(inputs[, combine_kwargs])

Consolidate chunks across offsets into (Key, Dataset) pairs.

consolidate_variables(inputs[, merge_kwargs])

Consolidate chunks across distinct variables into (Key, Dataset) pairs.

consolidate_fully(inputs, *[, merge_kwargs, ...])

Consolidate chunks via merge/concat into a single (Key, Dataset) pair.

split_chunks(key, dataset, target_chunks)

Split a single (Key, xarray.Dataset) pair into many chunks.

split_variables(key, dataset)

Split a single (Key, xarray.Dataset) pair into separate variables.

in_memory_rechunk(inputs, target_chunks)

Rechunk in-memory pairs of (Key, xarray.Dataset).