xarray_beam.Key

class xarray_beam.Key(offsets=None, vars=None)

A key for keeping track of chunks of a distributed xarray.Dataset.

Key object in Xarray-Beam include two components:

  • “offsets”: an immutable dict indicating integer offsets (total number of array elements) from the origin along each dimension for this chunk.

  • “vars”: either an frozenset or None, indicating the subset of Dataset variables included in this chunk. None means that all variables are included.

Key objects are “deterministically encoded” by Beam, which makes them suitable for use as keys in Beam pipelines, i.e., with beam.GroupByKey. They are also immutable and hashable, which makes them usable as keys in Python dictionaries.

Example usage:

>>> key = xarray_beam.Key(offsets={'x': 10}, vars={'foo'})

>>> key
xarray_beam.Key(offsets={'x': 10}, vars={'foo'})

>>> key.offsets
immutabledict({'x': 10})

>>> key.vars
frozenset({'foo'})

To replace some offsets:

>>> key.with_offsets(y=0)  # insert
xarray_beam.Key(offsets={'x': 10, 'y': 0}, vars={'foo'})

>>> key.with_offsets(x=20)  # override
xarray_beam.Key(offsets={'x': 20}, vars={'foo'})

>>> key.with_offsets(x=None)  # remove
xarray_beam.Key(offsets={}, vars={'foo'})

To entirely replace offsets or variables:

>>> key.replace(offsets={'y': 0})
xarray_beam.Key(offsets={'y': 0}, vars={'foo'})

>>> key.replace(vars=None)
xarray_beam.Key(offsets={'x': 10}, vars=None)
Parameters:

  • offsets (Optional[Mapping[str, int]]) –

  • vars (Optional[AbstractSet[str]]) –

__init__(offsets=None, vars=None)
Parameters:

  • offsets (Optional[Mapping[str, int]]) –

  • vars (Optional[AbstractSet[str]]) –

Methods

__init__([offsets, vars])

replace([offsets, vars])

with_offsets(**offsets)