comdas.arrays.duck_array¶

Implements a NumPy-like duck array for compressed Patch data.

Classes

DuckArray(payload, codec, *[, ...])

A NumPy-like duck array backed by a compressed payload and codec defining how the data is compressed and decompressed.

class comdas.arrays.duck_array.DuckArray(payload, codec, *, max_pending_writes=None)¶

Bases: NDArrayOperatorsMixin

A NumPy-like duck array backed by a compressed payload and codec defining how the data is compressed and decompressed.

Reads are performed via codec.decode_partial. Writes go into a sparse overlay dict rather than immediately forcing a complete recompression of the underlying compressed payload. consolidate() can be called to merge the overlay into the compressed payload. consolidate() will be called automatically when the number of pending writes exceeds the codec’s specified maximum number of pending writes.

If the specified codec implements partial writes efficiently, the codec’s partial writes function will always be used to update the compressed payload without accumulation.

Parameters:

payload (CompressedPayload) – The compressed payload backing this duck array.
codec (Codec) – The codec defining how the data is compressed and decompressed.
max_pending_writes (int | None) – Maximum number of pending writes before automatic consolidation.

property shape: tuple[int, ...]¶

Get the shape of the decompressed/original array.

Return type:: tuple[int, …]

property dtype: dtype¶

Get the data type of the decompressed/original array.

Return type:: numpy.dtype

property ndim: int¶

Get the number of dimensions of the decompressed/original array.

Return type:: int

property payload: CompressedPayload¶

Get the current compressed payload backing this array.

Return type:: CompressedPayload

property codec: Codec¶

Get the codec used to encode/decode payload.

Return type:: Codec

property overlay_size: int¶

Get the number of elements currently held in the write overlay.

Return type:: int

consolidate(*, dense=None, **encode_kwargs)¶

Fold the write overlay into a freshly-compressed payload.

By default, re-encodes using whatever compression level the current payload already used.

Parameters:

dense (ndarray | None) – Precomputed dense array to encode, if the caller already has one.
encode_kwargs – Additional codec-specific kwargs forwarded to codec.encode.

Return type:

None