comdas.io.h5_container

FiberIO plugin for the ComDAS HDF5 container format.

The ComDAS HDF5 container format is designed to store patches in a codec-agnostic manner

Registers the COMDAS format with DASCore’s FiberIO manager. Once this module is imported (or the package is pip-installed, via the dascore.fiber_io entry point in pyproject.toml), dc.spool(path) and dc.write(patches, path, file_format="COMDAS") work like any other DASCore format regardless of which Codec was used to compress each patch.

File layout

/                                       (root)
  attrs:
    __format__ = "COMDAS"
    __COMDAS_version__ = "1"
/patches
  /patch_0000
    attrs:
      codec_name         (str)   -- e.g. "SVD"; looked up in the codec registry
      codec_version      (str)   -- the codec's own version string
      compression_params (dict)  -- parameters used for compression
      original_shape     (tuple[int, ...])
      dtype              (str)   -- original array dtype
      dims               (str)   -- comma-separated dim names, in order
      attrs_json         (str)   -- PatchAttrs.model_dump_json()
    <codec-specific datasets/attrs, written by codec.payload_to_group>
    /coords
      <dim_name>              -- one dataset per dim, full coord array
        attrs: dtype (str), units (str, empty if none)
  /patch_0001
    ...

Everything above the codec-specific datasets is written and read by this module. Only what’s inside a patch’s own group (beyond coords) is delegated to codec.payload_to_group / codec.payload_from_group.

Patches are appended, not overwritten: calling write again on an existing file adds new patch_XXXX groups after the existing ones (H5Writer opens in "a" mode).

Once comdas is installed, plain dc.spool(path) finds and uses this format automatically. Writing has one convenience function, write_compressed().

Functions

write_compressed(source, path, codec, ...)

Compress source and write it to path as a COMDAS container.

Classes

ComdasV1()

FiberIO support for the codec-agnostic ComDAS HDF5 container, v1.
class comdas.io.h5_container.ComdasV1

Bases: FiberIO

FiberIO support for the codec-agnostic ComDAS HDF5 container, v1.

Each patch stores which Codec it was compressed with. This means a single file can hold patches compressed with different codecs.

get_format(resource, **kwargs)

Identify a COMDAS file via its root-level magic attrs.

Parameters:

  • resource (H5Reader) – An open, readable HDF5 file (auto-opened by DASCore from a path via the H5Reader type hint).

  • kwargs – Unused; accepted for FiberIO API compatibility.

Returns:

(format_name, format_version) if this file is a COMDAS container, else False.

Return type:

tuple[str, str] | bool

scan(resource, **kwargs)

Read every patch’s attrs without touching datasets.

Parameters:

  • resource (H5Reader) – An open, readable COMDAS HDF5 file.

  • kwargs – Unused; accepted for FiberIO API compatibility.

Returns:

One dascore.PatchAttrs per stored patch.

Return type:

list[PatchAttrs]

read(resource, **kwargs)

Read patches.

Honors dimension-range kwargs.

Parameters:

  • resource (H5Reader) – An open, readable COMDAS HDF5 file.

  • kwargs – Optional dimension-range constraints (e.g. time=(t1, t2)); any key not matching a coord’s dim name is ignored.

Returns:

A spool of the matching patches.

Return type:

BaseSpool

write(spool, resource, *, codec=None, **encode_kwargs)

Compress and write a Patch/spool of Patches to the container.

Parameters:

  • spool (dascore.Patch or dascore.BaseSpool or Sequence[dascore.Patch]) – A single Patch, or any iterable of Patches, to write.

  • resource (H5Writer) – An open, writable HDF5 file (auto-opened by DASCore from a path via the H5Writer type hint, in append mode).

  • codec (Codec | None) – The codec to compress every patch in spool with. (e.g. SVDCodec(rank=20).)

  • encode_kwargs – Forwarded to codec.encode for every patch.

Raises:

ValueError – If codec is not given.

Return type:

None

comdas.io.h5_container.write_compressed(source, path, codec, **encode_kwargs)

Compress source and write it to path as a COMDAS container.

This is the intended way to turn uncompressed DAS data into a compressed COMDAS file.

Parameters:

  • source (str or pathlib.Path or dascore.Patch or dascore.BaseSpool or Sequence[dascore.Patch]) – Where the uncompressed data comes from (a file or directory path, a single Patch, a sequence of Patches, or an existing Spool).

  • path (str or pathlib.Path) – Destination path for the COMDAS container. If it already exists as a COMDAS file, patches are appended rather than overwriting existing ones.

  • codec (Codec) – The codec to compress every patch with, e.g. SVDCodec(rank=20).

  • encode_kwargs – Forwarded to codec.encode for every patch (e.g. rank=/energy= for SVDCodec, overriding the codec’s own defaults for this call).

Example:

import dascore as dc
from comdas import write_compressed, SVDCodec

# from an existing uncompressed file
write_compressed("raw.h5", "compressed.h5", SVDCodec(rank=20))

# from a patch already in memory
patch = dc.get_example_patch()
write_compressed(patch, "compressed.h5", SVDCodec(rank=20))

# reading back needs nothing ComDAS-specific
read_back = dc.spool("compressed.h5")
Return type:

None