Home

The goal of the MKID Data Format collaboration is to develop an interoperable format for MKID observing data. This format should permit efficient processing of large amounts of data.

What follows below is my (Aled) recollection of things proposed during the meeting at LTD2025.

It was generally agreed upon that the format should support some level of extensibility determined through an RFC like process. Parsing however should be strict.

It was also agreed upon that the main performance critical aspect is managing the photon tables, and it was agreed upon that some effort should be spent on benchmarking this. The Mazin Group can provide large observing datasets to enable bench marking.

Information that needs to be captured:

• Metadata for the observing run/experiment ie Ra/Dec for astronomy, microscope info for bio imaging, etc.
• Photon Tables (Arrival time, some proxy statistic for energy, Resonator ID)
• Raw IQ/Phase captures from the device
• Beam maps
• Wavelength Cals
• VNA Sweeps
• Resonator Setup Info (Tone frequencies, powers)
• Output Science Products (Image cubes)

Proposed formats:

• Metadata
  • SQLite
  • JSON With Schema
  • DAG-CBOR
  • Protobuf
• Photon Tables
  • Parquet/Arrow
  • Numpy arrays
  • HDF5 < Nobody likes this option
  • ASDF Maybe?