channel nomenclature failing on mt_metadata main #385
Description
This is blocking passing tests on aurora features branch. It seems to be rooted in a change from mt_metadata's recently merged patches ... was passing on mt_metadata features.
@kujaku11 probably need to re-review the merge conflicts
pytest test_processing.py -k test_can_use_channel_nomenclature
2025-07-14T08:22:41.473137-0700 | ERROR | aurora.pipelines.process_mth5 | process_mth5_legacy | TF collection could not export to mt_metadata TransferFunction
Failed with exception write() got an unexpected keyword argument 'format'
Perhaps an unconventional mixture of input/output channels was used
Input channels were ['bx', 'by']
Output channels were ['e1', 'e2', 'bz']
No z-file will be written in this case
Will return a legacy TransferFunctionCollection object, not mt_metadata object.
2025-07-14T08:22:41.473464-0700 | INFO | mth5.mth5 | close_mth5 | Flushing and closing /home/kkappler/software/irismt/mth5/mth5/data/mth5/test1_LEMI12.h5
========================================================================== warnings summary ===========================================================================
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================= short test summary info =======================================================================
FAILED test_processing.py::TestSyntheticProcessing::test_can_use_channel_nomenclature - AttributeError: 'TransferFunctionCollection' object has no attribute 'write'
Seems that the problem is here; the tf_cls is created successfully (although it may be different than it used to be) but it fails to write to z_file
try:
tf_cls = tfk.export_tf_collection(tf_collection)
if z_file_path:
tf_cls.write(z_file_path)
except:
...
return collection = True
That non-writing to z-file actually doesn't make the test fail, what makes it fail is that the tf_collection (and not the TF object) is returned. So this could possibly be worked around by ignoring failure to write the z_file, but the question is, what happened to the z-file writing method to make it fail?
It looks like the TF object that gets initialized has channels ex, ey, instead of e1, e2 etc. Could it be that a change in TF has triggered this?
Update after debugging
- what triggered the change is still not clear, but a solution was to properly embed the station metadata into the TF object. By accessing the station object metadata from the TFKernel, and writing it to the TF I was able to pass the failing test. Alas, then other tests did not pass ... these seem to be due to this
ERROR | mt_metadata.transfer_functions.core | _validate_station_metadata | input metadata must be type <class 'mt_metadata.transfer_functions.tf.station.Station'> or dict, not <class 'mt_metadata.timeseries.station.Station'>
- Note that the z-files only output with default channel nomenclature (Ex, Ey, Hx, Hy, Hz), even when they are output from a TF object with a different labelling scheme, such as e1, e2, bx, by, bz.
The solution seems to be this:
# TODO: this should be done inside the TF initialization
for i_run, run in enumerate(tf_cls.station_metadata.runs):
for i_ch, channel in enumerate(run.channels):
new_ch = channel.copy()
default_component = channel.component
new_component = channel_nomenclature_dict[default_component]
new_ch.component = new_component
tf_cls.station_metadata.runs[i_run].remove_channel(default_component)
tf_cls.station_metadata.runs[i_run].add_channel(new_ch)
-
TODO:
-
decide whether the TF class in mt_metadata will support channel_nomenclature, and add a note to documentation.
If we decide to use the default nomenclature across the board then we can simply comment out the line ch.component = self.inverse_channel_nomenclature[comp] in tf core.py around line 2209 and the test will pass. If we decide that the TF object should serve back the nomenclature that was used in the time-series archive, then we need to address why tf object not receiving station metadata