Column ID 175, 176, 177, 182, 215, 475, 478, 481, 503 unknown

[nickbrady]

On different .mpr files I get the following errors when trying to use: mpr_f = BioLogic.MPRfile(f)

Column ID 182 after column Capacitance discharge/µF is unknown
Column ID 215 after column (Q-Qo)/mA.h is unknown
Column ID 475 after column |I h7|/A is unknown

Fiddling around in Biologic.py, I was able to determine the appropriate buffer sizes (maybe)

#    175: ("ColID_175", "<f4"),
#    176: ("ColID_176", "<f4"),
#    177: ("ColID_177", "<f4"),
#    182: ("ColID_182", "<f8"), #<f4 doesn't work
#    215: ("ColID_215", "<f4"),
#    475: ("ColID_475", "<f4"),
#    478: ("ColID_478", "<f4"),
#    481: ("ColID_481", "<f4"),
#    503: ("ColID_503", "<u4"), #<u1, <u4, <f4, <f8 don't work

I don't have the corresponding .mpt files yet, but perhaps I can get them to identify the column headers.

Used pip install git+https://github.com/echemdata/galvani
`pip show galvani' gives Version: 0.4.1 - which appears to be the latest version.

[ml-evs]

Could you open these files in ECLab and figure out what the column labels are please?

[nickbrady]

Going through the keys found in VMPdata_colID_dtype_map, VMPdata_colID_flag_map, and VMPdata_colID_dtype_map, these are the column headers in my .mpt files that are not in those dictionaries:

['Re(Y)/Ohm-1',
 'Im(Y)/Ohm-1',
 '|Y|/Ohm-1',
 'Phase(Y)/deg',
 'Re(C)/nF',
 'Im(C)/nF',
 '|C|/nF',
 'Phase(C)/deg',
 'Phase(Permittivity)/deg',
 'Re(Resistivity)/Ohm.cm',
 'Im(Resistivity)/Ohm.cm',
 '|Resistivity|/Ohm.cm',
 'Phase(Resistivity)/deg',
 'Re(Conductivity)/mS/cm',
 'Im(Conductivity)/mS/cm',
 '|Conductivity|/mS/cm',
 'Phase(Conductivity)/deg',
 'Loss Angle(Delta)/deg',
 'x',
 'THD Ece/%',
 'NSD Ece/%',
 'NSR Ece/%',
 '|Ece h2|/V',
 '|Ece h3|/V',
 '|Ece h4|/V',
 '|Ece h5|/V',
 '|Ece h6|/V',
 '|Ece h7|/V',
]

My guesses:
175:
176:
177:
182:
215:
475: THD Ece/%
478: NSD Ece/%
481: NSR Ece/%
503:

475 - 481 - these seem to be consistent with the other columns in this range

reading the warnings during import:

Column ID 503 after column Capacity/mA.h is unknown (Capacity/mA.h does not appear in the .mpt file)

Column ID 215 after column (Q-Qo)/mA.h is unknown (--> <Ece>/V)

Column ID 175 after column <Ewe>/V 2 is unknown (--> <I>/mA)

176: Cs/μF	
177: Cp/μF

But these don't seem to make sense.

I haven't opened these in ECLab. If I do that, does it explicitly say Col 175 --> '|Ece h6|/V'?

[ml-evs]

Interesting, thanks for checking this out. Unfortunately we never directly get the map from them, so it might be a case of playing around and plotting columns to see if they match.

Can I ask what kind of experiment this was for, which instruments, and which ECLab version was controlling it? The ECLab mpr format changed significantly recently and I just want to check we're talking about the same thing.

[fg-personal]

I realized I had the same problem

    175: ("Idk","<f4"), #To fix looking at data from EIS
    176: ("Idk","<f4"), #To fix looking at data from EIS
    177: ("Idk","<f4"), #To fix looking at data from EIS
    182: ("Idk","<f8"), #To fix looking at data from EIS
    185: ("Idk","<f4"), #To fix looking at data from CV
    215: ("Idk", "<u4"), #To fix looking at data from EIS

185 is new.
The files I had problem with were generated in EC-Lab v11.52

[Roy027]

I have same issue here.

> `---------------------------------------------------------------------------
> NotImplementedError                       Traceback (most recent call last)
> Cell In[1], line 8
>       4 import os, glob
>       6 files = glob.glob(os.path.join('','*.mpr'))
> ----> 8 mpr_file =  BioLogic.MPRfile(files[0])
> 
> File XXX\EnvGeneral\lib\site-packages\galvani\BioLogic.py:598, in MPRfile.__init__(self, file_or_path)
>     [592](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=591)     raise ValueError(
>     [593](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=592)         "Unrecognised version for data module: %d" % data_module["version"]
>     [594](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=593)     )
>     [596](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=595) assert not any(remaining_headers)
> --> [598](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=597) self.dtype, self.flags_dict = VMPdata_dtype_from_colIDs(column_types)
>     [599](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=598) self.data = np.frombuffer(main_data, dtype=self.dtype)
>     [600](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=599) assert self.data.shape[0] == n_data_points
> 
> File c:\XXX\Python39\EnvGeneral\lib\site-packages\galvani\BioLogic.py:471, in VMPdata_dtype_from_colIDs(colIDs)
>     [469](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=468)         type_list.append((unique_field_name, field_type))
>     [470](XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=469)     else:
> --> [471](file:///cXXX/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=470)         raise NotImplementedError(
>     [472](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=471)             "Column ID {cid} after column {prev} "
>     [473](file://XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=472)             "is unknown".format(cid=colID, prev=type_list[-1][0])
>     [474](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=473)         )
>     [475](file:///XXX/Python39/EnvGeneral/lib/site-packages/galvani/BioLogic.py?line=474) return np.dtype(type_list), flags_dict
> 
> NotImplementedError: Column ID 182 after column Capacitance discharge/µF is unknown`

[joel-kelly]

I have the same issue, too. Is there anything I can help to solve this issue?
Galvani solves a really common pain point for dealing with Biologic data, but it seems like Biologic doesn't play very nicely with the idea of open data...

• Galvani 0.4.1 installed via pip
• Data file collected from a VMP300 running bt-lab 11.50

`NotImplementedError Traceback (most recent call last)
Cell In[3], line 1
----> 1 mpr_file = BioLogic.MPRfile(r"F:\dept\celldev\Treq Results\9401-9500\9455 - Fast Charger Improvement\Data\Coin Cell\Symmetric cell rate map\23oC\RZ1-241114-001-24112201_75%SOC_23degC_rm_03_GCPL_C07.mpr"
2 )
4 pd.DataFrame(mpr_file.data)

File c:\ProgramData\miniforge3\envs\eis\Lib\site-packages\galvani\BioLogic.py:598, in MPRfile.init(self, file_or_path)
592 raise ValueError(
593 "Unrecognised version for data module: %d" % data_module["version"]
594 )
596 assert not any(remaining_headers)
--> 598 self.dtype, self.flags_dict = VMPdata_dtype_from_colIDs(column_types)
599 self.data = np.frombuffer(main_data, dtype=self.dtype)
600 assert self.data.shape[0] == n_data_points

File c:\ProgramData\miniforge3\envs\eis\Lib\site-packages\galvani\BioLogic.py:471, in VMPdata_dtype_from_colIDs(colIDs)
469 type_list.append((unique_field_name, field_type))
470 else:
--> 471 raise NotImplementedError(
472 "Column ID {cid} after column {prev} "
473 "is unknown".format(cid=colID, prev=type_list[-1][0])
474 )
475 return np.dtype(type_list), flags_dict

NotImplementedError: Column ID 182 after column Capacitance discharge/µF is unknown`

[ml-evs]

Just generally sharing the smallest possible files that break for you is helpful -- you can either upload in a comment here or share with me via email (should be easy enough to find).

No-one is really actively developing galvani at the moment and I can only justify spending the time if something breaks for the labs I work with, so generally developer time is what is needed! There's a few other libraries that read MPR too, which might be helpful, e.g., yadg, but its really not easy to keep up with the myriad ways that BioLogic break the format between EC-lab releases.

[joel-kelly]

Thanks Matthew!

Something weird has happened on our end-- after checking various files to look for the NotImplementedError, the issue seems to have resolved itself!

At first I thought it was something related to the length of the filepath string, but in fact my original files are now successfully parsing. We tried this on a few different Windows computers with different python environments, and all work. Very weird! I think the common theme across all would be that the galvani package was recently upgraded to 0.4.1.

Does galvani have some sort of caching that could explain this?

[AShnier]

Hi I just had a similar issue for column ID 182. Looking at my mpt, 182 seems to correspond "step time/s". Similar to nickbrady I added
182: ("step time/s", "<f8"),
to
VMPdata_colID_dtype_map = {} in the BioLogic.py file.

The mpr file came from a multistep CA measurement

EC-Lab: v11.62
Galvani: 0.4.1

[nickbrady]

I came across another unknown column

112: ("colID_112", "<f8"),

I wanted to ask, is it reasonable to make a pull request that includes the unknown column ID's. For me at least, often the information in the missing column is not critical for analysis, but the error will prevent the data from being imported.

[ml-evs]

I came across another unknown column

112: ("colID_112", "<f8"),

I wanted to ask, is it reasonable to make a pull request that includes the unknown column ID's. For me at least, often the information in the missing column is not critical for analysis, but the error will prevent the data from being imported.

See my comment here: #125 (comment) I added a mode (now merged) that will hopefully stop the unknown column errors by doing pretty much this in the general case (and trying to iteratively deduce the dtype too). It's obviously still beneficial if we can figure out what these columns are!