Logger cannot merge runs in output data when called out of context #39
Description
I'm interfacing my solver with IOHexperimenter at the problem level, because I wish to use my own experiment manager. Indeed, I wish to use my own batch system to place (down to) single runs as independent jobs on an arbitrary set of machines. However, this is not feasible with the current architecture.
The core of the problem is that the Logger interface expect to be initialized once, and then to be called on different target_problem, for several runs. However, if one wish to handle jobs at the runs level, the Logger will be initialized for each run. In that setting, the Logger avoid adding the results in the previously written data folder, and create an additional folder, leading to two different folders holding two separated runs.
Unfortunately, this data structure cannot be loaded as a single experiment by IOHanalyzer.
To test the problem, merge PR #38 and run t-bbob-csvlogger twice.
The solutions I can imagine are to:
- add a state manager to Logger that would allow to load previous data structures from saved files,
- allow IOHanalyzer to parse several folders as a single experiment,
- handle this specific case with an additional data merger.