Solution evaluation not consistent with sandbox dataset

[Robbybp]

From running python check_data.py --help, I understand that checking solution files is not fully supported yet, but until recently this was working with some simple solution files constructed from the sandbox dataset. After some recent commits to the main branch, I get an error when running the following command:

$ python check_data.py -p S0/D1/C3S0N00003/scenario_002.json -s solution.json
...
Traceback (most recent call last):
  File "C3DataUtilities/check_data.py", line 70, in <module>
    validation.check_data(problem, args.solution, args.configuration, args.summary_csv, args.summary_json, args.data_errors, args.ignored_errors, args.solution_errors)
  File "C3DataUtilities/datautilities/validation.py", line 776, in check_data
    problem_data_array.set_from_data_model(data_model)
  File "C3DataUtilities/datautilities/arraydata.py", line 12, in set_from_data_model
    self.set_scalars(data)
  File "C3DataUtilities/datautilities/arraydata.py", line 145, in set_scalars
    self.c_e = float(data.network.violation_cost.e_vio_cost)
AttributeError: 'ViolationCostsParameters' object has no attribute 'e_vio_cost'

This is with GO-3-data-model v1.0.0.

The error is that the input data file does not have an e_vio_cost attribute. It seems none of the input files in the sandbox dataset have this parameter, but recently this repo was updated to require it for checking solution files. Two possible solutions are:

1. A new version of the sandbox dataset is released that contains this e_vio_cost parameter
2. A tag or release is added to this repo at the last commit that can evaluate solutions using the sandbox dataset input files until a new version of the dataset is released. In some brief testing, this appears to be commit 50c8e81

Let me know if I've missed something and there is some other way to run the evaluator using these input files.

[ccoffrin]

Good point @Robbybp. I don't recall the timeline for getting the released datasets up the this latest data spec. but in the mean time reasonable work arounds are,

1. Use an older commit of this code, as you suggest.
2. Update a data set by hand, adding the e_vio_cost in the suitable location in the data is an easy modification to make.

@jesseholzerpnnl, I don't think we are expecting any further updated to the data format at this time, but I would suggest a best practice of keep updates to this repo in a separate branch and merge that branch into master at the same time revised datasets are released.

[jesseholzerpnnl]

Hi @Robbybp and @ccoffrin. Good points on all counts. At this point, the most realistic way to go is to add e_vio_cost manually.

[jesseholzerpnnl]

@Robbybp, checking solutions is fully supported. The help message might be out of date. I'll get that fixed.

[Robbybp]

@jesseholzerpnnl @ccoffrin Thanks for the input. My current workaround is Robbybp@f1c2cc5

[jesseholzerpnnl]

We now have a dev branch and a main branch. The dev branch will have all the features that are known to be working. The main branch will have only those features that work with the current public data. There may be other branches as needed.

[Robbybp]

Thanks for the update. Has the main branch been updated for consistency with the public data, or should I continue to use (and update) my patched branch until a new dataset is released?

[jesseholzerpnnl]

Thanks for the update. Has the main branch been updated for consistency with the public data, or should I continue to use (and update) my patched branch until a new dataset is released?

Please continue to use your patched branch until a new dataset is released.