How to submit results to DUE leaderboard?

[Cppowboy]

I am trying to submit our results to DUE leaderboard, what's the submission format? And is there any submission samples?

[Borchmann]

The submission format follows the format of the document.jsonl in du-schema. Therefore, you can take a look at dev set examples from du-schema/examples or complete datasets available at https://duebenchmark.com/data as a reference.

It is expected to upload multiple files after logging in at duebenchmark.com. The name of the file is irrelevant, and the system matches the output files with the dataset automatically based on their content. We calculate the mean and standard deviation if multiple files for the same dataset are provided. All you have to do is to keep the same line order as in the corresponding test set document.jsonl file.

In case of any doubts or problems with processing the files, please do not hesitate to contact me with the output file attached, either here or at lukasz.borchmann@applica.ai.