Questions about training scripts

[passing2961]

Hello,

I have two questions:

1. Would you be able to share the scripts for the Kernel, Algorithm, and Biology tasks that are presented in the main table of Figure 1? Do they share all same configurations?
2. In Table 2, it seems that you ran OpenEvolve with GPT-OSS-120B. Could you share the detailed configuration you used—for example, the max iterations, temperature, max_tokens, number of islands, and any other relevant settings?

[passing2961]

@mertyg Thank you for the updated codes. However, I couldn't find training scripts for each task (e.g., scripts/thinker/ac1.sh).

[vinid]

for openevolve:

configuration is the default configuration provided in openevolve for math tasks (4 islands, 60 population size...). We use the same initial program provided by openevolve as a starting point.

The only two very relevant differences are max_iterations: 25600 and parallel_evaluations: 512.

In addition to the above, note that we had to implement cpu pinning and assign two CPUs to each parallel evaluation (briefly mentioned this in #6).

[koceja]

All of the config and code for running those tasks should be contained to the env.py for each example, which you should be able to run as a simple python script. Please take a look at reproducing.md for instructions on how to reproduce results.

For multihost runs (useful for slurm across multiple nodes), please take a look at the circle packing example that uses our submitit slurm script.

I hope this helps!

[passing2961]

@koceja After training, did you use "env/all/raw_score" value as a final performance (i.e., reported performance in Table 1 in your paper)?

[koceja]

The raw score is supposed to be any main metric that you are monitoring. For many applications it is the final performance but it is not necessarily the case as in denoising.