Hi, thanks for releasing ColorBench.
I have two questions:
-
In the paper, ColorBench is described as covering optimal, sub-optimal, and recovery paths.
- Did you release any official trajectory annotations for these (e.g., per-task optimal trajectory / sub-optimal trajectory / recovery trajectory)?
- Or are users expected to reconstruct them from
graph.json and milestones?
-
I reconstructed task-conditioned trajectories from data/graph.json + data/tasks.json, and found that for some tasks, the shortest feasible path is mismatch with optimal_steps in tasks.json.
- For example, task_id 127 can reach the milestone-defined goal in about 5 steps, while
optimal_steps is larger.
- Is this expected? If yes, what is the intended meaning of
optimal_steps (strict ground-truth optimum vs approximate reference length)?
Hi, thanks for releasing ColorBench.
I have two questions:
In the paper, ColorBench is described as covering optimal, sub-optimal, and recovery paths.
graph.jsonand milestones?I reconstructed task-conditioned trajectories from
data/graph.json+data/tasks.json, and found that for some tasks, the shortest feasible path is mismatch withoptimal_stepsintasks.json.optimal_stepsis larger.optimal_steps(strict ground-truth optimum vs approximate reference length)?