Does some properties (like the temperature of an obj) will be changed if the seed is fixed?

[junleiz]

Hello, thank you very much for your work. I have a question. If for a specific task, I set the random number at the beginning of the environment initialization using random.seed(0), will the attributes of objects in the environment change, such as the temperature of an object? Thank you very much.

[PeterAJansen]

Hi @leoozy , apologies to take a few days to get back to you.

ScienceWorld is written in Scala and essentially runs completely independently of Python (with only a socket bridge using py4j), so any changes to random number generators on Python shouldn't have any effect on a given ScienceWorld environment.

A given ScienceWorld task + variation index should be deterministic, such that if you initialize the environment with a given taskIdx and varitionIdx, and run the exact same series of actions in that environment, exactly the same things should happen, and they should produce the same output.

For example, for this code block, the environment should be identical every time you load it:

taskIdx = 1
varIdx = 1
taskName = taskNames[taskIdx]
env.load(taskName, varIdx, simplificationStr, generateGoldPath=True)

Similarly, if you changed the variation index:

taskIdx = 1
varIdx = 2
taskName = taskNames[taskIdx]
env.load(taskName, varIdx, simplificationStr, generateGoldPath=True)

this would load an environment that requests the same overall task be performed, but would have variations in the task-critical objects (e.g. for a freezing task, maybe you have to freeze orange juice instead of water), and also the objects that populate the environment (e.g. variation 1 might have a couch and a bookshelf in the living room, where it variation 2 might have a chair and a desk, etc.).

Are you seeing a behavior that appears non-deterministic? ScienceWorld originally had some non-deterministic behavior, but I think we've removed at least 99.9%+ of cases that we know about.

[MarcCote]

Isn't this related to #33?