A question regarding the purpose of this challenge

[sudmat]

This is definitely the most interesting artificial challenge I’ve seen since the Abstract Reasoning Challenge (ARC). Before I dive into this and start coding, I have a small question. From what I understand, this challenge tests the agent’s ability to make (simple) decisions. However, when I tried the manual bot, I felt that the most difficult part was not decision making, but controlling the bot in a smooth and stable way so that it does not hit the wall. This was not easy for me: I tried about 10 times, and the maximum score I achieved was only 9 on the first task (I assume I made the right choice, but had difficulty executing it). Therefore, these tasks seem to be more of a low-level movement control challenge rather than a high-level navigation or decision-making challenge. Both aspects are important and not fully understood in biological intelligence, nor fully solved in artificial intelligence, but the intended focus may influence the design philosophy of the moving agent. Any clarification on this would be greatly appreciated!

[rougier]

For the manual player, you can probably add a small timer to slow it down and it should then be relatively easy to control. The task is not specifically low level but the real difficulty is to implement a decision strategy based on sensory-motor control. If you consider the first task, the decision is totally trivial if you abstract the problem into option A and option B, as it is usually done in a number of model: one option is always paying while the other is not.

However, in real life decision (embodied decision), things are not always so clear. "Choosing option A" means here to issue a motor sequence that leads you to location A where you can experience "refill" and then you can decide if you want to return to A or test B. In terms of (reinforcement) learning, this make things are much more difficult because you have no "action A", no "option A" nor reward signal. Still, problem can be solved as it has been demonstrated for task 1.

The idea is thus to remove any abstraction we may have in your head when considering a problem (choice, reward, value, etc) and see how a bot can survive. Note that sub-optimal solution are totally ok as long as your bot survive for a few iterations.