Area Learning Example: Loop Closure correction not working

[LudwikJaniuk]

Final result: the red markers should be "leading up" to the blue one.

It seems that the loop closure correction on ARMarkers in the AreaLearning example (this code) is not working. I have created a test to make this visible, and even tho a loop closure event occurs, the markers are not moved to the "correct" positions on save.

The way I test this is the following:

1. I create a new area, and scan a little in a specific location, to make it possible to relocalize later.
2. I exit the area and load it again, with area learning disabled.
3. I place the blue marker in the place that is scanned in the ADF (this is where the loop closure correction will happen, where Tango will realize that is has travelled in a loop, and is supposed to update the path it's taken to reflect that. So when querying with GetPoseAtTimestamp later, one should get the the pose from the new, updated trajectory).
4. I walk away about ten meters, and spin around slowly to accumulate noticeable drift.
5. I walk back to the place I placed the blue marker, and on my way, continually place red markers. At first, the blue marker is not in the place I placed it (because of the built-up drift, the marker is several meters away).
6. When tango sees the starting area where the blue marker was placed, a loop closure event seems to happen, because the blue marker teleports back to its proper location, and the red markers I just recently placed instead become offset by the same distance that the blue marker was offset before, in the opposite direction. What's actually happening of course is that the markers' positions are not changed, but the tango camera changes its position drastically, because it realizes it is in the starting position again. Up until now, everything has worked as expected. The red markers became offset because when they were being placed, the camera thought it was a few meters away, and so that's the position they got.
7. I save the scene. Here, I expect the code I linked to from AreaLearningInGameController.cs to go through the markers' positions and update them based on the "updated trajectory" that Tango computes upon a loop closure event, according to the docs about poses as I understand them. So since they were placed at the same "real" position, the red markers should end up saved very close to the blue marker (they should form a trail leading up to it, that's how they were placed).
8. Instead, when I load the scene again, the red markers remain at the same offset positions as before saving.

So it seems that for some reason, the loop closure correction is not working. However, the code does seem to be doing the right thing (it is correctly asking for poses for the areaLearning/Device frame pair, instead of the startOfSetvice/Device frame pair. If it was asking for the second one, it would indeed not get the updated trajectory, according to the docs on frames of reference. But now, it is doing the right thing (I think), so what might be causing this behaviour? Is this an error in the sample, or something wrong with the Unity SDK itself? Or am I simply misunderstanding what loop closure correction is really supposed to be doing?

Any indication on what might be the cause of this problem is greatly appreciated, as I'm trying to implement loop closure correction on my own project, and using this code as an example. For example is it possible that the tango poses are actually not being updated, even tho a loop closure event happens?

[chaosemer]

Tango's loop closure optimization pass can improve the remembered trajectory, but it won't be perfect. It sounds like the red markers were off before you saved and they got stored in that off state. Further, repeated traversal along the path of the markers can help Tango learn that you just walked back and forth along a line.

I hope this clarifies things. Please tell me if you have any more questions.

[LudwikJaniuk]

Thank you for replying. I was specifically trying to make the red markers be placed in an off state, to see loop closure correction "in action". While I understand the improved trajectory won't be perfect, the results look like it didn't happen at all, leading me to believe that there's rather something wrong with the code that updates the markers' locations to match the updated estimated trajectory. Are you saying that the loop closure correction effect is really so miniscule that it's consistent with what I saw in my experiment? What it looks like is that Tango updates its camera position by several meters on the loop closure event, but the remembered trajectory is improved only by a few centimeters. To clarify my question, I modified the picture from the docs:

When saying the improved trajectory won't be perfect, do you mean it will be closer to the magenta or orange trajectory I drew? Because from the experiment it looks like the orange one, but that would be quite disappointing.

Also, if that's the case, could you propose a different experiment I could perform to see loop closure correction in action? I want to extend it to other objects, but don't I like the idea of coding it without being able to validate that it works.

[LudwikJaniuk]

This issue is still bugging me. I'm currently resorting to scanning the whole area once before doing anything in it, but it would be nice to knwo what is going on here.