registry exit: decrement local span ref only

[davidhewitt]

Motivation

Fixes #3223

Solution

From my understanding, it's sufficient for Registry::exit just to call self.try_close() and do local bookkeeping:

• The attempt to use the thread dispatcher leads directly to the issues observed in tracing & tracing-subscriber have a design defect I believe #3223, and
• I don't think it's correct to call try_close on the whole Layer stack at this point anyway, a span being exited is not yet ready to close. All that is needed is to decrement the reference count within the current registry.

I've added a test which spawns a thread and enters (and exits, and drops) a span created on a dispatcher not registered to that thread.

Before this patch, the test fails with the span never being closed (because there is no thread dispatcher when the span is exited, and so a reference is leaked in Registry::exit).

With this patch, the bookkeeping demonstrated in the test seems correct to me.

[davidhewitt]

Note that changing Registry::exit to just call self.try_close() is also presumably a performance gain, because there is no need to dispatch through TLS on each span exit.

[vlovich]

FWIW I can also confirm this fixes the issue I've seen in real-world situations so this seems like a very simple fix that also improves performance. Great job!

[kevinji]

@jplatte are you able to review this change?

[jplatte]

Hey, I'm sorry but I don't actually have a good understanding of tracing internals at all. I've just been reviewing and merging PRs that do not need much knowledge of the internals. I'm afraid this will have to wait for @hds.

[hds]

Hey everyone. Just to let you know that I'm looking at this PR now. Hopefully get through it in the next couple of days.

[hds]

Thank you very much for your PR! I think it makes sense, I a couple of comments on the test.

[hds]

To reduce the code in the test a bit, I think we could put all the atomics into a single struct and put that in an Arc. Then the counts could be cloned into the subscriber and the layer and compared that way.

This is really with a mind to reducing the number of lines we have in this test so that it's easier to understand.

[davidhewitt]

I pushed c2f9f94 which had a go at this - I replaced the atomics with a struct behind a mutex, seems to have reduced the visual noise a bit. Let me know what you think.

[davidhewitt]

Thanks will try to get feedback addressed later today 👍