Cancelled async `java.net.http.HttpRequest` does not complete exceptionally when agent is present

[steffan-westcott]

Describe the bug

This re-opens a previous issue Cancellation of async java.net.http.HttpRequest no longer works when agent is present.

The fix applied in response to the previous issue does not fully work, as the .exceptionally(ex -> "** cancelled **") is somehow ignored. This causes applications awaiting completion of the future (either normally or exceptionally) to hang if the request was cancelled.

Steps to reproduce

Exactly the same as the previous issue

Expected behavior

The following should be printed to the terminal:

Request was cancelled with java.util.concurrent.CancellationException: Request cancelled
Got response: ** cancelled **

Actual behavior

The following is printed to the terminal:

Request was cancelled with java.util.concurrent.CancellationException: Request cancelled
Exception in thread "main" java.util.concurrent.CancellationException
        at java.base/java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2510)
        at io.opentelemetry.javaagent.shaded.instrumentation.javahttpclient.internal.CompletableFutureWrapper.cancel(CompletableFutureWrapper.java:46)
        at org.example.ReqCloseIssue.main(ReqCloseIssue.java:55)

Note the lack of output Got response: ** cancelled **, which would seem to imply the future does not complete exceptionally, which is contrary to the output Request was cancelled with java.util.concurrent.CancellationException: Request cancelled. I don't understand why this is happening. Is there some confusion with the wrapped futures?

Javaagent or library instrumentation version

opentelemetry-javaagent version 2.21.0

Environment

JDK: OpenJDK Runtime Environment Temurin-21.0.8
OS: Ubuntu 22.04.5 LTS

Additional context

As before, the application behaves as expected if the agent is not present. This demonstrates that the presence of the agent changes the application behaviour.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[laurit]

I tried running the sample app from #14696 and sometimes it produces Got response: ** cancelled ** as you expect and sometimes it gets Exception in thread "main" java.util.concurrent.CancellationException. My understanding is that there is a race here. Calling cancel first triggers the cancellation of the request and then sets the result for the future to CancellationException. If the request cancellation completes before the result is set and runs the code in exceptionally then you get the result you expect otherwise CancellationException it is. I'd suggest trying to step through the cancellation and having an extra breakpoint in the exceptionally if you wish to study this. Now I guess with the agent we always get CancellationException due to the way the

opentelemetry-java-instrumentation/instrumentation/java-http-client/library/src/main/java/io/opentelemetry/instrumentation/javahttpclient/internal/CompletableFutureWrapper.java

Lines 45 to 49 in 5c145a8

	public boolean cancel(boolean mayInterruptIfRunning) {
	boolean result = super.cancel(mayInterruptIfRunning);
	future.cancel(mayInterruptIfRunning);
	return result;
	}

is arranged. If we'd move future.cancel(mayInterruptIfRunning) before super.cancel(mayInterruptIfRunning) we could probably get similar racy behavior as without the instrumentation.