WASI OTel

[asteurer]

Overview

This is an implementation of WASI OTel which gives guest applications the ability to export traces and metrics (and eventually logs) with OpenTelemetry SDKs.

We see two options for adding this to Spin:

• Require the user to compile Spin with configuration flags that will enable this feature.
• Have this feature available out-of-the-box and add an experimental-features flag that the user will pass to spin up.

To help ease adoption, we would prefer to pass a flag to spin up, but we know there was some previous work around this with WASI P3 so we're happy to defer to that process.

Some additional notes:

• In the root Cargo.toml, you will notice that certain experimental features are being enabled for the opentelemetry-sdk dependency, and that we are using reqwest-client instead of reqwest-blocking-client in the opentelemetry dependency. These are all to prevent runtime errors that come from the default OpenTelemetry dependency configurations conflicting with Spin's async runtime.
• Currently wasi-otel is a phase 0 proposal but we are presenting it to the WASI subgroup this month in the hopes of moving it to phase 1.

Usage

To try the features added in this PR, do the following:

1. Build Spin from this branch:

git clone --branch wasi-otel --depth 1 https://github.com/asteurer/spin
cd spin
cargo install --path .
spin plugin update
spin plugin install otel

2. Clone OpenTelemetry WASI and try the Rust example applications:

git clone https://github.com/calebschoepp/opentelemetry-wasi
spin otel setup

###############
### Tracing ###
###############
cd opentelemetry-wasi/rust/examples/spin-basic
spin build
spin otel up

# In a different terminal window
spin otel open jaeger
curl localhost:3000

################
###  Metrics ###
################
cd opentelemetry-wasi/rust/examples/spin-metrics
spin build
spin otel up

# In a different terminal window
spin otel open prometheus
curl localhost:3000

[calebschoepp]

@lann I think you would likely be the most relevant reviewer for this

[lann]

This was mostly just a shallow pass here with a few surface-level comments.

Looking at how this code is working, I think there may be a viable approach that wouldn't require changes to the outbound factors. I only have a minute right now but briefly:

• Use a task-local to manage state through the host-guest-host burger
• Use an otel span processor that uses that task-local to conditionally reparent spans

[lann]

There is already https://docs.rs/opentelemetry/latest/opentelemetry/context/struct.Context.html; maybe OtelState or OtelFactorState or SpinOtel or something?

[lann]

It isn't immediately obvious why these visibility changes are necessary?

[lann]

Is this expect (and the one below) actually impossible?

[lann]

Is this intentional here?

[lann]

My immediate hot take is that we should break this out into a separate wasi-otel crate. We've been discussing removing this spin-world crate entirely and I think these conversion impls are a good example of why we'd rather split up these import bindings.