feat: Add opentelemetry initial declarative configuration

[andborja]

Changes

Adds initial declarative configuration support.

Merge requirement checklist

• CONTRIBUTING guidelines followed
• Unit tests added/updated (if applicable)
• Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
• Changes in public API reviewed (if applicable)

[codecov]

Codecov Report

❌ Patch coverage is 82.62295% with 106 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.8%. Comparing base (d88641c) to head (84137e9).

Files with missing lines	Patch %	Lines
opentelemetry-config/src/providers.rs	74.6%	38 Missing ⚠️
opentelemetry-config/src/lib.rs	73.7%	32 Missing ⚠️
...ig/src/providers/meter_provider/reader_provider.rs	84.3%	29 Missing ⚠️
opentelemetry-config/src/model/metrics/reader.rs	92.8%	5 Missing ⚠️
opentelemetry-config/src/model.rs	97.1%	1 Missing ⚠️
...entelemetry-config/src/providers/meter_provider.rs	97.9%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #487     +/-   ##
=======================================
+ Coverage   54.4%   55.8%   +1.3%     
=======================================
  Files         71      77      +6     
  Lines      11892   12502    +610     
=======================================
+ Hits        6476    6980    +504     
- Misses      5416    5522    +106     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[utpilla]

We now seem to have a good approach that offers extensibility well. However, there are a lot of unnecessary APIs and a lot of APIs that shouldn't be public.

I don't think we need more public APIs than these for Metrics:

For opentelemetry-config

• ConfigurationProvidersRegistry (struct)
• ConfigurationProvidersRegistry::new()
• ConfigurationProvidersRegistry::register_metrics_config()
• TelemetryProviders (struct)
• TelemetryProviders::generate_from_yaml_file() -> Result<TelemetryProviders, ConfigurationError>`
• ConfigurationError (struct or enum that would be used an error type)
• TelemetryProviders::meter_provider() which can be used to retrieve the MeterProvider generated

For opentelemetry-config-stdout

• register_console_exporter()

[andborja]

• ConfigurationProvidersRegistry

• ConfigurationProviderRegistry is the composition of the other registries. It adds value to group the registries passed as parameter for configuration.
• register_metrics_config cannot be a single method, as the contract is different for periodic and pull readers. It is reader based in the latest revision.
• We need the 3 functions, and the order of relevance is actually different:
  generate_from(struct) > generate_from_yaml(string) > generate_from_yaml_file(file). The relevance order is because as this is a library, it will be linked from the host application configuration as a primarily use case. The configuration will not live in a separate file for most of cases.

[utpilla]

• ConfigurationProviderRegistry is the composition of the other registries. It adds value to group the registries passed as parameter for configuration.

What additional value is added by metrics_mut method if the method used to register the configuration already has the word metrics in it and also requires a mutable reference &mut to ConfigurationProviderRegistry?

• register_metrics_config cannot be a single method, as the contract is different for periodic and pull readers. It is reader based in the latest revision.

Why do we need different contracts? The only contract needed is access to MeterProviderBuilder and the associated yaml config data for the component. What's the benefit of moving away from a single register_metrics_config method approach?

• We need the 3 functions, and the order of relevance is actually different:
  generate_from(struct) > generate_from_yaml(string) > generate_from_yaml_file(file). The relevance order is because as this is a library, it will be linked from the host application configuration as a primarily use case. The configuration will not live in a separate file for most of cases.

"The configuration will not live in a separate file for most of cases." Where is this coming from? This is a new feature, and we don't have any existing users. The examples in the opentelemetry config are file based. Please only add the bare minimum APIs for now. If there is a legit need in the future, we can always offer more APIs.

[andborja]

• ConfigurationProviderRegistry is the composition of the other registries. It adds value to group the registries passed as parameter for configuration.

What additional value is added by metrics_mut method if the method used to register the configuration already has the word metrics in it and also requires a mutable reference &mut to ConfigurationProviderRegistry?

The method inside the metrics registry does not have "metric" in it. That is part of the value, the "suffix" is taken from the object it belongs.

• register_metrics_config cannot be a single method, as the contract is different for periodic and pull readers. It is reader based in the latest revision.

Why do we need different contracts? The only contract needed is access to MeterProviderBuilder and the associated yaml config data for the component. What's the benefit of moving away from a single register_metrics_config method approach?

Take a look at the latest version. It is reader level, and it can be Periodic or Pull.

• We need the 3 functions, and the order of relevance is actually different:
  generate_from(struct) > generate_from_yaml(string) > generate_from_yaml_file(file). The relevance order is because as this is a library, it will be linked from the host application configuration as a primarily use case. The configuration will not live in a separate file for most of cases.

"The configuration will not live in a separate file for most of cases." Where is this coming from? This is a new feature, and we don't have any existing users. The examples in the opentelemetry config are file based. Please only add the bare minimum APIs for now. If there is a legit need in the future, we can always offer more APIs.

I'm implementing this to be embedded into the OTEL Arrow project. That will be the first client, and it will use the generate_from(struct).

[lalitb]

PR has changed quite a bit since my approval, keeping it open till we agree on the design.

[utpilla]

@andborja @lalitb and I discussed the design and public API design offline. Here's the approach:

From a user's perspective the workflow would like this:

1. They create a ConfigurationRegistry
2. They register configurations by calling register_metrics_config() on the ConfigurationRegistry instance
3. register_metrics_config would register a unique component name and a factory which receives serde_yaml::Value and &mut MeterProviderBuilder.
4. The factories are offered by the component crates.
5. Once the user has registered their desired factories, they will call TelemetryProviders::build_from_yaml_str(yaml : &str, configuration_registry) which would return TelemetryProviders (a type that would contain the different signal providers)
6. They could then access the generated MeterProvider by calling TelemetryProviders::meter_provider() and set it as global MeterProvvider.

Public APIs:

• pub struct ConfigurationRegistry
• pub ConfigurationRegistry::register_metrics_config(&mut self, ...)
• pub struct TelemetryProviders
• pub TelemetryProviders::build_from_yaml_str(yaml: &str) (static method)
• TelemetryProviders.meter_provider() to access the generated MeterProvider

From the perspective of components wanting to offer factories for their registration:

• They only need to offer a public method that would accept &mut MeterProviderBuilder and serde_yaml::Value and make the necessary changes to the MeterProviderBuilder and then return the builder.
• The yaml Value passed to them would contain information just enough for them to make changes to the builder. For example, a push metric exporter would be provided everything under the PeriodicReader section to which they belong.

@andborja Could you please confirm that you agree to this approach? If yes, let's have the PR updated to reflect the abovementioned design.

[andborja]

@andborja @lalitb and I discussed the design and public API design offline. Here's the approach:

From a user's perspective the workflow would like this:

1. They create a ConfigurationRegistry
2. They register configurations by calling register_metrics_config() on the ConfigurationRegistry instance
3. register_metrics_config would register a unique component name and a factory which receives serde_yaml::Value and &mut MeterProviderBuilder.
4. The factories are offered by the component crates.
5. Once the user has registered their desired factories, they will call TelemetryProviders::build_from_yaml_str(yaml : &str, configuration_registry) which would return TelemetryProviders (a type that would contain the different signal providers)
6. They could then access the generated MeterProvider by calling TelemetryProviders::meter_provider() and set it as global MeterProvvider.

Public APIs:

• pub struct ConfigurationRegistry
• pub ConfigurationRegistry::register_metrics_config(&mut self, ...)
• pub struct TelemetryProviders
• pub TelemetryProviders::build_from_yaml_str(yaml: &str) (static method)
• TelemetryProviders.meter_provider() to access the generated MeterProvider

From the perspective of components wanting to offer factories for their registration:

• They only need to offer a public method that would accept &mut MeterProviderBuilder and serde_yaml::Value and make the necessary changes to the MeterProviderBuilder and then return the builder.
• The yaml Value passed to them would contain information just enough for them to make changes to the builder. For example, a push metric exporter would be provided everything under the PeriodicReader section to which they belong.

@andborja Could you please confirm that you agree to this approach? If yes, let's have the PR updated to reflect the abovementioned design.

Confirmed