-
Couldn't load subscription status.
- Fork 2.7k
feat(storage): Add SigV4 authentication support for Elasticsearch/OpenSearch storage backends #7611
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat(storage): Add SigV4 authentication support for Elasticsearch/OpenSearch storage backends #7611
Conversation
|
@yurishkuro This PR adds SigV4 authentication support for ES/OS backends (both trace and metric storage) by reusing the existing HTTP RoundTripper pattern. The smaller diff compared to #7520 is because ES/OS already has auth infrastructure, so I just threaded the |
7279cf9 to
c0d9da3
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #7611 +/- ##
==========================================
- Coverage 96.59% 96.50% -0.10%
==========================================
Files 384 384
Lines 19404 19428 +24
==========================================
+ Hits 18744 18749 +5
- Misses 477 489 +12
- Partials 183 190 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Metrics Comparison SummaryTotal changes across all snapshots: 53 Detailed changes per snapshotsummary_metrics_snapshot_cassandra📊 Metrics Diff SummaryTotal Changes: 53
❌ Removed Metrics
View diff sample-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...View diff sample-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.005",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.01",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.025",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.05",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.075",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.1",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...View diff sample-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
... |
c0d9da3 to
397a84b
Compare
- Add AuthExtension field to ES/OS configuration - Implement authenticator resolution in jaegerstorage extension - Update factory chain to pass httpAuth parameter - Add support for both trace and metric storage - Update all tests to handle new httpAuth parameter Fixes jaegertracing#7468 Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
- Add AuthExtension field to Elasticsearch/OpenSearch configuration - Implement getAuthenticator method in jaegerstorage extension - Update factory chain (v1/v2/metrics) to accept httpAuth parameter - Support authentication for both trace and metric storage - Add test coverage for authenticator resolution - Update all existing tests to handle new httpAuth parameter This enables using OpenTelemetry HTTP authenticator extensions (e.g., sigv4auth) with Elasticsearch and OpenSearch backends. The authenticator is resolved from the host and wrapped into the HTTP transport layer for secure AWS authentication. Fixes jaegertracing#7468 Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com> Signed-off-by: Soumya Raikwar <164396577+SoumyaRaikwar@users.noreply.github.com>
- Add resolveAuthenticator helper to reduce code duplication - Simplifies ES/OS backend initialization in extension.go Addresses review feedback from @yurishkuro Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
397a84b to
d6f9241
Compare
- Test nil config handling (returns nil auth) - Test empty authenticator string (returns nil auth) - Test valid authenticator resolution - Test authenticator not found error case - Improves test coverage for authentication code Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
| cfg.UseReadWriteAliases = true | ||
| } | ||
| coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger) | ||
| coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, nil) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The nil parameter passed to NewFactoryBase() should be replaced with the HTTP authenticator. The function signature has been updated to accept an authenticator parameter, but this call site is still passing nil, which means SigV4 authentication won't work when using this factory.
Consider updating this to pass the authenticator through, similar to how it's done in other factory implementations in this PR.
| coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, nil) | |
| coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, authenticator) |
Spotted by Graphite Agent
Is this helpful? React 👍 or 👎 to let us know.
- Test configuration with auth extension - Test configuration without auth extension - Test configuration with empty authenticator - Improves coverage for auth extension config Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
- Add TestNewFactoryWithAuthenticator for metricstore factory - Add TestElasticsearchFactoryBaseWithAuthenticator for v1 factory - Tests verify factory initialization with HTTP authenticator - Improves coverage for authenticator integration paths Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
|
@yurishkuro Done! Introduced |
…kends - Add auth_extension config support for ES/OS trace and metric backends - Implement resolveAuthenticator helper to reduce code duplication - Add comprehensive unit tests for authenticator resolution - Add factory creation tests with HTTP authenticators - Fix error handling and variable shadowing issues Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Keep Prometheus using Auth config (PR jaegertracing#7520 already merged). ES/OS use new Authentication.AuthExtension pattern. Wrapper approach preserves backward compatibility while allowing all backends to share resolveAuthenticator helper internally. Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
- Change Authentication struct to use direct Authenticator field - Remove nested AuthExtension config for cleaner API - Match OpenTelemetry Collector's standard auth pattern - Update all tests and references accordingly Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com> Signed-off-by: Soumya Raikwar <164396577+SoumyaRaikwar@users.noreply.github.com>
- Embed configauth.Config in Authentication struct as per @yurishkuro's suggestion - Initialize embedded Config in test literals explicitly - Add configauth import to config_test.go Fixes the simpler authentication configuration pattern. Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
|
Hi @yurishkuro, I've refactored the implementation following your suggestion to embed Changes made:
This follows the same pattern used in other OTel collector components and keeps the code simple. |
- Remove AuthExtensionConfig type alias - Pass Authentication struct directly to resolveAuthenticator - Update Prometheus auth to use Authentication struct - Update all tests to match new signature Follows OTEL pattern by embedding configauth.Config with squash tag Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
|
@yurishkuro Changes implemented! Removed the
The embedded |
| ) | ||
| } | ||
|
|
||
| func TestAuthExtensionConfig(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see the need for this test. First, wantErr is always false, but more importantly what is it actually validating? This package has no knowledge of "extensions".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed! Removed TestAuthExtensionConfig - you're right, it provided no real validation.
Authentication is properly tested in extension_test.go where the actual resolution logic lives.
| // Resolve authenticator if configured | ||
| var httpAuthenticator extensionauth.HTTPClient | ||
| var promAuth config.Authentication | ||
| if cfg.Prometheus.Auth != nil && cfg.Prometheus.Auth.Authenticator != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the fact that this is different from ES above tells me the config structure is still different. Can we make them identical? The Prometheus change was merged but not released yet, so we have an opportunity for making a consistent config API change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done! Made Prometheus config identical to Elasticsearch:
- Removed custom
AuthConfigwrapper - Now uses
escfg.Authenticationdirectly (same as ES/OpenSearch) - Eliminated boilerplate conversion code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prometheus config structure is slightly different from ES/OpenSearch since it wraps promcfg.Configuration with ,squash, but the Authentication field itself is now identical across all backends.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also added ctx context.Context parameter to prometheus.NewFactoryWithConfig() for consistency with Elasticsearch/OpenSearch backends and to enable proper context propagation for authentication and cancellation.
- Removed TestAuthExtensionConfig from ES config tests (no actual validation) - Changed PrometheusConfiguration to use escfg.Authentication directly - Removed custom AuthConfig wrapper and boilerplate conversion code - Updated config_test to access embedded Configuration.ServerURL Both Elasticsearch and Prometheus now use identical config structures following OpenTelemetry patterns with configauth.Config. Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Made Prometheus metrics backend authentication configuration identical to Elasticsearch and OpenSearch for consistency: - Removed custom AuthConfig wrapper in Prometheus config - Now uses escfg.Authentication directly (same as ES/OpenSearch) - Eliminated boilerplate authentication conversion code - Added ctx parameter to NewFactoryWithConfig() for proper context propagation - Maintains Prometheus-specific jsquash wrapper for config parsing This ensures consistent authentication patterns across all storage backends while enabling proper context propagation for authentication and cancellation. Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
Signed-off-by: SoumyaRaikwar <somuraik@gmail.com>
|
@yurishkuro could you please review again when you have time . |
This PR enables Jaeger to use AWS Managed Elasticsearch/OpenSearch for trace and metrics storage by adding SigV4 HTTP authentication support to Elasticsearch and OpenSearch backends.
Summary of changes
Configuration
jaeger_storage.backends.<name>.<elasticsearch|opensearch>.auth_extension.authenticatorto reference an OpenTelemetry HTTP authenticator extension by namejaeger_storage.metric_backends.<name>.<elasticsearch|opensearch>.auth_extension.authenticatorfor metric storage backendsElasticsearch/OpenSearch backends
sigv4authextension)GetHTTPRoundTripper()to accept and apply the HTTP authenticatorConfiguration example
Implementation
auth_extension.authenticatorScope
Related issue
Part of #7468