Skip to content

[3/7] Telemetry Client Management: TelemetryClient and Provider#326

Open
samikshya-db wants to merge 19 commits intotelemetry-2-infrastructurefrom
telemetry-3-client-management
Open

[3/7] Telemetry Client Management: TelemetryClient and Provider#326
samikshya-db wants to merge 19 commits intotelemetry-2-infrastructurefrom
telemetry-3-client-management

Conversation

@samikshya-db
Copy link
Collaborator

@samikshya-db samikshya-db commented Jan 28, 2026

Part 3 of 7-part Telemetry Implementation Stack

This layer adds per-host HTTP client management with reference counting.

Summary

Implements TelemetryClient for HTTP export and TelemetryClientProvider for efficient per-host client lifecycle management.

Components

TelemetryClient (lib/telemetry/TelemetryClient.ts)

Placeholder HTTP client for telemetry export:

  • Per-host isolation for connection pooling
  • Lifecycle management (open/close)
  • Foundation for future HTTP implementation
  • Clean interface for export operations

TelemetryClientProvider (lib/telemetry/TelemetryClientProvider.ts)

Per-host client lifecycle manager:

  • Reference counting: Tracks active connections per host
  • Automatic creation: Creates client on first connection to host
  • Automatic cleanup: Closes and removes client when refCount reaches zero
  • Thread-safe: Per-host management prevents race conditions
  • Efficient: One HTTP client shared across multiple connections

Reference Counting Flow:

  1. Connection 1 to host → refCount=1, create client
  2. Connection 2 to same host → refCount=2, reuse client
  3. Connection 1 closes → refCount=1, keep client
  4. Connection 2 closes → refCount=0, close and remove client

Design Pattern

Follows JDBC driver pattern for resource management:

  • One client per host (efficient resource utilization)
  • Shared across connections (connection pooling)
  • Reference counting lifecycle (automatic cleanup)
  • Per-host isolation (failure isolation)

Testing

  • 31 unit tests for TelemetryClient (100% function coverage)
  • 31 unit tests for TelemetryClientProvider (100% function coverage)
  • Tests verify reference counting accuracy
  • Tests verify lifecycle management (create, share, cleanup)
  • Tests verify per-host isolation

Next Steps

This PR is followed by:

  • [4/7] Event & Aggregation: EventEmitter and MetricsAggregator
  • [5/7] Export: DatabricksTelemetryExporter
  • [6/7] Integration: Wire into DBSQLClient
  • [7/7] Testing & Documentation

Dependencies

Depends on:

This was referenced Jan 28, 2026
Open
Merged
Merged
Merged
@samikshya-db samikshya-db force-pushed the telemetry-2-infrastructure branch from 06b1f7a to 1cdb716 Compare January 29, 2026 20:20
samikshya-db and others added 6 commits January 29, 2026 20:20
@samikshya-db
Add telemetry infrastructure: CircuitBreaker and FeatureFlagCache
8721993 
This is part 2 of 7 in the telemetry implementation stack.

Components:
- CircuitBreaker: Per-host endpoint protection with state management
- FeatureFlagCache: Per-host feature flag caching with reference counting
- CircuitBreakerRegistry: Manages circuit breakers per host

Circuit Breaker:
- States: CLOSED (normal), OPEN (failing), HALF_OPEN (testing recovery)
- Default: 5 failures trigger OPEN, 60s timeout, 2 successes to CLOSE
- Per-host isolation prevents cascade failures
- All state transitions logged at debug level

Feature Flag Cache:
- Per-host caching with 15-minute TTL
- Reference counting for connection lifecycle management
- Automatic cache expiration and refetch
- Context removed when refCount reaches zero

Testing:
- 32 comprehensive unit tests for CircuitBreaker
- 29 comprehensive unit tests for FeatureFlagCache
- 100% function coverage, >80% line/branch coverage
- CircuitBreakerStub for testing other components

Dependencies:
- Builds on [1/7] Types and Exception Classifier
@samikshya-db
Add telemetry client management: TelemetryClient and Provider
fd10a69 
This is part 3 of 7 in the telemetry implementation stack.

Components:
- TelemetryClient: HTTP client for telemetry export per host
- TelemetryClientProvider: Manages per-host client lifecycle with reference counting

TelemetryClient:
- Placeholder HTTP client for telemetry export
- Per-host isolation for connection pooling
- Lifecycle management (open/close)
- Ready for future HTTP implementation

TelemetryClientProvider:
- Reference counting tracks connections per host
- Automatically creates clients on first connection
- Closes and removes clients when refCount reaches zero
- Thread-safe per-host management

Design Pattern:
- Follows JDBC driver pattern for resource management
- One client per host, shared across connections
- Efficient resource utilization
- Clean lifecycle management

Testing:
- 31 comprehensive unit tests for TelemetryClient
- 31 comprehensive unit tests for TelemetryClientProvider
- 100% function coverage, >80% line/branch coverage
- Tests verify reference counting and lifecycle

Dependencies:
- Builds on [1/7] Types and [2/7] Infrastructure
@samikshya-db @claude
Add authentication support for REST API calls
dd2bac8 
Implements getAuthHeaders() method for authenticated REST API requests:
- Added getAuthHeaders() to IClientContext interface
- Implemented in DBSQLClient using authProvider.authenticate()
- Updated FeatureFlagCache to fetch from connector-service API with auth
- Added driver version support for version-specific feature flags
- Replaced placeholder implementation with actual REST API calls

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@samikshya-db
Fix feature flag and telemetry export endpoints
4437ae9 
- Change feature flag endpoint to use NODEJS client type
- Fix telemetry endpoints to /telemetry-ext and /telemetry-unauth
- Update payload to match proto with system_configuration
- Add shared buildUrl utility for protocol handling
@samikshya-db
Match JDBC telemetry payload format
e9c3138 
- Change payload structure to match JDBC: uploadTime, items, protoLogs
- protoLogs contains JSON-stringified TelemetryFrontendLog objects
- Remove workspace_id (JDBC doesn't populate it)
- Remove debug logs added during testing
@samikshya-db
Fix lint errors
32003e9 
- Fix import order in FeatureFlagCache
- Replace require() with import for driverVersion
- Fix variable shadowing
- Disable prefer-default-export for urlUtils
@samikshya-db samikshya-db force-pushed the telemetry-3-client-management branch from 87d1e85 to 32003e9 Compare January 29, 2026 20:21
@samikshya-db
Add missing getAuthHeaders method to ClientContextStub
4df6ce0 
Fix TypeScript compilation error by implementing getAuthHeaders
method required by IClientContext interface.
@samikshya-db samikshya-db force-pushed the telemetry-3-client-management branch from 2893a89 to 4df6ce0 Compare January 29, 2026 20:27
samikshya-db added a commit that referenced this pull request Feb 5, 2026
@samikshya-db
Merge PR #326 fixes into PR #327
bdf8c5a
samikshya-db and others added 6 commits February 5, 2026 11:55
@samikshya-db
Merge telemetry-2-infrastructure proxy fix into telemetry-3-client-ma…
bda2cac 
…nagement
@madhav-db @claude
Add token provider infrastructure for token federation (Token Federat…
870bcb3 
…ion 1/3) (#318)

* Add token provider infrastructure for token federation

This PR introduces the foundational token provider system that enables
custom token sources for authentication. This is the first of three PRs
implementing token federation support.

New components:
- ITokenProvider: Core interface for token providers
- Token: Token class with JWT parsing and expiration handling
- StaticTokenProvider: Provides a constant token
- ExternalTokenProvider: Delegates to a callback function
- TokenProviderAuthenticator: Adapts token providers to IAuthentication

New auth types in ConnectionOptions:
- 'token-provider': Use a custom ITokenProvider
- 'external-token': Use a callback function
- 'static-token': Use a static token string

* Fix TokenProviderAuthenticator test - remove log assertions

LoggerStub doesn't have a logs property, so removed tests that
checked for debug and warning log messages. The important behavior
(token provider authentication) is still tested.

* Fix prettier formatting in TokenProviderAuthenticator

* Fix Copilot issues: update fromJWT docs and remove TokenCallback duplication

- Updated Token.fromJWT() documentation to reflect that it handles
  decoding failures gracefully instead of throwing errors
- Removed duplicate TokenCallback type definition from IDBSQLClient.ts
- Now imports TokenCallback from ExternalTokenProvider.ts to maintain
  a single source of truth

* address comments

* Retry token fetch when expired before throwing error

TokenProviderAuthenticator now requests a fresh token from the provider
when the initial token is expired, only throwing if the retry also
returns an expired token.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@madhav-db @claude
(Token Federation 2/3) (#319)
6f49f6c 
* Add token provider infrastructure for token federation

This PR introduces the foundational token provider system that enables
custom token sources for authentication. This is the first of three PRs
implementing token federation support.

New components:
- ITokenProvider: Core interface for token providers
- Token: Token class with JWT parsing and expiration handling
- StaticTokenProvider: Provides a constant token
- ExternalTokenProvider: Delegates to a callback function
- TokenProviderAuthenticator: Adapts token providers to IAuthentication

New auth types in ConnectionOptions:
- 'token-provider': Use a custom ITokenProvider
- 'external-token': Use a callback function
- 'static-token': Use a static token string

* Add token federation and caching layer

This PR adds the federation and caching layer for token providers.
This is the second of three PRs implementing token federation support.

New components:
- CachedTokenProvider: Wraps providers with automatic caching
  - Configurable refresh threshold (default 5 minutes before expiry)
  - Thread-safe handling of concurrent requests
  - clearCache() method for manual invalidation

- FederationProvider: Wraps providers with RFC 8693 token exchange
  - Automatically exchanges external IdP tokens for Databricks tokens
  - Compares JWT issuer with Databricks host to determine if exchange needed
  - Graceful fallback to original token on exchange failure
  - Supports optional clientId for M2M/service principal federation

- utils.ts: JWT decoding and host comparison utilities
  - decodeJWT: Decode JWT payload without verification
  - getJWTIssuer: Extract issuer from JWT
  - isSameHost: Compare hostnames ignoring ports

New connection options:
- enableTokenFederation: Enable automatic token exchange
- federationClientId: Client ID for M2M federation

* Fix TokenProviderAuthenticator test - remove log assertions

LoggerStub doesn't have a logs property, so removed tests that
checked for debug and warning log messages. The important behavior
(token provider authentication) is still tested.

* Fix TokenProviderAuthenticator test - remove log assertions

LoggerStub doesn't have a logs property, so removed tests that
checked for debug and warning log messages. The important behavior
(token provider authentication) is still tested.

* Fix prettier formatting in TokenProviderAuthenticator

* Fix Copilot issues: update fromJWT docs and remove TokenCallback duplication

- Updated Token.fromJWT() documentation to reflect that it handles
  decoding failures gracefully instead of throwing errors
- Removed duplicate TokenCallback type definition from IDBSQLClient.ts
- Now imports TokenCallback from ExternalTokenProvider.ts to maintain
  a single source of truth

* Fix prettier formatting in TokenProviderAuthenticator

* Fix Copilot issues: update fromJWT docs and remove TokenCallback duplication

- Updated Token.fromJWT() documentation to reflect that it handles
  decoding failures gracefully instead of throwing errors
- Removed duplicate TokenCallback type definition from IDBSQLClient.ts
- Now imports TokenCallback from ExternalTokenProvider.ts to maintain
  a single source of truth

* Simplify FederationProvider tests - remove nock dependency

Removed nock dependency from FederationProvider tests since it's not
available in package.json. Simplified tests to focus on the pass-through
logic without mocking HTTP calls:
- Pass-through when issuer matches host
- Pass-through for non-JWT tokens
- Case-insensitive host matching
- Port-ignoring host matching

The core logic (determining when exchange is needed) is still tested.

* Fix prettier formatting in DBSQLClient.ts

* Fix ESLint errors in token provider code

- Remove unused decodeJWT import from FederationProvider
- Move extractHostname before isSameHost to fix use-before-define
- Add empty hostname validation to isSameHost

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* address comments

* address comments

* lint fix

* Retry token fetch when expired before throwing error

TokenProviderAuthenticator now requests a fresh token from the provider
when the initial token is expired, only throwing if the retry also
returns an expired token.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Run prettier formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
@madhav-db @claude
Token Federation Examples (Token Federation 3/3) (#320)
538556d 
* Add token provider infrastructure for token federation

This PR introduces the foundational token provider system that enables
custom token sources for authentication. This is the first of three PRs
implementing token federation support.

New components:
- ITokenProvider: Core interface for token providers
- Token: Token class with JWT parsing and expiration handling
- StaticTokenProvider: Provides a constant token
- ExternalTokenProvider: Delegates to a callback function
- TokenProviderAuthenticator: Adapts token providers to IAuthentication

New auth types in ConnectionOptions:
- 'token-provider': Use a custom ITokenProvider
- 'external-token': Use a callback function
- 'static-token': Use a static token string

* Add token federation and caching layer

This PR adds the federation and caching layer for token providers.
This is the second of three PRs implementing token federation support.

New components:
- CachedTokenProvider: Wraps providers with automatic caching
  - Configurable refresh threshold (default 5 minutes before expiry)
  - Thread-safe handling of concurrent requests
  - clearCache() method for manual invalidation

- FederationProvider: Wraps providers with RFC 8693 token exchange
  - Automatically exchanges external IdP tokens for Databricks tokens
  - Compares JWT issuer with Databricks host to determine if exchange needed
  - Graceful fallback to original token on exchange failure
  - Supports optional clientId for M2M/service principal federation

- utils.ts: JWT decoding and host comparison utilities
  - decodeJWT: Decode JWT payload without verification
  - getJWTIssuer: Extract issuer from JWT
  - isSameHost: Compare hostnames ignoring ports

New connection options:
- enableTokenFederation: Enable automatic token exchange
- federationClientId: Client ID for M2M federation

* Add token federation examples and public exports

This PR adds usage examples and exports token provider types for public use.
This is the third of three PRs implementing token federation support.

Examples added (examples/tokenFederation/):
- staticToken.ts: Simple static token usage
- externalToken.ts: Dynamic token from callback
- federation.ts: Token federation with external IdP
- m2mFederation.ts: Service principal federation with clientId
- customTokenProvider.ts: Custom ITokenProvider implementation

Public API exports:
- Token: Token class with JWT handling
- StaticTokenProvider: Static token provider
- ExternalTokenProvider: Callback-based token provider
- CachedTokenProvider: Caching decorator
- FederationProvider: Token exchange decorator
- ITokenProvider: Interface type (TypeScript)

Also:
- Updated tsconfig.build.json to exclude examples from build

* Fix TokenProviderAuthenticator test - remove log assertions

LoggerStub doesn't have a logs property, so removed tests that
checked for debug and warning log messages. The important behavior
(token provider authentication) is still tested.

* Fix TokenProviderAuthenticator test - remove log assertions

LoggerStub doesn't have a logs property, so removed tests that
checked for debug and warning log messages. The important behavior
(token provider authentication) is still tested.

* Fix TokenProviderAuthenticator test - remove log assertions

LoggerStub doesn't have a logs property, so removed tests that
checked for debug and warning log messages. The important behavior
(token provider authentication) is still tested.

* Fix prettier formatting in TokenProviderAuthenticator

* Fix Copilot issues: update fromJWT docs and remove TokenCallback duplication

- Updated Token.fromJWT() documentation to reflect that it handles
  decoding failures gracefully instead of throwing errors
- Removed duplicate TokenCallback type definition from IDBSQLClient.ts
- Now imports TokenCallback from ExternalTokenProvider.ts to maintain
  a single source of truth

* Fix prettier formatting in TokenProviderAuthenticator

* Fix Copilot issues: update fromJWT docs and remove TokenCallback duplication

- Updated Token.fromJWT() documentation to reflect that it handles
  decoding failures gracefully instead of throwing errors
- Removed duplicate TokenCallback type definition from IDBSQLClient.ts
- Now imports TokenCallback from ExternalTokenProvider.ts to maintain
  a single source of truth

* Fix prettier formatting in TokenProviderAuthenticator

* Fix Copilot issues: update fromJWT docs and remove TokenCallback duplication

- Updated Token.fromJWT() documentation to reflect that it handles
  decoding failures gracefully instead of throwing errors
- Removed duplicate TokenCallback type definition from IDBSQLClient.ts
- Now imports TokenCallback from ExternalTokenProvider.ts to maintain
  a single source of truth

* Simplify FederationProvider tests - remove nock dependency

Removed nock dependency from FederationProvider tests since it's not
available in package.json. Simplified tests to focus on the pass-through
logic without mocking HTTP calls:
- Pass-through when issuer matches host
- Pass-through for non-JWT tokens
- Case-insensitive host matching
- Port-ignoring host matching

The core logic (determining when exchange is needed) is still tested.

* Simplify FederationProvider tests - remove nock dependency

Removed nock dependency from FederationProvider tests since it's not
available in package.json. Simplified tests to focus on the pass-through
logic without mocking HTTP calls:
- Pass-through when issuer matches host
- Pass-through for non-JWT tokens
- Case-insensitive host matching
- Port-ignoring host matching

The core logic (determining when exchange is needed) is still tested.

* Fix prettier formatting in DBSQLClient.ts

* Fix prettier formatting in DBSQLClient.ts

* Fix prettier formatting in token federation examples

* Fix ESLint errors in token provider code

- Remove unused decodeJWT import from FederationProvider
- Move extractHostname before isSameHost to fix use-before-define
- Add empty hostname validation to isSameHost

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix ESLint errors in token provider code

- Remove unused decodeJWT import from FederationProvider
- Move extractHostname before isSameHost to fix use-before-define
- Add empty hostname validation to isSameHost

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* address comments

* address comments

* address comments

* lint fix

* lint fix

* Retry token fetch when expired before throwing error

TokenProviderAuthenticator now requests a fresh token from the provider
when the initial token is expired, only throwing if the retry also
returns an expired token.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Run prettier formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
@samikshya-db
prepare release 1.13.0 (#336)
775e642
@samikshya-db
Merge latest main into telemetry-3-client-management
fce4499
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jadewang-db jadewang-db Awaiting requested review from jadewang-db

@sreekanth-db sreekanth-db Awaiting requested review from sreekanth-db

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@samikshya-db @madhav-db