Skip to content

Understand benchmark abnormalities #1307

@clhain

Description

@clhain

There are a few unexpected behaviors visible in the benchmark results here: https://open-telemetry.github.io/otel-arrow/benchmarks/continuous/

We should seek to understand why these occur and apply any fixes as needed.

  1. The avg cpu utilization for all tests involving at least 1 OTAP input or output swings fairly wildly relative to the otlp->otlp case, with e.g. otap->otap seeing swings from 0.1 cores to 0.18 cores between 2 tests (80% change between runs that made no updates to the rust engine).
  2. It seems unusal that otlp->otap is computationally more expensive than otlp->otlp where we have an additional conversion back to otlp after it passes through the attribute processor.
  3. Network TX traffic (network_tx_bytes_rate_avg) for otap->otlp and otlp->otlp should be nearly identical - instead they are way off. The rx traffic from the load generator (which is just another df_engine with fake-signal generator connected to an otlp exporter) for the 2 tests where otlp is the input are consistent with expectations, so it appears to be something related to the presence of either the otlp receiver or attribute processor in the pipeline.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions