From 2ce67d93f869d49969cd0960465e076a925cd851 Mon Sep 17 00:00:00 2001 From: Mano Toth Date: Wed, 23 Jul 2025 10:55:28 +0200 Subject: [PATCH 01/48] Add info about support --- platform-overview/roadmap.mdx | 3 ++- query-data/metrics/overview.mdx | 0 query-data/metrics/query-metrics.mdx | 0 reference/system-requirements.mdx | 16 +++++++++++++++- send-data/opentelemetry.mdx | 14 ++++++++------ 5 files changed, 25 insertions(+), 8 deletions(-) create mode 100644 query-data/metrics/overview.mdx create mode 100644 query-data/metrics/query-metrics.mdx diff --git a/platform-overview/roadmap.mdx b/platform-overview/roadmap.mdx index f83f1258..7826c7c3 100644 --- a/platform-overview/roadmap.mdx +++ b/platform-overview/roadmap.mdx @@ -57,4 +57,5 @@ Current private preview features: Current public preview features: - [Cursor-based pagination](/restapi/pagination) - [`externaldata` operator](/apl/tabular-operators/externaldata-operator) -- [`join` operator](/apl/tabular-operators/join-operator) \ No newline at end of file +- [`join` operator](/apl/tabular-operators/join-operator) +- [OTel metrics](/query-data/metrics/overview) \ No newline at end of file diff --git a/query-data/metrics/overview.mdx b/query-data/metrics/overview.mdx new file mode 100644 index 00000000..e69de29b diff --git a/query-data/metrics/query-metrics.mdx b/query-data/metrics/query-metrics.mdx new file mode 100644 index 00000000..e69de29b diff --git a/reference/system-requirements.mdx b/reference/system-requirements.mdx index ade7055e..011d42bf 100644 --- a/reference/system-requirements.mdx +++ b/reference/system-requirements.mdx @@ -19,6 +19,8 @@ Some actions in the Dashboards tab, such as moving dashboard elements, aren’t ## OpenTelemetry +### Semantic conventions + Axiom supports the following versions of OTel semantic conventions: | Version | Date when supported added | Schema in OTel docs | @@ -32,7 +34,7 @@ Axiom supports the following versions of OTel semantic conventions: | 1.27.0 | 12-06-2025 | [1.27.0](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.27.0) | | 1.26.0 | 03-07-2024 | [1.26.0](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.26.0) | | 1.25.0 | 26-04-2024 | [1.25.0](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.25.0) | -| 1.24.0 | 19-01-2024| [1.24.0](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.24.0) | +| 1.24.0 | 19-01-2024 | [1.24.0](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.24.0) | | 1.23.1 | 26-03-2024 | [1.23.1](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.23.1) | | 1.23.0 | 26-03-2024 | [1.23.0](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.23.0) | | 1.22.0 | 26-03-2024 | [1.22.0](https://github.com/open-telemetry/semantic-conventions/blob/main/schemas/1.22.0) | @@ -41,3 +43,15 @@ Axiom supports the following versions of OTel semantic conventions: Version 1.29.0 of OTel semantic conventions isn’t supported. For more information, see [Semantic conventions](/reference/semantic-conventions). + +### Logs, traces, and metrics + +| OpenTelemetry component | Support | +| ------------------------------------------------------------------ | ------------------- | +| [Logs](https://opentelemetry.io/docs/concepts/signals/logs/) | ✓ | +| [Traces](https://opentelemetry.io/docs/concepts/signals/traces/) | ✓ | +| [Metrics](https://opentelemetry.io/docs/concepts/signals/metrics/) | Public preview | + + +Support for OTel metrics is currently in public preview. For more information, see [Feature states](/platform-overview/roadmap#feature-states). + diff --git a/send-data/opentelemetry.mdx b/send-data/opentelemetry.mdx index 95d59f3f..93729666 100644 --- a/send-data/opentelemetry.mdx +++ b/send-data/opentelemetry.mdx @@ -19,13 +19,15 @@ The OpenTelemetry project has published strong specifications for the three main OpenTelemetry-compatible events flow into Axiom, where they’re organized into datasets for easy segmentation. Users can create a dataset to receive OpenTelemetry data and obtain an API token for ingestion. Axiom provides comprehensive observability through browsing, querying, dashboards, and alerting of OpenTelemetry data. -OTel traces and OTel logs support are already live. Axiom will soon support OpenTelemetry Metrics (OTel Metrics). - -| OpenTelemetry component | Currently supported | +| OpenTelemetry component | Support | | ------------------------------------------------------------------ | ------------------- | -| [Traces](https://opentelemetry.io/docs/concepts/signals/traces/) | Yes | -| [Logs](https://opentelemetry.io/docs/concepts/signals/logs/) | Yes | -| [Metrics](https://opentelemetry.io/docs/concepts/signals/metrics/) | No (coming soon) | +| [Logs](https://opentelemetry.io/docs/concepts/signals/logs/) | ✓ | +| [Traces](https://opentelemetry.io/docs/concepts/signals/traces/) | ✓ | +| [Metrics](https://opentelemetry.io/docs/concepts/signals/metrics/) | Public preview | + + +Support for OTel metrics is currently in public preview. For more information, see [Feature states](/platform-overview/roadmap#feature-states). + From 59d56d05d68dd1046aa58733401a5cbebf46697a Mon Sep 17 00:00:00 2001 From: Mano Toth Date: Wed, 23 Jul 2025 17:03:31 +0200 Subject: [PATCH 02/48] First draft --- docs.json | 12 +- platform-overview/roadmap.mdx | 2 +- query-data/metrics/functions.mdx | 189 ++++++++++++++ query-data/metrics/migrate-metrics.mdx | 3 + query-data/metrics/overview.mdx | 22 ++ query-data/metrics/query-metrics.mdx | 336 +++++++++++++++++++++++++ query-data/metrics/sample-queries.mdx | 1 + reference/datasets.mdx | 2 + send-data/opentelemetry.mdx | 6 +- 9 files changed, 569 insertions(+), 4 deletions(-) create mode 100644 query-data/metrics/functions.mdx create mode 100644 query-data/metrics/migrate-metrics.mdx create mode 100644 query-data/metrics/sample-queries.mdx diff --git a/docs.json b/docs.json index 72a3345a..0458031c 100644 --- a/docs.json +++ b/docs.json @@ -52,7 +52,17 @@ "query-data/visualizations", "query-data/views", "query-data/virtual-fields", - "query-data/traces" + "query-data/traces", + { + "group": "Metrics", + "pages": [ + "query-data/metrics/overview", + "query-data/metrics/query-metrics", + "query-data/metrics/functions", + "query-data/metrics/sample-queries", + "query-data/metrics/migrate-metrics" + ] + } ] }, { diff --git a/platform-overview/roadmap.mdx b/platform-overview/roadmap.mdx index 7826c7c3..9fe8a2c5 100644 --- a/platform-overview/roadmap.mdx +++ b/platform-overview/roadmap.mdx @@ -58,4 +58,4 @@ Current public preview features: - [Cursor-based pagination](/restapi/pagination) - [`externaldata` operator](/apl/tabular-operators/externaldata-operator) - [`join` operator](/apl/tabular-operators/join-operator) -- [OTel metrics](/query-data/metrics/overview) \ No newline at end of file +- [OTel Metrics](/query-data/metrics/overview) \ No newline at end of file diff --git a/query-data/metrics/functions.mdx b/query-data/metrics/functions.mdx new file mode 100644 index 00000000..3db3a930 --- /dev/null +++ b/query-data/metrics/functions.mdx @@ -0,0 +1,189 @@ +--- +title: AxQL functions reference +description: This page explains all the functions you can use in AxQL. +--- + +AxQL is a purpose-built query language for metrics in Axiom. Unlike APL (Axiom Processing Language), which is used to query logs and traces, AxQL focuses on manipulating metrics data. This reference lists all available AxQL functions and operators, grouped by category. + +## Summary of all functions + +### Mapping + +- `rate()` +- `min()` +- `max()` +- `abs()` +- `fill_prev()` +- `fill_const(VALUE)` + +### Arithmetic + +- `+`, `-`, `*`, `/` + +### Aggregation + +- `sum`, `avg`, `min`, `max`, `count` + +### Alignment + +- `align sum(...)`, etc. + +### Grouping + +- `group by ... using ...` + +### Histogramming + +- `histogram(...)` +- `interpolate_histogram(...)` + +### Compute + +- `compute name using ...` + +### Replacement + +- `replace TAG~s/.../.../` +- `replace A=B` +- `replace A=B~s/.../.../` + +## Map functions + +Map functions transform individual time series points. + +| Function | Syntax | Description | +| ------------ | ------------------- | ------------------------------------------------------------ | +| `rate` | `rate()` | Computes the per-second rate of change for a counter metric. | +| `min` | `min()` | Returns the minimum value in the series. | +| `max` | `max()` | Returns the maximum value in the series. | +| `abs` | `abs()` | Returns the absolute value of each data point. | +| `fill_prev` | `fill_prev()` | Fills missing values using the previous non-null value. | +| `fill_const` | `fill_const(VALUE)` | Fills missing values with a constant. | + +**Examples:** + +```kusto +| map rate() +| map fill_const(0) +``` + +## Map evaluations + +Map evaluations apply arithmetic operations to each data point. + +| Operator | Example | Description | +| -------------- | ---------- | ------------------------------------ | +| `+` | `map +2` | Adds 2 to each value. | +| `-` | `map -5` | Subtracts 5 from each value. | +| `*` | `map *1.5` | Multiplies each value by 1.5. | +| `/` | `map /10` | Divides each value by 10. | +| Comparison ops | `map >100` | Filters for values greater than 100. | + +## Align functions + +Align functions resample time series to fixed intervals. + +| Function | Syntax | Description | +| -------- | ----------------------- | ------------------------------------- | +| `rate` | `align rate(INTERVAL)` | Calculates rate over aligned windows. | +| `avg` | `align avg(INTERVAL)` | Averages values in each interval. | +| `sum` | `align sum(INTERVAL)` | Sums values in each interval. | +| `min` | `align min(INTERVAL)` | Takes the minimum value per interval. | +| `max` | `align max(INTERVAL)` | Takes the maximum value per interval. | +| `count` | `align count(INTERVAL)` | Counts non-null values per interval. | + +Optional dual-resolution syntax: + +```kusto +align avg(1m,5s) +``` + +Specifies an outer alignment window (1 minute) with finer internal granularity (5 seconds). + +## Group-by functions + +Group metrics by tags and aggregate them. + +| Function | Syntax | Description | +| -------- | ------------------------------ | -------------------------------------- | +| `sum` | `group by tag1,tag2 using sum` | Groups by tag(s) and sums values. | +| `avg` | `group using avg` | Groups all series and averages values. | +| `min` | `group by host using min` | Minimum value per group. | +| `max` | `group using max` | Maximum value per group. | +| `count` | `group by service using count` | Counts series per group. | + +## Bucket-by functions + +Bucket data over time using histogram or interpolation. + +| Function | Syntax | Description | +| ----------------------- | ------------------------------------------------------- | --------------------------------------------- | +| `histogram` | `bucket using histogram(INTERVAL, BINS...)` | Buckets data into fixed-width bins. | +| `interpolate_histogram` | `bucket using interpolate_histogram(INTERVAL, BINS...)` | Interpolates between points before bucketing. | + +**Example:** + +```kusto +| bucket by status using histogram(1m, 0, 10, 100) +``` + +## Replace functions + +Modify tags or tag values using regex or renaming. + +| Type | Syntax | Description | +| -------------------- | ---------------------------- | ---------------------------------- | +| Tag value replace | `replace host~s/api/v1/api/` | Rewrites tag values using regex. | +| Tag renaming | `replace env=environment` | Renames a tag. | +| Tag rename + replace | `replace name=name~s/_/-/` | Combines renaming and replacement. | + +## Compute functions + +Combine multiple queries into a computed result. + +| Function | Syntax | Description | +| -------- | -------------------------- | --------------------------------- | +| `+` | `compute result using +` | Adds query results. | +| `-` | `compute result using -` | Subtracts one query from another. | +| `*` | `compute result using *` | Multiplies results. | +| `/` | `compute result using /` | Divides one query by another. | +| `min` | `compute result using min` | Minimum across result series. | +| `max` | `compute result using max` | Maximum across result series. | +| `avg` | `compute result using avg` | Average across result series. | + +**Example:** + +```kusto +{ + metric1:cpu.usage | map *100; + metric2:cpu.limit +}; +| compute usage_ratio using / +``` + +## Filter expressions + +Use filters to restrict series by tag values or regex. + +| Type | Syntax | | +| ----- | ------------------------- | ------------------------- | +| Value | filter status == "500"\` | Filters by equality. | +| Regex | where host != /prod.\*/\` | Regex match or non-match. | + +Operators: + +- Equality: `==`, `!=` +- Comparisons: `<`, `<=`, `>`, `>=` +- Regex: `== /.../`, `!= /.../` + +## Join + +Join time series from another metric using shared tags. + +**Syntax:** + +```kusto +| join tag1,tag2 from metric:cpu.limit by tag1,tag2 +``` + +Joins are tag-based and do not support arbitrary conditions. diff --git a/query-data/metrics/migrate-metrics.mdx b/query-data/metrics/migrate-metrics.mdx new file mode 100644 index 00000000..35fc5f5d --- /dev/null +++ b/query-data/metrics/migrate-metrics.mdx @@ -0,0 +1,3 @@ +AxQL, PromQL, Grafana + +https://docs.google.com/document/d/1F2_OBRPSJCyDiG4R35_jIVz5PxS6X-YW-VCRA4sLI0Y/edit?tab=t.0#heading=h.bihow3bsv2sw \ No newline at end of file diff --git a/query-data/metrics/overview.mdx b/query-data/metrics/overview.mdx index e69de29b..daa39ce4 100644 --- a/query-data/metrics/overview.mdx +++ b/query-data/metrics/overview.mdx @@ -0,0 +1,22 @@ +--- +title: Metrics +description: This page explains how to query OTel metrics. +--- + +import Prerequisites from "/snippets/standard-prerequisites.mdx" + +OTel Metrics + + +Support for OTel metrics is currently in public preview. For more information, see [Feature states](/platform-overview/roadmap#feature-states). + + + +info about dataset types, add OTel Metrics dataset + +## Ingest metrics +ingest data using [X](send-data/opentelemetry) + +## Query metrics + +## Dashboards and monitors diff --git a/query-data/metrics/query-metrics.mdx b/query-data/metrics/query-metrics.mdx index e69de29b..1684238a 100644 --- a/query-data/metrics/query-metrics.mdx +++ b/query-data/metrics/query-metrics.mdx @@ -0,0 +1,336 @@ +--- +title: Query metrics +description: This page explains how to query OpenTelemetry metrics. +--- + +In Axiom, you can use two query languages: +- Axiom Processing Language (APL) lets you query datasets with events, OTel logs, and OTel traces. For more information, see [Introduction to APL](/apl/introduction). +- AxQL lets you query OTel metrics. This page explains how to use AxQL. + +AxQL is a metric-focused query language that combines the simplicity of APL with the expressive power of PromQL. It enables effective querying, transformation, and aggregation of metric data, supporting diverse observability use cases. It addresses the shortcomings of existing metric query languages, which are either overly complex or too rigidly tied to specific storage formats. + +AxQL is a superset of PromQL. Existing PromQL expressions can be translated to AxQL for quick onboarding and greater flexibility. For more information, see [Migrate metrics to Axiom](/query-data/metrics/migrate-metrics). + + +Support for OTel metrics is currently in public preview. For more information, see [Feature states](/platform-overview/roadmap#feature-states). + + +## Limitations + +The current implementation of AxQL comes with the following limitations: + +- You can query single metrics. +TODO other limitations? + +## Concepts + +- **Dataset:** A group of related metrics. +- **Metric:** One-dimensional time series data. +- **Tag:** Key-value pair identifying a series. +- **Series:** A unique combination of a metric and tag set. + +## Query structure + +A typical AxQL query contains the following: + +1. **Source**: Defines dataset, metric, and optional time range +1. **Filter**: Applies conditions to series via tags +1. **Transformation** can be the following: + - **Mapping:** Maps the data to a new value. + - **Aggregation:** Aggregates the data over a given time to a single value. + - **Grouping:** Groups the data by a set of tags, combining overlapping values. + - **Bucketing:** A two-dimensional transformation that combines the time and tag dimension. + +**Example:** + +```kusto +k8s:cpu_usage[1h..] +| where namespace == "prod" +| align avg(5m) +``` + +### Elements of queries + +The following explains each element of an AxQL query. + +For an overview of all the functions in AxQL, see [AxQL functions reference](/query-data/metrics/functions). + +To learn more about the language features of AxQL, see [Language features](#language-features). + +### Source + +Specify data source and optional time bounds. + +```kusto +:[