Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ Notice that the second string returned is the user agent successfully parsed fro

Before looking at the final `CREATE MATERIALIZED VIEW` command, let's view a couple more functions used to clean up the data.
For example, the value of `RequestMethod` is `"GET` containing an unwanted double-quote.
You can use the [`trim`](/sql-reference/functions/string-functions#trim) function to remove the double quote:
You can use the [`trimBoth` (alias `trim`)](/sql-reference/functions/string-functions#trimBoth) function to remove the double quote:

```sql
SELECT trim(LEADING '"' FROM '"GET')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import search_view from '@site/static/images/clickstack/nginx-logs-search-view.p

# Monitoring Nginx Logs with ClickStack {#nginx-clickstack}

::::note[TL;DR]
:::note[TL;DR]
This guide shows you how to monitor nginx with ClickStack by configuring the OpenTelemetry collector to ingest nginx access logs. You'll learn how to:

- Configure nginx to output JSON-formatted logs
Expand All @@ -29,7 +29,7 @@ This guide shows you how to monitor nginx with ClickStack by configuring the Ope
A demo dataset with 10,000 sample logs is provided to test the integration before connecting your production nginx instances.

Time Required: 5-10 minutes.
::::
:::

## Prerequisites {#prerequisites}
- ClickStack instance running
Expand Down Expand Up @@ -118,12 +118,12 @@ This configuration:
- Adds source: nginx attribute for filtering in HyperDX
- Routes logs to the ClickHouse exporter via a dedicated pipeline

::::note
:::note
- You only define new receivers and pipelines in the custom config
- The processors (memory_limiter, transform, batch) and exporters (clickhouse) are already defined in the base ClickStack configuration - you just reference them by name
- The time_parser operator extracts timestamps from nginx's time_local field to preserve original log timing
- The pipelines route data from your receivers to the ClickHouse exporter via the existing processors
::::
:::

## Configure ClickStack to load custom configuration {#load-custom}

Expand Down Expand Up @@ -161,9 +161,9 @@ docker run --name clickstack \
docker.hyperdx.io/hyperdx/hyperdx-all-in-one:latest
```

::::note
:::note
Ensure the ClickStack collector has appropriate permissions to read the nginx log files. In production, use read-only mounts (:ro) and follow the principle of least privilege.
::::
:::

## Verifying Logs in ClickStack {#verifying-logs}
Once configured, log into HyperDX and verify logs are flowing:
Expand Down Expand Up @@ -255,9 +255,9 @@ Once ClickStack is running (you may have to create an account and login first):

1. Open [HyperDX](http://localhost:8080/search?from=1760976000000&to=1761062400000&isLive=false&source=690235c1a9b7fc5a7c0fffc7&select=Timestamp,ServiceName,SeverityText,Body&where=&whereLanguage=lucene&filters=[]&orderBy=)

::::note
:::note
It is important to use the link above to get the correct time range, if you don't use this link set your time range to Oct 20 11:00:00 - Oct 21 11:00:00 to see proper results.
::::
:::

Here's what you should see in your search view:

Expand All @@ -274,7 +274,7 @@ To help you get started monitoring nginx with ClickStack, we provide essential v
<VerticalStepper>
## <a href={useBaseUrl('/examples/example-logs-dashboard.json')} download="nginx-logs-dashboard.json">Download</a> the dashboard configuration.

## Import Pre-built Dashboard {#import-dashboard}
## Import the pre-built dashboard {#import-dashboard}
1. Open HyperDX and navigate to the Dashboards section.
2. Click "Import Dashboard" in the upper right corner under the ellipses.

Expand Down Expand Up @@ -333,7 +333,7 @@ docker exec `<container>` cat /etc/otel/supervisor-data/effective.yaml | grep fi
docker exec `<container>` cat /etc/otel/supervisor-data/agent.log
```

## Next Steps {#next-steps}
## Next steps {#next-steps}
If you want to explore further, here are some next steps to experiment with your dashboard

- Set up alerts for critical metrics (error rates, latency thresholds)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ pagination_prev: null
pagination_next: null
description: 'Monitoring Nginx Traces with ClickStack'
doc_type: 'guide'
keywords: ['ClickStack', 'nginx', 'traces', 'otel']
---

import Image from '@theme/IdealImage';
Expand All @@ -17,7 +18,7 @@ import view_traces from '@site/static/images/clickstack/nginx-traces-search-view

# Monitoring Nginx Traces with ClickStack {#nginx-traces-clickstack}

::::note[TL;DR]
:::note[TL;DR]
This guide shows you how to capture distributed traces from your existing nginx installation and visualize them in ClickStack. You'll learn how to:

- Add the OpenTelemetry module to nginx
Expand All @@ -26,7 +27,7 @@ This guide shows you how to capture distributed traces from your existing nginx
- Use a pre-built dashboard to visualize request performance (latency, errors, throughput)

Time Required: 5-10 minutes.
::::
:::

## Prerequisites {#prerequisites}
- ClickStack instance running with OTLP endpoints accessible (ports 4317/4318)
Expand Down Expand Up @@ -55,9 +56,9 @@ image: nginx:1.27-otel

This image includes the `ngx_otel_module.so` pre-installed and ready to use.

::::note
:::note
If you're running nginx outside of Docker, refer to the [OpenTelemetry nginx documentation](https://github.com/open-telemetry/opentelemetry-cpp-contrib/tree/main/instrumentation/nginx) for manual installation instructions.
::::
:::

## Configure nginx to send traces to ClickStack {#configure-nginx}

Expand Down Expand Up @@ -125,13 +126,13 @@ services:

Replace `<clickstack-host>` with your ClickStack instance hostname or IP address.

::::note
:::note
- **Port 4317** is the gRPC endpoint used by the nginx module
- **otel_service_name** should be descriptive of your nginx instance (e.g., "api-gateway", "frontend-proxy")
- Change **otel_service_name** to match your environment for easier identification in HyperDX
::::
:::

### Understanding the Configuration {#understanding-configuration}
### Understanding the configuration {#understanding-configuration}

**What gets traced:**
Each request to nginx creates a trace span showing:
Expand Down Expand Up @@ -225,19 +226,19 @@ curl -X POST http://localhost:4318/v1/traces \
-d @nginx-traces-sample.json
```

::::note[Running on localhost]
:::note[Running on localhost]
This demo assumes ClickStack is running locally on `localhost:4318`. For remote instances, replace `localhost` with your ClickStack hostname.
::::
:::

You should see a response like `{"partialSuccess":{}}` indicating the traces were successfully sent. All 1,000 traces will be ingested into ClickStack.

## Verify traces in HyperDX {#verify-demo-traces}

1. Open [HyperDX](http://localhost:8080/search?from=1761501600000&to=1761588000000&isLive=false&source=69023d1b4f1d41a964641b09&where=&select=Timestamp,ServiceName,StatusCode,round(Duration/1e6),SpanName&whereLanguage=lucene&orderBy=&filters=[])

::::note
:::note
It is important to use the link above to get the correct time range, if you don't use this link set your time range to Oct 26 13:00:00 - Oct 27 13:00:00 to see proper results.
::::
:::

Here's what you should see in your search view:

Expand All @@ -253,7 +254,7 @@ To help you get started monitoring traces with ClickStack, we provide essential

## <a href={useBaseUrl('/examples/example-traces.json')} download="example-traces.json">Download</a> the dashboard configuration. {#download}

## Import Pre-built Dashboard {#import-dashboard}
## Import the pre-built dashboard {#import-dashboard}
1. Open HyperDX and navigate to the Dashboards section.
2. Click "Import Dashboard" in the upper right corner under the ellipses.

Expand Down Expand Up @@ -307,7 +308,7 @@ Look for OpenTelemetry-related errors.
tail -f /var/log/nginx/access.log
```

## Next Steps {#next-steps}
## Next steps {#next-steps}
If you want to explore further, here are some next steps to experiment with your dashboard

- Set up alerts for critical metrics (error rates, latency thresholds)
Expand Down
2 changes: 1 addition & 1 deletion docusaurus.config.en.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ const config = {
onBrokenLinks: "throw",
onBrokenMarkdownLinks: "warn",
onDuplicateRoutes: "throw",
onBrokenAnchors: "warn",
onBrokenAnchors: process.env.ON_BROKEN_ANCHORS ?? "throw",
favicon: "img/docs_favicon.ico",
organizationName: "ClickHouse",
trailingSlash: false,
Expand Down