Skip to content

[Module components] Add provides receivers module#15

Open
qclaogui wants to merge 7 commits intografana:mainfrom
qclaogui:main
Open

[Module components] Add provides receivers module#15
qclaogui wants to merge 7 commits intografana:mainfrom
qclaogui:main

Conversation

@qclaogui
Copy link

@qclaogui qclaogui commented Apr 17, 2024

What this PR does

This PR comes from receivers provider, a module provides receivers components for collecting data(logs metrics traces profiles).

An simple use case config.alloy like so:

import.git "provider" {
  repository     = "https://github.com/qclaogui/codelab-monitoring.git"
  revision       = "main"
  path           = "alloy-modules/provider"
  pull_frequency = "24h"
}

// get the receivers from provider
provider.self_hosted_stack "compose" {
  metrics_endpoint_url  = "http://mimir:8080/api/v1/push"
}

// scrape metrics and write to metric receiver
prometheus.scrape "default" {
  targets = [
    {"__address__" = "127.0.0.1:12345"},
  ]

  forward_to = [
    provider.self_hosted_stack.compose.metrics_receiver,
  ]
}

Here is an example of my use of all-in-one.alloy in a docker compose scenario:

logging {
	level  = coalesce(env("ALLOY_LOG_LEVEL"), "info")
	format = "logfmt"
}

/********************************************
 * Grafana LGTMP Stack Receiver Provider
 ********************************************/
import.git "provider" {
	repository     = "https://github.com/qclaogui/codelab-monitoring.git"
	revision       = "main"
	path           = "alloy-modules/provider"
	pull_frequency = "24h"
}

+provider.self_hosted_stack "compose" {
+	metrics_endpoint_url  = coalesce(env("SELF_HOSTED_METRICS_ENDPOINT_URL"), "http://gateway:8080/api/v1/push")
+	logs_endpoint_url     = coalesce(env("SELF_HOSTED_LOGS_ENDPOINT_URL"), "http://gateway:3100/loki/api/v1/push")
+	traces_endpoint_url   = coalesce(env("SELF_HOSTED_TRACES_ENDPOINT_URL"), "http://gateway:4318")
+	profiles_endpoint_url = coalesce(env("SELF_HOSTED_PROFILES_ENDPOINT_URL"), "http://gateway:4040")
+}

/********************************************
 * Metrics
 ********************************************/
import.file "metrics" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/metrics"
}

metrics.labels_scrape "compose" {
	label_prefix    = "metrics.grafana.com"
+	forward_to      = [provider.self_hosted_stack.compose.metrics_receiver]
	scrape_interval = "15s"
}

/********************************************
 * Logs
 ********************************************/
import.file "logs" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/logs"
}

logs.labels_scrape "compose" {
	label_prefix = "logs.grafana.com"
	forward_to   = [logs.keep_labels.compose.receiver]
}

// logs Processing And Transformation
logs.keep_labels "compose" {
+	forward_to = [provider.self_hosted_stack.compose.logs_receiver]
}

/********************************************
 * Traces
 ********************************************/
import.file "traces" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/traces"
}

// traces Processing And Transformation process_and_transform
traces.process_and_transform "compose" {
+	metrics_forward_to = [provider.self_hosted_stack.compose.metrics_receiver]
+	logs_forward_to    = [provider.self_hosted_stack.compose.logs_receiver]
+	traces_forward_to  = [provider.self_hosted_stack.compose.traces_receiver]
}

tracing {
	// Write all spans. Don't do this in production!
	sampling_fraction = 1

	// Forward Alloy internal spans to traces process.
	write_to = [traces.process_and_transform.compose.alloy_traces_input]
}

/********************************************
 * Profiles
 ********************************************/
import.file "profiles" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/profiles"
}

profiles.labels_scrape "compose" {
	label_prefix = "profiles.grafana.com"
+	forward_to   = [provider.self_hosted_stack.compose.profiles_receiver]
}

Which issue(s) this PR fixes or relates to

#12
In addition, I also add the grafana_cloud component in the receivers provider moudle

Signed-off-by: Weifeng Wang <qclaogui@gmail.com>
@bentonam
Copy link
Collaborator

bentonam commented May 2, 2024

@qclaogui thank you for the contribution. Would you mind ensuring that the CLA is signed, all checks should pass, it is currently failing the markdownlint checks. Also please ensure the branch is up-to-date. Thanks!

@CLAassistant
Copy link

CLAassistant commented May 3, 2024

CLA assistant check
All committers have signed the CLA.

@qclaogui
Copy link
Author

qclaogui commented May 4, 2024

it is currently failing the markdownlint checks. Also please ensure the branch is up-to-date.

Hi @bentonam, Thank you for taking the time to reviews, I have run make lint-markdown and it has passed the markdownlint check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants