diff --git a/content/docs/configuration/output.md b/content/docs/configuration/output.md index ca791275d..9632d0eb5 100644 --- a/content/docs/configuration/output.md +++ b/content/docs/configuration/output.md @@ -3,7 +3,7 @@ title: Output and ClusterOutput weight: 50 --- -Outputs are the destinations where your log forwarder sends the log messages, for example, to Sumo Logic, or to a file. Depending on which log forwarder you use, you have to configure different custom resources. +Outputs are the destinations where your log forwarder sends the log messages, for example, to a file. Depending on which log forwarder you use, you have to configure different custom resources. ## Fluentd outputs diff --git a/content/docs/configuration/plugins/_index.md b/content/docs/configuration/plugins/_index.md index f4ade1654..9a4bc36d5 100644 --- a/content/docs/configuration/plugins/_index.md +++ b/content/docs/configuration/plugins/_index.md @@ -14,7 +14,6 @@ For more information please click on the plugin name | **[Dedot](filters/dedot/)** | filters | Concatenate multiline log separated in multiple events | GA | [1.0.0](https://github.com/lunardial/fluent-plugin-dedot_filter) | | **[Exception Detector](filters/detect_exceptions/)** | filters | Exception Detector | GA | [0.0.14](https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions) | | **[ElasticsearchGenId](filters/elasticsearch_genid/)** | filters | | | []() | -| **[Enhance K8s Metadata](filters/enhance_k8s/)** | filters | Fluentd output plugin to add extra Kubernetes metadata to the events. | GA | [2.0.0](https://github.com/SumoLogic/sumologic-kubernetes-collection/tree/main/fluent-plugin-enhance-k8s-metadata) | | **[Geo IP](filters/geoip/)** | filters | Fluentd GeoIP filter | GA | [1.3.2](https://github.com/y-ken/fluent-plugin-geoip) | | **[Grep](filters/grep/)** | filters | Grep events by the values | GA | [more info](https://docs.fluentd.org/filter/grep) | | **[Kubernetes Events Timestamp](filters/kube_events_timestamp/)** | filters | Fluentd Filter plugin to select particular timestamp into an additional field | GA | [0.1.4](https://github.com/banzaicloud/fluentd-filter-kube-events-timestamp) | @@ -23,7 +22,6 @@ For more information please click on the plugin name | **[Record Modifier](filters/record_modifier/)** | filters | Modify each event record. | GA | [2.1.0](https://github.com/repeatedly/fluent-plugin-record-modifier) | | **[Record Transformer](filters/record_transformer/)** | filters | Mutates/transforms incoming event streams. | GA | [more info](https://docs.fluentd.org/filter/record_transformer) | | **[Stdout](filters/stdout/)** | filters | Prints events to stdout | GA | [more info](https://docs.fluentd.org/filter/stdout) | -| **[SumoLogic](filters/sumologic/)** | filters | Sumo Logic collection solution for Kubernetes | GA | [2.3.1](https://github.com/SumoLogic/sumologic-kubernetes-collection) | | **[Tag Normaliser](filters/tagnormaliser/)** | filters | Re-tag based on log metadata | GA | [0.1.1](https://github.com/banzaicloud/fluent-plugin-tag-normaliser) | | **[Throttle](filters/throttle/)** | filters | A sentry plugin to throttle logs. Logs are grouped by a configurable key. When a group exceeds a configuration rate, logs are dropped for this group. | GA | [0.0.5](https://github.com/rubrikinc/fluent-plugin-throttle) | | **[Amazon Elasticsearch](outputs/aws_elasticsearch/)** | outputs | Fluent plugin for Amazon Elasticsearch | Testing | [2.4.1](https://github.com/atomita/fluent-plugin-aws-elasticsearch-service) | @@ -52,7 +50,6 @@ For more information please click on the plugin name | **[Amazon S3](outputs/s3/)** | outputs | Store logs in Amazon S3 | GA | [1.6.1](https://github.com/fluent/fluent-plugin-s3/releases/tag/v1.6.1) | | **[Splunk Hec](outputs/splunk_hec/)** | outputs | Fluent Plugin Splunk Hec Release | GA | [1.2.9]() | | **[SQS](outputs/sqs/)** | outputs | Output plugin writes fluent-events as queue messages to Amazon SQS | Testing | [v2.1.0](https://github.com/ixixi/fluent-plugin-sqs) | -| **[SumoLogic](outputs/sumologic/)** | outputs | Send your logs to Sumologic | GA | [1.8.0](https://github.com/SumoLogic/fluentd-output-sumologic/releases/tag/1.8.0) | | **[Syslog](outputs/syslog/)** | outputs | Output plugin writes events to syslog | GA | [0.9.0.rc.8](https://github.com/cloudfoundry/fluent-plugin-syslog_rfc5424) | diff --git a/content/docs/examples/logging_output_sumologic.yaml b/content/docs/examples/logging_output_sumologic.yaml deleted file mode 100644 index 481b495de..000000000 --- a/content/docs/examples/logging_output_sumologic.yaml +++ /dev/null @@ -1,14 +0,0 @@ -apiVersion: logging.banzaicloud.io/v1beta1 -kind: Output -metadata: - name: sumologic-output-sample -spec: - sumologic: - endpoint: - valueFrom: - secretKeyRef: - name: sumologic - key: endpoint - log_format: json - source_category: prod/someapp/logs - source_name: AppA diff --git a/content/docs/examples/sumologic.md b/content/docs/examples/sumologic.md deleted file mode 100644 index ef999745b..000000000 --- a/content/docs/examples/sumologic.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -title: Sumo Logic with Logging operator and Fluentd -linktitle: Sumo Logic with Fluentd -weight: 300 -aliases: - - /docs/one-eye/logging-operator/quickstarts/sumologic/ ---- - -This guide walks you through a simple Sumo Logic setup using the Logging operator. -Sumo Logic has Prometheus and logging capabilities as well. Now we only focus on the logging part. - -## Configuration - -There are 3 crucial plugins needed for a proper Sumo Logic setup. - -1. Kubernetes metadata enhancer -2. Sumo Logic filter -3. Sumo Logic output - -Let's setup the logging first. - -### GlobalFilters - -The first thing we need to ensure is that the `EnhanceK8s` filter is present in the `globalFilters` section of the Logging spec. -This adds additional data to the log lines (like deployment and service names). - -```bash -kubectl apply -f - <<"EOF" -apiVersion: logging.banzaicloud.io/v1beta1 -kind: Logging -metadata: - name: sumologic -spec: - controlNamespace: logging - enableRecreateWorkloadOnImmutableFieldChange: true - globalFilters: - - enhanceK8s: {} - fluentbit: - bufferStorage: - storage.backlog.mem_limit: 256KB - inputTail: - Mem_Buf_Limit: 256KB - storage.type: filesystem - metrics: - serviceMonitor: true - serviceMonitorConfig: {} - fluentd: - disablePvc: true - metrics: - serviceMonitor: true - serviceMonitorConfig: {} -EOF -``` - -### ClusterFlow - -Now we can create a ClusterFlow. Add the Sumo Logic filter to the `filters` section of the ClusterFlow spec. -It will use the Kubernetes metadata and moves them to a special field called `_sumo_metadata`. -All those moved fields will be sent as HTTP Header to the Sumo Logic endpoint. - -> Note: As we are using Fluent Bit to enrich Kubernetes metadata, we need to specify the field names where this data is stored. - -```bash -kubectl -n logging apply -f - <<"EOF" -apiVersion: logging.banzaicloud.io/v1beta1 -kind: ClusterFlow -metadata: - name: sumologic -spec: - filters: - - sumologic: - source_name: kubernetes - log_format: fields - tracing_namespace: namespace_name - tracing_pod: pod_name - match: - - select: {} - globalOutputRefs: - - sumo -EOF -``` - -### ClusterOutput - -Create a Sumo Logic output secret from the URL. - -```bash -kubectl create secret generic logging-sumo -n logging --from-literal "sumoURL=https://endpoint1.collection.eu.sumologic.com/......" -``` - -Finally create the Sumo Logic output. - -```bash -kubectl -n logging apply -f - <<"EOF" -apiVersion: logging.banzaicloud.io/v1beta1 -kind: ClusterOutput -metadata: - name: sumo -spec: - sumologic: - buffer: - flush_interval: 10s - flush_mode: interval - endpoint: - valueFrom: - secretKeyRef: - name: logging-sumo - key: sumoURL - source_name: kubernetes -EOF -```