fluent-bit kafka data pipeline is not working. #100
Description
Hi Team,
We have configured the fluent-bit->Kafka data pipeline with the below mentioned link.
https://github.com/fluent/fluent-bit-kubernetes-logging/tree/master/output/kafka
When we have deployed this pipeline, pods are coming up, Kafka topic was created, but logs are not flowing to Kafka cluster.
We could not see any error in the pod logs, we are seeing below info.
#########################################################
[root@centos]# kubectl logs fluent-bit-nb886 -n ec2-kafka
Fluent Bit v1.5.7
- Copyright (C) 2019-2020 The Fluent Bit Authors
- Copyright (C) 2015-2018 Treasure Data
- Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
- https://fluentbit.io
[2022/09/12 07:11:40] [ info] [engine] started (pid=1)
[2022/09/12 07:11:40] [ info] [storage] version=1.0.5, initializing...
[2022/09/12 07:11:40] [ info] [storage] in-memory
[2022/09/12 07:11:40] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/09/12 07:11:40] [ info] [filter:kubernetes:kubernetes.0] https=1 host=kubernetes.default.svc port=443
[2022/09/12 07:11:40] [ info] [filter:kubernetes:kubernetes.0] local POD info OK
[2022/09/12 07:11:40] [ info] [filter:kubernetes:kubernetes.0] testing connectivity with API server...
[2022/09/12 07:11:40] [ info] [filter:kubernetes:kubernetes.0] API server connectivity OK
[2022/09/12 07:11:40] [ info] [output:kafka:kafka.0] brokers='x.x.x.x:9092,x.x.x.x:9092,x.x.x.x:9092' topics='kafka-bit'
[2022/09/12 07:11:40] [ info] [sp] stream processor started
##########################################################################
Please let us know how to solve this issue.
Thanks in advance..!!