IT Depends #8 CDC Workshop

Prequisites

To run this workshop you need several tools installed:

1. The Docker Engine
2. The Docker Compose

How to

Run the application

# start all services defined in the docker-compose.yaml
docker-compose -f docker-compose.yaml up

Connect to the database

Linux / macOS (bash):

docker-compose -f docker-compose.yaml exec postgres env PGOPTIONS="--search_path=campaigns" bash -c 'psql -U $POSTGRES_USER postgres'

Consume messages from the Kafka topic

Linux / macOS (bash):

docker-compose -f docker-compose.yaml exec kafka /kafka/bin/kafka-console-consumer.sh \
    --bootstrap-server kafka:9092 \
    --from-beginning \
    --property print.key=true \
    --topic dbserver1.campaigns.customers

Windows (PowerShell):

docker-compose -f docker-compose.yaml exec kafka /kafka/bin/kafka-console-consumer.sh `
    --bootstrap-server kafka:9092 `
    --from-beginning `
    --property print.key=true `
    --topic dbserver1.campaigns.customers

List available Kafka topics

docker-compose -f docker-compose.yaml exec kafka /kafka/bin/kafka-topics.sh --bootstrap-server=kafka:9092 --list

Add Debezium connector

Linux / macOS (bash):

curl -i -X POST -H "Accept:application/json" -H  "Content-Type:application/json" http://localhost:8083/connectors/ -d @connector-config.json

Windows (PowerShell):

curl -i -X POST -H "Accept:application/json" -H  "Content-Type:application/json" http://localhost:8083/connectors/ -d @connector-config.json

List DB connectors

Linux / macOS (bash):

curl -i -X GET -H "Accept:application/json" -H  "Content-Type:application/json" http://localhost:8083/connectors/

Windows (PowerShell):

curl -i -X GET -H "Accept:application/json" -H  "Content-Type:application/json" http://localhost:8083/connectors/

Useful documentation

• Quick guide to psql / PostgreSQL
• Debezium
• Debezium PostgreSQL connector
• Kafka

TODOs

1. DB diagram
2. Add connector curl command + Powershell?

Acknowledgements

For the workshop we used the config files and images from the Debezium Team Repositories. Cheers!