5.8 Running the Lab

Run the Lab with Docker Compose

We have finished our work on the environment and our order microservice. We are now ready to start the environment and setup our debezium connectors.

Running

If you have any previous lab running stop these containers

1
docker-compose down --remove-orphans

Task 5.8.1 - Starting with Docker Compose

Enter a shell and change your directory to the <PROJECT_ROOT>/code/debezium/docker folder.

1
cd <PROJECT_ROOT>/code/debezium/docker

Start the microservices with

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
$ docker-compose up -d
Starting debezium-zookeeper    ... done
Starting debezium-order-db     ... done
Starting debezium-grafana      ... done
Starting debezium-prometheus   ... done
Starting debezium-alertmanager ... done
Starting debezium-jaeger       ... done
Starting debezium-stock-db     ... done
Starting debezium-order        ... done
Starting debezium-stock        ... done
Starting debezium-kafka        ... done
Starting debezium-connect      ... done
Starting debezium-viewer       ... done

Debezium Connectors

With starting the environment we have also started Kafka Connect. We need to configure our Debezium PostgreSQL Connector to listen for events on the order-db and stock-db database.

Task 5.8.2 - Configuring Debezium Connectors

Configuring the Debezium PostgreSQL Connector for the order-db database.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
curl --request POST \
  --url http://localhost:8083/connectors \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "order-events",
    "config": {
      "connector.class" : "io.debezium.connector.postgresql.PostgresConnector",
      "tasks.max" : "1",
      "database.hostname" : "order-db",
      "database.port" : "5432",
      "database.user" : "admin",
      "database.password" : "1234",
      "database.dbname" : "admin",
      "database.server.name" : "order-db",
      "schema.include.list" : "public",
      "table.include.list" : "public.outboxevent",
      "tombstones.on.delete" : "false",
      "transforms" : "outbox,topiccase",
      "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter",
      "transforms.outbox.route.by.field" : "type",
      "transforms.outbox.route.topic.replacement" : "order-${routedByValue}-events",
      "transforms.outbox.table.field.event.timestamp" : "timestamp",
      "transforms.outbox.table.fields.additional.placement" : "type:header:eventType",
      "transforms.topiccase.type" : "ch.puzzle.mm.debezium.connect.smt.ChangeTopicCase",
      "transforms.topiccase.toCase" : "lower"
    }
}'

Configuring the Debezium PostgreSQL Connector for the stock-db database.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
curl --request POST \
  --url http://localhost:8083/connectors \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "stock-events",
    "config": {
      "connector.class" : "io.debezium.connector.postgresql.PostgresConnector",
      "tasks.max" : "1",
      "database.hostname" : "stock-db",
      "database.port" : "5432",
      "database.user" : "admin",
      "database.password" : "1234",
      "database.dbname" : "admin",
      "database.server.name" : "stock-db",
      "schema.include.list" : "public",
      "table.include.list" : "public.outboxevent",
      "tombstones.on.delete" : "false",
      "transforms" : "outbox,topiccase",
      "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter",
      "transforms.outbox.route.by.field" : "type",
      "transforms.outbox.route.topic.replacement" : "stock-${routedByValue}-events",
      "transforms.outbox.table.field.event.timestamp" : "timestamp",
      "transforms.outbox.table.fields.additional.placement" : "type:header:eventType",
      "transforms.topiccase.type" : "ch.puzzle.mm.debezium.connect.smt.ChangeTopicCase",
      "transforms.topiccase.toCase" : "lower"
    }
}'

As you can see in the configuration of the connectors we specify our connection information and our tables with the table.include.list and our transforms. We use the EventRouter transformation which handles our outboxevent table.

Check if the containers are created

1
2
3
4
5
6
7
8
curl --request GET \
  --url http://127.0.0.1:8083/connectors \
  --header 'Accept: application/json'

[
  "order-events",
  "stock-events"
]

Sample Requests

We provide a requests/api.http file containing all requests needed for this lab. This file is compatible with IntelliJ IDEA and also includes the curl version for the requests.

Get article stock information

1
curl --request GET --url http://localhost:8081/article-stocks

Get order information

1
curl --request GET --url http://localhost:8080/shop-orders

Create new order

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
curl --request POST \
  --url http://localhost:8080/shop-orders \
  --header 'Content-Type: application/json' \
  --data '{
    "articleOrders" : [
        {
            "articleId" : 1,
            "amount" : 1
        }
    ]
}'

Task 5.8.3 - Using the application

There is a Viewer Microservice running which provides an overview of events in the Kafka Topics. It uses websockets and shows newly arriving events.

Feel free to test the following scenarios:

  • Order Article 1 with quantity 9 (e.g. orderId: 100000)
  • Order Article 1 with quantity 3

What do you expect? Was the second order rejected?

What happens if you proceed like this?

  • Cancel OrderId 100000
  • Order Article 1 with quantity 3

What do you expect? Was the second order rejected?

What happens if you proceed like this?

  • Delete the connectors (order-events and stock-events)
  • Order Article 1 with quantity 3
  • Order Article 1 with quantity 3
  • Order Article 1 with quantity 3
  • Recreate the connectors (order-events and stock-events)

Were there any errors when the connectors were down? Do the microservices catch up after recreating?

What does the Jaeger Tracing show?