Skip to content

Latest commit

 

History

History
81 lines (53 loc) · 1.98 KB

07.5-streaming-groupby.md

File metadata and controls

81 lines (53 loc) · 1.98 KB

<< back to main index

Lab 7.5: Kafka Streaming - Groupby

Overview

Apply groupby transformation on KStreams

Depends On

7.1 streaming intro

Run time

20 mins

Step 1: Streaming Consumer 5

File : src/main/java/x/lab07_streams/StreamsConsumer5_GroupBy.java

This consumer will apply the following transformation to a KafkaStream

  • map the incoming record into key value pair with (action,1)
  • aggregate all actions by
    • groupby action
    • and counting them

=> Inspect the file and fix the TODO items.

Run the lab07_streams/StreamsConsumer5_GroupBy in Eclipse

Step 2: Run Producer

Run the utils.ClickStreamProducer in Eclipse

Expected output:

....

[KTABLE-TOSTREAM-0000000009]: clicked, 13
[KTABLE-TOSTREAM-0000000009]: blocked, 7
[KTABLE-TOSTREAM-0000000009]: viewed, 3
[KTABLE-TOSTREAM-0000000009]: clicked, 14
[KTABLE-TOSTREAM-0000000009]: blocked, 8
[KTABLE-TOSTREAM-0000000009]: clicked, 15
[KTABLE-TOSTREAM-0000000009]: blocked, 9
[KTABLE-TOSTREAM-0000000009]: clicked, 16
[KTABLE-TOSTREAM-0000000009]: clicked, 17
[KTABLE-TOSTREAM-0000000009]: blocked, 10

Step 3: Run Producer Again

Run the utils.ClickStreamProducer in Eclipse

Notice the counts, what do you see?

Step 4: Bonus Lab: Inspect action-count topic

In this bonus lab, instead of printing the results to console, we are going to write it out to another Kafka topic.

Once you make the code changes, you can inspect Kafka topic as follows

Using Kafkacat

$   kafkacat -q -C -b localhost:9092 -t action-count  -s key=s  -s value=q -f '%k:%s\n'

or using console consumer

  $   ~/apps/kafka/bin/kafka-console-consumer.sh\
      --bootstrap-server "localhost:9092"\
      --from-beginning\
      --property print.key=true\
      --property value.deserializer=org.apache.kafka.common.serialization.LongDeserializer\
      --topic action-count