Skip to content

Latest commit

 

History

History
76 lines (48 loc) · 2.27 KB

07.1-streaming-intro.md

File metadata and controls

76 lines (48 loc) · 2.27 KB

<< back to main index

Lab 7.1: Kafka Streams - intro

Overview

Setup a Kafka streaming application

Depends On

Run time

20 mins

Step-1: Run Consumer

On console...

Use kafkacat

$    kafkacat -q -C -b localhost:9092 -t clickstream -f 'Partition %t[%p], offset: %o, key: %k, value: %s\n'

or console-consumer

$   ~/apps/kafka/bin/kafka-console-consumer.sh \
        --bootstrap-server localhost:9092 \
        --property print.key=true --property key.separator=":" \
        --topic clickstream

Step-2: Clickstream Producer

File : src/main/java/x/utils/ClickStreamProducer.java

We will continue use this producer

  • Inspect file and make any fixes
  • Run the producer in Eclipse, Right click on the file and run as 'Java Application'
  • Make sure it is sending messages as follows
    • key : Domain
    • value : clickstream data
    • example:
  key=facebook.com, value={"timestamp":1451635200005,"session":"session_251","domain":"facebook.com","cost":91,"user":"user_16","campaign":"campaign_5","ip":"ip_67","action":"clicked"}

Step-3: Streaming Consumer 1

This consumer will read and print a KafkaStream.

File : src/main/java/x/lab07_streams/StreamsConsumer1.java

  • Inspect the file and fix the TODO items.
    Use reference Kafka Java API

Run the lab07_streams/StreamsConsumer1 in Eclipse

Step-4: Run ClickStreamProducer

Run the utils.ClickStreamProducer in Eclipse

Expected output

[INFO ] 2018-07-22 14:15:41.248 [main] StreamsConsumer1:main(48) - kstreams starting on clickstream

[KSTREAM-SOURCE-0000000000]: facebook.com, {"timestamp":1451635200005,"session":"session_251","domain":"facebook.com","cost":91,"user":"user_16","campaign":"campaign_5","ip":"ip_67","action":"clicked"}

[KSTREAM-SOURCE-0000000000]: foxnews.com, {"timestamp":1451635200010,"session":"session_224","domain":"foxnews.com","cost":17,"user":"user_89","campaign":"campaign_4","ip":"ip_57","action":"viewed"}

[KSTREAM-SOURCE-0000000000]: facebook.com, {"timestamp":1451635200015,"session":"session_160","domain":"facebook.com","cost":73,"user":"user_53","campaign":"campaign_1","ip":"ip_20","action":"blocked"}