Skip to content

Latest commit

 

History

History
138 lines (88 loc) · 2.85 KB

02.2-kafkacat.md

File metadata and controls

138 lines (88 loc) · 2.85 KB

<< back to main index

Lab 2.2: Kafkacat

Overview

kafkacat is really useful tool to deal with Kafka. In this lab, we will setup Kafkacat and use it

Run time

15 minutes

Step-1: Install kafkacat

On ubuntu systems:

$   sudo apt update

$   sudo apt install -y kafkacat

Step-2: Get Kafka Cluster Info

$   kafkacat -L   -b localhost:9092

Step-3: Producer / Consumer mode

On terminal-1 start Kafkacat in producer mode

$   kafkacat -P -b localhost:9092   -t test

On another terminal(2) start kafkacat in consumer mode

# wait for new messages
$   kafkacat -C -b localhost:9092   -t test

Then go ahead and type some messages in terminal-1 (producer) and observe the output on consumer terminal (2)

Step-4: More Formatting Options

Close the kafkacat consumer by hitting Ctrl+C on terminal-2

And try the following:

$   kafkacat -q -C -b localhost:9092 -t test -f 'Partition %t[%p], offset: %o, key: %k, value: %s\n'

Steps below are optional...

Step-5: More Options for Producing

Produce messages, read from STDIN to topic test

$   kafkacat -P -b localhost:9092   -t test

Or you can pipe a file into a topic

$   cat README.txt   |  kafkacat -P  -b localhost:9092  -t test

Producing with key-values

$   kafkacat -P -b localhost:9092 -t test -K :
# Then type
#    k1:v1
#    k2:v2

Produce from a file. Say our file data.txt has data in this format:

k1:v1
k2:v2
k3:v3

Send it like this

$   kafkacat -P -b localhost:9092 -t test -K: -l data.txt

Step-6: More Options for Consuming

# quiet mode
$   kafkacat -q -C -b localhost:9092   -t test

# wait for new messages
$   kafkacat -C -b localhost:9092   -t test

# exit after reading all messages
$   kafkacat -C -b localhost:9092   -t test  -e

# only read last 10 messages and exit
$   kafkacat -C -b localhost:9092   -t test  -o -10 -e

More formatting options

# print key, value
$   kafkacat -C -b localhost:9092 -t test -f 'key: %k, value: %s \n'

# more detailed output
$   kafkacat -C -b localhost:9092 -t test -f 'Topic %t[%p], offset: %o, key: %k, value: %s, (length: %S bytes) \n'

Print timestamp of messages (%T)

$    kafkacat -q -C -b localhost:9092 -t test -f 'Partition %t[%p], offset: %o, timestamp: %T,  key: %k, value: %s\n'

More usage details see Kafkacat page

References