05.1-manual-offset.md

Lab 5.1: Manual Offset

Overview

Seek within partition

Depends On

Run time

20 mins

Step-1 : Consumer

Inspect file : src/main/java/x/lab05_offsets/ManualOffsetConsumer.java

Fix the TODO-1

Disable auto commit like this

props.put("enable.auto.commit", "false");

Run the consumer in Eclipse, by

• Right click on file
• Run as 'Java Application'

Step-2: Inspect the output from Consumer

You will see consumer processing messages.

Pay special attention to OFFSETs.

Now stop the consumer program

Step-3: Rerun the consumer

Note: It may take a few seconds to establish connection to Kafka (because we didn't do a clean shutdown)

Observe the consumer output, you will see the same messages. Can you explain why?

Stop the consumer

Step-4: Enable commitSync

Uncomment the following line to enable commitSync

consumer.commitSync();

Step-5: Rerun Consumer

• Re-run consumer again
• Observe what data it is getting
• stop the consumer
• Start the consumer again.
• Observe the data. What do you see? Is it still getting 'old data'?

Step-6: Send Some New Data

Use kafkacat to send a couple of new messages into the topic

$    kafkacat -P -b localhost:9092   -t test

Type in some data

earth
mars

You should see it comes out via consumer.

Step-7: Discuss Your Findings