Skip to content

Latest commit

 

History

History
47 lines (32 loc) · 1.44 KB

04.3-producer-compress.md

File metadata and controls

47 lines (32 loc) · 1.44 KB

<< back to main index

Lab 4.2: Compression Benchmarking

Overview

Understand different compression codecs available for compression of messages and their performance

Depends On

lab 3

Run time

30 mins

Step 1 : Create a Compression topic

    $   ~/apps/kafka/bin/kafka-topics.sh  --bootstrap-server localhost:9092  --create \
        --topic compression --replication-factor 1  --partitions 2

Step 2 : Producer

  • Inspect file : src/main/java/x/lab04_benchmark/BenchmarkCompressionSchemes.java
  • Fix all TODO items, using Eclipse (or any other editor).

Step 2 : Run the producer

  • Right click on 'src/main/java/x/lab04_benchmark/BenchmarkCompressionSchemes.java'
  • Run as 'Java Application'

Step 3: Record producer times for this codec

  • Producer console output will show total time and throughput. Copy this output to notepad/excel

Step 4: Try other compression codecs

  • Edit BenchmarkCompressionSchemes.java and change the codec
  • Re-run Step 3 above and record benchmark times for this codec

Step 5: Create performance table

  • Create a table showing performance numbers for different codecs

Bonus lab :

Compression can help to reduce the size of data we are sending out.
Can you figure out a way to find the amount of data sent by the producer?
You may want to use OS level tools to measure this?
Share what you find with the class.