Skip to content

Latest commit

 

History

History
189 lines (120 loc) · 3.44 KB

08.1-install-confluent.md

File metadata and controls

189 lines (120 loc) · 3.44 KB

<< back to main index

Lab: Install Confluent

Overview

Install and run Confluent platform

Depends On

None

Run time

15 mins

Step 1: Shutdown any running Kafka instance

Run the following in a terminal

$   ~/apps/kafka/bin/kafka-server-stop.sh

$   ~/apps/kafka/bin/zookeeper-server-stop.sh

Verify the services have stopped by doing a jps command

$   jps

If the above commands fail, send kill signal to kafka and zookeeper pids. Here is an example

$   jps
1111  QuroumPeerMain
2222  Kafka
3333  Worker

Then send kills signal as follwos

# kill kafka first
# TODO: Adjust PID to match yours
$   kill 2222

Use jps to make sure kafka process is terminated.

Then kill zookeeper

# TODO: Adjust PID to match yours
$   kill 1111

And also kill Worker

$   kill 3333

Make sure the output doesn't have QuroumPeerMain or Kafka or Worker

Also cleanup storage directories

$   rm -rf  /tmp/zookeeper*
$   rm -rf  /tmp/kafka-logs*

Step 2 : Download confluent

$   cd   ~/apps

$   wget http://packages.confluent.io/archive/7.2/confluent-7.2.0.tar.gz

# alternate download
# $   wget https://elephantscale-public.s3.amazonaws.com/downloads/confluent-7.2.0.tar.gz

$   tar xf  confluent-7.2.0.tar.gz

$   mv  confluent-7.2.0    confluent

Now Confluent is installed under ~/apps/confluent

Step 3 : Run Confluent Stack

Start the following components. You need to open new terminals, run the commands and leave them running:

3.1: Start zookeeper (terminal-1 )

# be in confluent directory
$   cd  ~/apps/confluent

$   ./bin/zookeeper-server-start  ./etc/kafka/zookeeper.properties

Leave this terminal-1 running, do not close it!

3.2: Start Kafka (terminal-2)

# be in confluent directory
$   cd  ~/apps/confluent

$   JMX_PORT=9999   ./bin/kafka-server-start  ./etc/kafka/server.properties

Leave this terminal-2 running, do not close it!

3.3: start Schema Registry (terminal-3)

# be in confluent directory
$   cd  ~/apps/confluent

$   ./bin/schema-registry-start  ./etc/schema-registry/schema-registry.properties

Leave this terminal-3 running, do not close it!

Troubleshooting

Schema registry uses port 8081. Sometimes other programs like 'Spark worker' may be using the same port. If that's the case use the following to stop Worker

$   jps
5555   Worker

Now send a kill signal to Spark worker

# TODO: Adjust the PID to match yours!
$   kill 5555

And then restart Schema registry again.

Step-4: Verify Confluent Stack is Running

Open a new terminal and run this command

$   jps

You might see output like this:

2812156 QuorumPeerMain
2815132 SchemaRegistryMain
2813284 Kafka
2817717 Jps

Step-5: Register Cluster in Kafka Manager

You can optionally register the cluster in Kafka Manager

(Optional) Step-6: Confluent Control Center

Open another terminal and run the following:

$   cd ~/apps/confluent

$   ./bin/control-center-start  ./etc/confluent-control-center/control-center-minimal.properties

Leave this terminal up

And from VNC, launch a browser and go to localhot:9021 to see confluent control center UI