Skip to content

Latest commit

 

History

History
77 lines (61 loc) · 2.29 KB

08.3-ksql-intro.md

File metadata and controls

77 lines (61 loc) · 2.29 KB

<< back to main index

Lab 1 : Run KSQL

Step 1: Start the KSQL CLI from Home directory

First create a directory where KSQL will store logs

    $ cd confluent; mkdir -p ksql_logs
    $ cd confluent
    $ LOG_DIR=./ksql_logs bin/ksql

You should now be in the KSQL CLI

ksql>

Step 2: Create a KSQL Stream

ksql> CREATE STREAM clickviews 
WITH  (KAFKA_TOPIC='avroclicks', VALUE_FORMAT='AVRO');

ksql> SHOW streams;
 Stream Name | Kafka Topic | Format 
------------------------------------
 CLICKVIEWS  | avroclicks  | AVRO   
------------------------------------

ksql>DESCRIBE clickviews;

Name                 : CLICKVIEWS
 Field     | Type                      
---------------------------------------
 ROWTIME   | BIGINT           (system) 
 ROWKEY    | VARCHAR(STRING)  (system) 
 TIMESTAMP | BIGINT                    
 SESSION   | VARCHAR(STRING)           
 DOMAIN    | VARCHAR(STRING)           
 USER      | VARCHAR(STRING)           
 COST      | BIGINT                    
 CAMPAIGN  | VARCHAR(STRING)           
 IP        | VARCHAR(STRING)           
---------------------------------------

ksql> select * from clickviews;

The select statement will wait for data to appear in the stream. Leave it running.

Step 3: Start the AvroClickstreamProducer from Lab 8.2

Refer to instructions in that lab to start it

In the ksql terminal, you should start seeing output like:

1548522101848 | session_251 | 1451635200005 | session_251 | facebook.com | user_16 | 91 | campaign_5 | ip_67
1548522102856 | session_224 | 1451635200010 | session_224 | foxnews.com | user_89 | 17 | campaign_4 | ip_57
1548522103857 | session_160 | 1451635200015 | session_160 | facebook.com | user_53 | 73 | campaign_1 | ip_20
1548522104857 | session_66 | 1451635200020 | session_66 | cnn.com | user_29 | 31 | campaign_3 | ip_49
...

Now terminate the SELECT statement using Ctrl-C.

Step 4: Create a KSQL Table to count number of clicks by domain

ksql> CREATE TABLE click_counts AS
      SELECT domain, count(*) FROM clickviews 
          GROUP BY domain;

ksql> SELECT * from click_counts;

Leave the select running and start the AvroClickstreamProducer again.

What output are you seeing in KSQL?