Skip to content

Commit 2f560cb

Browse files
committed
docs: add cardano to kafka example
1 parent 3ac89e4 commit 2f560cb

File tree

2 files changed

+69
-0
lines changed

2 files changed

+69
-0
lines changed

README.md

+2
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ Similar to the well-known db-sync tool provided by IOHK, _Oura_ can be used as a
4646

4747
Given its small memory / cpu footprint, _Oura_ can be deployed side-by-side with your Cardano node even in resource-constrained environments, such as Raspberry PIs.
4848

49+
For an example of how to use _Oura_ as a bridge, check the [Cardano => Kafka](docs/kafka.md) setup instructions.
50+
4951
### As a Trigger of Custom Actions
5052

5153
_Oura_ running in `daemon` mode can be configured to use custom filters to pinpoint particular transaction patterns and trigger actions whenever it finds a match. For example: send an email when a particular policy / asset combination appears in a transaction; call an AWS Lambda function when a wallet delegates to a particular pool; send a http-call to a webhook each time a metadata key appears in the TX payload;

docs/kafka.md

+67
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Cardano => Kafka Example
2+
3+
This example shows how to leverage _Oura_ to stream data from a Cardano node into a _Kafka_ topic.
4+
5+
## About Kafka
6+
7+
> Apache Kafka is a framework implementation of a software bus using stream-processing. It is an open-source software platform developed by the Apache Software Foundation written in Scala and Java.
8+
9+
Find [more info](https://en.wikipedia.org/wiki/Apache_Kafka) about _Kafka_ in wikipedia.
10+
11+
## Prerequisites
12+
13+
This examples assumes the following prerequisites:
14+
15+
- A running Cardano node locally accesible via a unix socket.
16+
- A Kafka cluster accesible through the network.
17+
- An already existing Kafka topic where to output events
18+
- _Oura_ binary release installed in local system
19+
20+
## Instructions
21+
22+
### 1. Create an Oura configuration file `caradno2kafka.toml`
23+
24+
```toml
25+
[source]
26+
type = "N2C"
27+
address = ["Unix", "/opt/cardano/cnode/sockets/node0.socket"]
28+
magic = "testnet"
29+
30+
[sink]
31+
type = "Kafka"
32+
brokers = ["kafka-broker-0:9092"]
33+
topic = "cardano-events"
34+
```
35+
36+
Some comments regarding the above configuration:
37+
38+
- the `[source]` section indicates _Oura_ from where to pull chain data.
39+
- the `N2C` source type value tells _Oura_ to get the data from a Cardano node using Node-to-Client mini-protocols (chain-sync instantiated to full blocks).
40+
- the `address` field indicates that we should connect via `Unix` socket at the specified path. This value should match the location of your local node socket path.
41+
- the `magic` field indicates that our node is running in the `testnet` network. Change this value to `mainnet` if appropriate.
42+
- the `[sink]` section tells _Oura_ where to send the information it gathered.
43+
- the `Kafka` sink type value indicates that _Oura_ should use a _Kafka_ producer client as the output
44+
- the `brokers` field indicates the location of the _Kafka_ brokers within the network. Several hostname:port pairs can be added to the array for a "cluster" scenario.
45+
- the `topic` fields indicates which _Kafka_ topic to used for the outbound messages.
46+
47+
### 2. Run _Oura_ in `daemon` mode
48+
49+
Run the following command from your terminal to start the daemon process:
50+
51+
```sh
52+
RUST_LOG=info oura daemon --config cardano2kafka.toml
53+
```
54+
55+
You should see an output similar to the following:
56+
57+
```
58+
[2021-12-13T22:16:43Z INFO oura::sources::n2n::setup] handshake output: Accepted(7, VersionData { network_magic: 764824073, initiator_and_responder_diffusion_mode: false })
59+
[2021-12-13T22:16:43Z INFO oura::sources::n2n::setup] chain point query output: Some(Tip(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"), 6624258))
60+
[2021-12-13T22:16:43Z INFO oura::sources::n2n::setup] node tip: Point(47867448,"f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6")
61+
[2021-12-13T22:16:44Z INFO oura::sources::n2n] rolling block to point Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6")
62+
[2021-12-13T22:16:52Z INFO oura::sources::n2n] requesting block fetch for point Some(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"))
63+
[2021-12-13T22:17:15Z INFO oura::sources::n2n] requesting block fetch for point Some(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"))
64+
[2021-12-13T22:17:20Z INFO oura::sources::n2n] requesting block fetch for point Some(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"))
65+
```
66+
67+

0 commit comments

Comments
 (0)