Skip to content

Event Stream Processor for Event Stream (logs, audits, errors, trace, etc) PoC

License

Notifications You must be signed in to change notification settings

mojaloop/event-stream-processor

Folders and files

NameName
Last commit message
Last commit date

Latest commit

45faab5 · Apr 13, 2022
Jul 1, 2021
Jul 8, 2021
May 14, 2021
Jul 19, 2019
Sep 4, 2019
Sep 4, 2019
Sep 4, 2019
Jul 1, 2021
Aug 20, 2021
Feb 13, 2020
Dec 9, 2020
Aug 20, 2021
Dec 19, 2019
Aug 20, 2021
Apr 13, 2022
Aug 20, 2021

Repository files navigation

event-stream-processor

EXPERIMENTAL Event Stream Processor (ESP) for Event Stream (logs, audits, errors, trace, etc)

Todo

  • Improve unit tests.
  • Improve code-coverage to 90% across the board: .nycrc.yml.
  • Add NPM script test as a pre-commit in the package.json.
  • Update CI-CD to include unit and code-coverage checks.

1. Pre-requisites

1.1 Elasticsearch

Ensure that you have created the following resources prior to deploying the ESP component:

  1. Create a Policy with the desired Rollover configurations: policy-rollover-mojaloop.json
  2. Create a Template to associate the Policy Settings to the Logstash index based on the following config: template-mojaloop.json

NOTE: If ESP component is running prior to configuring the above template, the index mapping will be auto-created and will not take effect.

1.1.1 Mojaloop Index Rollover Policy

1.1.1.1 Create
curl -X PUT "http://elasticsearch:9200/_ilm/policy/mojaloop_rollover_policy?pretty" -H 'Content-Type: application/json' -d @config/policy-rollover-mojaloop.json
1.1.1.2 Delete

NOTE: only needed if you need to remove the policy_

curl -X DELETE "http://elasticsearch:9200/_ilm/policy/mojaloop_rollover_policy?"

1.1.1.3 Get

NOTE: useful for debugging issues_

curl -X GET "http://elasticsearch:9200/_ilm/policy/mojaloop_rollover_policy?"

1.1.2 Mojaloop Index Template

1.1.2.1 Create
curl -X PUT "http://elasticsearch:9200/_template/moja_template?pretty" -H 'Content-Type: application/json' -d @config/template-mojaloop.json'
1.1.2.2 Delete

NOTE: only needed if you need to remove the template_

curl -X DELETE "http://elasticsearch:9200/_template/moja_template"

1.1.3 Get Template

NOTE: useful for debugging template issues_

curl -X GET "http://elasticsearch:9200/_template/moja_template"