Event Catalog

=======

Introduction

This repository contains scripts and configurations for generating and managing an event catalog automatically. As long as you are running the node server main.js any changes made in eventNames.json, eventPatterns.json, consumer.json will automatically reflect the changes in event catalog without the need to recreate events manually.

Setup Instructions

1. Extract Payload from kibana for each eventType

• Extract all event types from the kibana using this query POST /eventstore/_search

  "size": 0,
  "aggs": {
    "unique_event_types": {
      "terms": {
        "field": "type",
        "size": 1000000
      },
      "aggs": {
        "example_event": {
          "top_hits": {
            "size": 1
          }
        }
      }
    }
  }
}

and save it in event-catalog-automation/json/originalPayload.json file.

2. Extract Event Types

• Extract all event types from the event-catalog-automation/json/originalPayload.json and save them in evemt-catalog-automation/json/eventNames.json using the python script eventtype_extractor.py.

3. Anonymize Information

• Run the PIFaker.js script located at event-catalog-automation/js using the command:

This will fake all the information and create a JSON file named anonymized_events.json in event-catalog-automation/json/autogenerated.

3. Add Event Patterns

• Add all event patterns to the event-catalog-automation/json/eventPatterns.json file for each service.
• For example, if a service name is SampleService, then it should have all the events like sample.test, sample.test.new, etc. So, the pattern will be sample.test.

4. Configure Producer to Consumer Mapping

• Add all producers to the consumer configuration in event-catalog-automation/json/consumer.json.
• The producer is the key, and the consumer is a list of consumers for that producer.

5. Run the Main Script

• Execute the main.js script located at event-catalog-automation/js using the command:

6. Start Development Server

• Navigate to the event-catalog/ directory.
• Run the following command to start the development server:

7. View the Catalog

• Once the development server is running, you can access the catalog at localhost:3000.

Notes

• Make sure to follow the steps in the specified order for proper setup and execution.
• Ensure that all required dependencies are installed before running the scripts.