Skip to main content

Change Events

With Levitate, track change events such as configuration changes and deployment events with ease along with other metrics.

Why Change Events Matter?​

Software systems and their observability are not just about the telemetry data emitted from these systems. The software systems also get affected by external change events. These events can be from domains such as deployment, configuration, or external 3rd party systems. Levitate allows tracking such change events along with other metrics, seamlessly providing more context to the system observability.

This document will show how to start tracking change events with Levitate.

Levitate offers an HTTP API that can be used to track any domain change event. Each event has two states, start and stop, and both can be tracked with Levitate. Once Levitate receives the event, it converts it into a metric that can be used with other metrics for querying using PromQL in Grafana dashboards or alerting.

The data flow for change events is as follows.

Change Events Data Flow

Levitate Change Events API​

Levitate offers a REST HTTP endpoint that can be used to send the change events. The API endpoint is as follows.

curl -XPUT https://app.last9.io/api/v4/organizations/{org_slug}/change_events \
--header 'Content-Type: application/json' \
--header 'X-LAST9-API-TOKEN: Bearer: <LAST9_API_WRITE_TOKEN>' \
--data-raw '
{
  "timestamp": "2024-01-15T17:57:22+05:30",
  "event_name": "new_deployment",
  "event_state": "start",
  "data_source_name": {levitate_cluster_data_source_name},
  "attributes": {
    "env": "production",
    "k8s_cluster": "prod-us-east-1",
    "app": "backend-api"
  }
}'
tip

Refer to the Getting Started with API guide to obtain the token required for the change events API.

  • timestamp is the iso8601 formatted timestamp of the event. It is optional. If not passed, current timestamp is used.
  • event_name can be any event name depending on the context. It will be added as a label to the resulting time series.
  • event_state can be start or stop.
  • attributes will be used as labels while converting the change event to a metric.
  • data_source_name is the name of the Levitate cluster where the events will be stored. It is an optional field and can be obtained as described in the Change Events Storage section.

Levitate will convert the events into a metric named last9_change_events.

Change Events Storage​

It is possible that you might be using multiple Levitate clusters. In such scenario, you can choose to store the change events in a Levitate cluster of your choice. The optional data_source_name attribute is used to specify the cluster where change event will be stored.

If this attribute is not passed, then Levitate will store the change event in a default cluster designated for change events. The default cluster for change events is set as follows.

Default Cluster for Change Events

You can override this by specifying the data_source_name in the request payload. Obtain the cluster name from the Data Sources section as follows.

Data Sources Copy the Data Source Name

info

Store events and related metrics in same Levitate cluster for automatic correlation between them.

Visualize Change Events in Grafana​

The change events can be visualized in Grafana just like any other metrics.

Change Events in Grafana

Troubleshooting​

Please get in touch with us on Discord or Email if you have any questions.