OpenTelemetry Collector

This document describes a sample setup for sending metrics to Last9 via OpenTelemetry Collector.

Prerequisites

1. Create a Last9 cluster by following Getting Started
2. Keep the following information handy after creating the cluster:
   • $levitate_remote_write_url: Last9's Remote write endpoint
   • $levitate_remote_write_username: Cluster ID
   • $levitate_remote_write_password: Write token created for the cluster

OpenTelemetry Collector Basics

OpenTelemetry collectors consist of three parts:

1. Receivers
   • A receiver, which can be push or pull-based sends data to the collector
   • In this doc, we will use Prometheus receiver, as an example, which will receive metrics
2. Processors
   • Processors are run on data between being received and being exported
   • Processors are not mandatory for sending data to Last9
3. Exporters
   • An exporter, which can be push or pull-based, sends data to one or more backends/destinations
   • We will use Last9 as an exporter destination

Setup

1. Create a docker-compose.yaml file with the following config:

   Note that the image used for OpenTelemetry Collector is otel/opentelemetry-collector-contrib:0.64.0 which has support for the Basic authentication extension.

   version: "3.5"
   services:
     otel-collector:
       container_name: otel-collector-contrib
       image: otel/opentelemetry-collector-contrib:0.64.0
       command: ["--config=/etc/otelcol-config.yaml"]
       ports:
         - "8888:8888" # Prometheus metrics exposed by the collector
         - "13133:13133" # health_check extension
       volumes:
         - ./otel-config.yaml:/etc/otelcol-config.yaml
       restart: always
       networks:
         - monitoring
   networks:
     monitoring:
       driver: bridge

2. Create otel-config.yaml with a sample scrape config. Replace this scrape config by your desired config:

   extensions:
     basicauth/prw:
       client_auth:
         # Replace these variables by their actual values
         username: $levitate_remote_write_username
         password: $levitate_remote_write_password
   
   receivers:
     prometheus:
       config:
         scrape_configs:
         - job_name: 'otel-collector-01'
           scrape_interval: 60s
           static_configs:
           - targets: ['otel-collector:8888']
   
   exporters:
     prometheusremotewrite:
       auth:
         authenticator: basicauth/prw
       endpoint: "$levitate_remote_write_url" # Replace this variable by its actual value
   
   service:
     extensions: [basicauth/prw]
     pipelines:
       metrics:
         receivers: [prometheus]
         exporters: [prometheusremotewrite]

3. Deploy the setup using:

   docker-compose up

As per the config, the entire flow is combined in the pipelines section under the service where the data flows from OpenTelemetry Collector to Last9 using the basic auth mechanism.

info

The same config file otel-config.yaml can be used when using another compute layer e.g. K8s.

Troubleshooting

Please get in touch with us on Discord or Email if you have any questions.