Fluent Bit

This document describes a sample setup for sending logs to Levitate via Fluent bit

Introduction

Fluent Bit is a fast and lightweight telemetry agent for logs, metrics, and traces. This document provides step by step instructions to send logs from Fluent Bit to Levitate.

Prerequisites

1. Create a Levitate cluster by following Getting Started guide.
2. Keep the following information handy after creating the cluster from the Integrations page:
   • $levitate_otlp_endpoint: Levitate's OTLP endpoint
   • $levitate_basic_auth_header: OTLP Basic authorization header
   • $levitate_username: OTLP username
   • $levitate_password: OTLP password

Fluent Bit Configuration

Fluent bit supports shipping logs to Levitate using the opentelemetry output plugin.

Add following output stanza in the Fluent Bit configuration file.

    [OUTPUT]
        Name                 opentelemetry
        Match                kube.*
        Host                 otlp.last9.io
        http_user            <levitate_username>
        http_passwd          <levitate_password>
        Port                 443
        Logs_uri             /v1/logs
        Tls                  On
        Tls.verify           On
        logs_body_key_attributes true

Read more on the configuration options provided by the opentelemetry plugin here.

info

Please note that the Match field must match the value of Match field in the Input or Filter sections in the Fluent Bit configuration.

This is the bare minimum configuration that you can add in the Fluent Bit config to start sending logs to Levitate.

Advanced Configuration

You can add advanced lua functions to parse the logs before sending to Levitate.

    function set_fields(tag, timestamp, record)
      local new_record = {}

      local function flatten_table(prefix, t)
        for k, v in pairs(t) do
            if type(v) == "table" then
                flatten_table(prefix .. k .. ".", v)
            else
                new_record[prefix .. k] = v
            end
        end
      end
      flatten_table("", record)

      return 1, timestamp, new_record
    end

As an example, above lua function flattens the nested maps present in logs.

Additionally, it is highly recommended to add a unique tag to the log records per cluster or environment. It can be done by adding a modify Filter.

    [FILTER]
        Name modify
        Match kube.*
        Add env production

Read more on Fluent bit modify filter here.

Verification

Login to Levitate and verify logs in Managed Grafana.

Troubleshooting

Please get in touch with us on Discord or Email if you have any questions.