Fluent Bit

This document describes a sample setup for sending logs to Last9 via Fluent bit

Introduction

Fluent Bit is a fast and lightweight telemetry agent for logs, metrics, and traces. This document provides step by step instructions to send logs from Fluent Bit to Last9.

Prerequisites

1. Create a Last9 cluster by following Getting Started guide.
2. Keep the following information handy after creating the cluster from the Integrations page:
   • $last9_otlp_endpoint: Last9's OTLP endpoint
   • $last9_basic_auth_header: OpenTelemetry Basic authorization header
   • $last9_username: OpenTelemetry username
   • $last9_password: OpenTelemetry password

Fluent Bit Configuration

Fluent bit supports shipping logs to Last9 using the opentelemetry output plugin.

Add following output stanza in the Fluent Bit configuration file.

    [OUTPUT]
        Name                 opentelemetry
        Match                kube.*
        Host                 otlp.last9.io
        http_user            <last9_username>
        http_passwd          <last9_password>
        Port                 443
        Logs_uri             /v1/logs
        Tls                  On
        Tls.verify           On
        logs_body_key_attributes true

Read more on the configuration options provided by the opentelemetry plugin here.

info

Please note that the Match field must match the value of Match field in the Input or Filter sections in the Fluent Bit configuration.

This is the bare minimum configuration that you can add in the Fluent Bit config to start sending logs to Last9.

Advanced Configuration

You can add advanced lua functions to parse the logs before sending to Last9.

    function set_fields(tag, timestamp, record)
      local new_record = {}

      local function flatten_table(prefix, t)
        for k, v in pairs(t) do
            if type(v) == "table" then
                flatten_table(prefix .. k .. ".", v)
            else
                new_record[prefix .. k] = v
            end
        end
      end
      flatten_table("", record)

      return 1, timestamp, new_record
    end

As an example, above lua function flattens the nested maps present in logs.

Additionally, it is highly recommended to add a unique tag to the log records per cluster or environment. It can be done by adding a modify Filter.

    [FILTER]
        Name modify
        Match kube.*
        Add env production

Read more on Fluent bit modify filter here.

Verification

Login to Last9 and visit the Logs panel.

Troubleshooting

Please get in touch with us on Discord or Email if you have any questions.