Gin

Send distributed traces and metrics to Last9 from Golang Gin application using OpenTelemetry

Introduction

Gin is a web framework written in Go (Golang). This comprehensive guide will help you instrument your Gin application with OpenTelemetry and smoothly send the traces to Last9. You can also check out the example application on GitHub↗.

Pre-requisites

1. You have a Gin application.
2. You have signed up for Last9, and obtained the following OTLP credentials from the Integrations page:
   • endpoint
   • auth_header

Install OpenTelemetry packages

To install the required packages, run the following command:

  go get -u go.opentelemetry.io/otel
  go get -u go.opentelemetry.io/otel/sdk
  go get -u go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
  go get -u go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin
  go get -u go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc
  go get -u go.opentelemetry.io/otel/sdk/metric
  go get -u go.nhat.io/otelsql

For setting up Redis instrumentation, first verify which go-redis version you are using.

If you are using go-redis v8 then

  go get -u github.com/go-redis/redis/extra/redisotel/v8

If you are using go-redis v9 then

  go get github.com/redis/go-redis/extra/redisotel/v9

Using these packages, you can instrument your Gin application to send traces and metrics to Last9.

Traces

This application generates traces for the following:

• HTTP requests using otelgin
• Database queries using otelsql
• Redis commands using redisotel

For HTTP requests, wrap the Gin router with the otelgin.Middleware middleware.

Refer to main.go for more details.

For database queries, use the otelsql package to wrap the sql.DB object.

Refer to initDB() in users/controller.go for more details.

For Redis commands, use the redisotel package to wrap the redis.Client object.

Refer to initRedis() in users/controller.go for more details.

Metrics

The application also generates metrics for the following:

• Database queries using otelsql

Set the environment variables

Set the following environment variables:

export OTEL_SERVICE_NAME=gin-app-service
export OTEL_EXPORTER_OTLP_ENDPOINT=<endpoint>
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <auth_header>"

These environment variables are used to configure the OpenTelemetry SDK to send traces and metrics to Last9.

Instrument HTTP requests

In your Gin application, create a new file instrumentation.go and add the following code: Refer to main.go for more details.

package main

import (
	"context"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
	"go.opentelemetry.io/otel/propagation"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
	semconv "go.opentelemetry.io/otel/semconv/v1.26.0"
	"go.opentelemetry.io/otel/trace"
)

type Instrumentation struct {
	TracerProvider *sdktrace.TracerProvider
	Tracer         trace.Tracer
}

func initTracerProvider() *sdktrace.TracerProvider {
	exporter, err := otlptracegrpc.New(context.Background())

	if err != nil {
		panic(err)
	}

	attr := resource.WithAttributes(
		semconv.DeploymentEnvironmentKey.String("production"), // You can change this value to "development" or "staging" or you can get the value from the environment variables
		semconv.ServiceNameKey.String("gin-server"), // You can change this value to the name of your service
		// You can add more attributes here
	)

	resources, err := resource.New(context.Background(),
		resource.WithFromEnv(),
		resource.WithTelemetrySDK(),
		resource.WithProcess(),
		resource.WithOS(),
		resource.WithContainer(),
		resource.WithHost(),
		attr)

	if err != nil {
		panic(err)
	}

	tp := sdktrace.NewTracerProvider(
		sdktrace.WithBatcher(exporter),
		sdktrace.WithResource(resources),
	)

	otel.SetTracerProvider(tp)
	otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{}))

	return tp
}

func NewInstrumentation() *Instrumentation {
	tp := initTracerProvider()

	return &Instrumentation{
		TracerProvider: tp,
		Tracer:         tp.Tracer("gin-server"),
	}
}

The above code configures the OpenTelemetry SDK to use the OTLP exporter and initializes the TracerProvider.

Next, at the entry point of your application, add the following code to instrument your application: Refer to main.go for more details.

	i := NewInstrumentation()

	defer func() {
		if err := i.TracerProvider.Shutdown(context.Background()); err != nil {
			log.Printf("Error shutting down tracer provider: %v", err)
		}
	}()

Now, add the otel gin middleware to your application:

  import (
    // other imports
	  "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
  )

  // other code

  r := gin.Default()
  // Make sure this is added before any routes are defined
  r.Use(otelgin.Middleware("gin-server"))

  // other code

Refer to main.go for more details.

Use the tracer object to create spans in your application as follows with custom attributes such as user id:

Refer to users/controller.go for more details.

	_, span := u.tracer.Start(c.Request.Context(), "UpdateUser", oteltrace.WithAttributes(
		attribute.String("user.id", c.Param("id")),
	))
	defer span.End()

Instrument database operations

Instrumenting with sql.DB

Add the following code to instrument the database queries. It uses the otelsql package to wrap the sql.DB object and emit traces and metrics for database queries and connections. Refer to users/controller.go for more details.

func initDB() (*sql.DB, error) {
	driverName, err := otelsql.Register("postgres",
		// Read more about the options here: https://github.com/nhatthm/otelsql?tab=readme-ov-file#options
		otelsql.AllowRoot(),
		otelsql.TraceQueryWithoutArgs(),
		otelsql.TraceRowsClose(),
		otelsql.TraceRowsAffected(),
		otelsql.WithDatabaseName("otel_demo"), // database name
		otelsql.WithSystem(semconv.DBSystemPostgreSQL),
	)
	if err != nil {
		return nil, fmt.Errorf("failed to register driver: %v", err)
	}

	db, err := sql.Open(driverName, dsnName)
	if err != nil {
		return nil, fmt.Errorf("failed to connect to database: %v", err)
	}

	// Record stats to expose metrics
	if err := otelsql.RecordStats(db); err != nil {
		return nil, err
	}

	return db, nil
}

Instrumenting with pgx

For database instrumentation where pgx is used, we use otelpgx to wrap the pgx connection pool. Add the following code to instrument the database queries.

	var connString = os.Getenv("DATABASE_URL")
	cfg, err := pgxpool.ParseConfig(connString)
	if err != nil {
		fmt.Fprintf(os.Stderr, "create connection pool: %w", err)
		os.Exit(1)
	}

    // Add the tracer to the connection pool configuration
	cfg.ConnConfig.Tracer = otelpgx.NewTracer()
    // Create a new connection pool with the tracer
	conn, err = pgxpool.NewWithConfig(context.Background(), cfg)
	if err != nil {
		fmt.Fprintf(os.Stderr, "Unable to connection to database: %v\n", err)
		os.Exit(1)
	}

Refer to the complete example using pgx adapter and Otel instrumentation here.

Instrument Redis operations

tip

Note that between redis-go versions v8 and v9 the import paths are changed.

Add the following code to instrument the Redis operations. It uses the redisotel package to wrap the redis.Client object and emit traces for Redis commands.

go-redis v9

If you are using go-redis v9 then use the following code.

Refer to users/controller.go for more details.

func initRedis() *redis.Client {
	rdb := redis.NewClient(&redis.Options{
		Addr: "localhost:6379", // Update this with your Redis server address
	})

	// Setup traces for redis instrumentation
	if err := redisotel.InstrumentTracing(rdb); err != nil {
		log.Fatalf("failed to instrument traces for Redis client: %v", err)
		return nil
	}
	return rdb
}

go-redis v8

If you are using go-redis v8 then use the following code.

func initRedis() *redis.Client {
	rdb := redis.NewClient(&redis.Options{
		Addr: "localhost:6379", // Update this with your Redis server address
	})

	// Setup traces for redis instrumentation
    rdb.AddHook(redisotel.NewTracingHook())
	return rdb
}

Refer to main.go for more details.

Run the application

Start your Gin application by running the following command:

go run main.go

This will start the Gin application and the OpenTelemetry SDK will automatically collect traces and metrics from the Gin application. These traces and metrics will be sent to Last9 automatically based on the environment variables set.

View traces and metrics in Last9

After running the Gin app, you can visualize the traces and metrics in Last9's APM dashboard.

Troubleshooting

Please get in touch with us on Discord or Email if you have any questions.