OpenTelemetry Integration

Configure OpenTelemetry observability in Gate Minecraft proxy for comprehensive monitoring with metrics, traces, and logs

OpenTelemetry Integration

Gate includes built-in support for OpenTelemetry, providing comprehensive observability through metrics, distributed tracing, and structured logging. OpenTelemetry is a vendor-agnostic observability framework that allows you to instrument your application and export telemetry data to various backends.

Overview

OpenTelemetry is an observability framework designed to facilitate the generation, export, and collection of telemetry data such as traces, metrics, and logs. Gate leverages the otel-config-go library, which provides a straightforward method to configure observability through environment variables.

Gate automatically initializes OpenTelemetry with sensible defaults. You only need to configure the endpoints and credentials for your observability backend.

Architecture

The typical Gate observability setup follows this architecture:

Gate → OpenTelemetry Collector → Observability Backends
                                  ├── Prometheus (Metrics)
                                  ├── Tempo/Jaeger (Traces)
                                  └── Loki (Logs)

Gate emits telemetry data using the OpenTelemetry Protocol (OTLP), which can be sent to:

• OpenTelemetry Collector (recommended) - Processes and routes telemetry
• Direct backends - Services like Grafana Cloud, Honeycomb, or Datadog
• Self-hosted solutions - Prometheus, Jaeger, Tempo, etc.

Configuration

Gate’s OpenTelemetry implementation is configured entirely through environment variables, making it easy to integrate with container orchestration platforms and cloud environments.

Core Settings

Environment Variable	Default	Description
OTEL_SERVICE_NAME	gate	Name of your service
OTEL_SERVICE_VERSION	-	Version of your service
OTEL_EXPORTER_OTLP_ENDPOINT	localhost:4317	Endpoint for OTLP export
OTEL_EXPORTER_OTLP_PROTOCOL	grpc	Protocol for OTLP (grpc or http/protobuf)
OTEL_METRICS_ENABLED	false	Enable metrics collection
OTEL_TRACES_ENABLED	false	Enable trace collection
OTEL_LOG_LEVEL	info	OpenTelemetry SDK logging level
OTEL_PROPAGATORS	tracecontext,baggage	Configured trace propagators

Exporter Configuration

Environment Variable	Default	Description
OTEL_EXPORTER_OTLP_HEADERS	{}	Global headers for OTLP exporter
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT	localhost:4317	Endpoint for trace export
OTEL_EXPORTER_OTLP_TRACES_HEADERS	{}	Headers specific to trace exporter
OTEL_EXPORTER_OTLP_TRACES_INSECURE	false	Allow insecure trace connections
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT	localhost:4317	Endpoint for metrics export
OTEL_EXPORTER_OTLP_METRICS_HEADERS	{}	Headers specific to metrics exporter
OTEL_EXPORTER_OTLP_METRICS_INSECURE	false	Allow insecure metrics connections
OTEL_EXPORTER_OTLP_METRICS_PERIOD	30s	Metrics reporting interval

Resource Attributes

Use OTEL_RESOURCE_ATTRIBUTES to add additional context to your telemetry:

# Single attribute
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production"

# Multiple attributes
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production,cloud.region=us-east-1,service.instance.id=gate-1"

Quick Start Examples

Local OpenTelemetry Collector

export OTEL_SERVICE_NAME="gate-proxy"
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_METRICS_ENABLED="true"
export OTEL_TRACES_ENABLED="true"

gate

Grafana Cloud

export OTEL_SERVICE_NAME="gate-proxy-prod"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp-gateway-prod-us-central-0.grafana.net/otlp"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64_credentials>"
export OTEL_METRICS_ENABLED="true"
export OTEL_TRACES_ENABLED="true"

gate

Honeycomb

export OTEL_SERVICE_NAME="gate-proxy"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io:443"
export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=<your-api-key>"
export OTEL_METRICS_ENABLED="true"
export OTEL_TRACES_ENABLED="true"

gate

Implementation Details

Gate’s OpenTelemetry integration is implemented in pkg/internal/otelutil/otel.go:15:

func Init(ctx context.Context) (clean func(), err error) {
    serviceName := os.Getenv("OTEL_SERVICE_NAME")
    if serviceName == "" {
        serviceName = "gate"
    }
    
    otelShutdown, err := otelconfig.ConfigureOpenTelemetry(
        otelconfig.WithServiceName(serviceName),
        otelconfig.WithServiceVersion(version.String()),
        otelconfig.WithMetricsEnabled(os.Getenv("OTEL_METRICS_ENABLED") == "true"),
        otelconfig.WithTracesEnabled(os.Getenv("OTEL_TRACES_ENABLED") == "true"),
    )
    // ...
}

The SDK automatically:

• Initializes trace and metric providers
• Configures OTLP exporters (gRPC or HTTP)
• Sets up context propagation
• Handles graceful shutdown

Best Practices

1. Use Meaningful Service Names

# Good - Identifies the specific instance and purpose
export OTEL_SERVICE_NAME="gate-proxy-lobby-us-east"

# Bad - Too generic
export OTEL_SERVICE_NAME="gate"

2. Track Service Versions

# Semantic versioning
export OTEL_SERVICE_VERSION="v1.2.3"

# Git commit hash
export OTEL_SERVICE_VERSION="$(git rev-parse --short HEAD)"

# Build number
export OTEL_SERVICE_VERSION="build-${BUILD_NUMBER}"

3. Add Deployment Context

export OTEL_RESOURCE_ATTRIBUTES="\
deployment.environment=production,\
cloud.provider=aws,\
cloud.region=us-east-1,\
service.namespace=minecraft,\
service.instance.id=${HOSTNAME}"

4. Secure Production Endpoints

# Always use HTTPS in production
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel-collector.example.com:4317"

# Use authentication headers
export OTEL_EXPORTER_OTLP_HEADERS="api-key=${SECRET_API_KEY}"

5. Use OpenTelemetry Collector

For production deployments, route telemetry through an OpenTelemetry Collector for:

• Batching - Reduces network overhead
• Filtering - Removes unnecessary data
• Enrichment - Adds metadata and context
• Retry logic - Handles transient failures
• Multi-backend support - Send to multiple destinations

Observability Solutions

Gate works with any OpenTelemetry-compatible backend:

Cloud Platforms

• Grafana Cloud - Fully managed with Prometheus, Tempo, and Loki
• Honeycomb - Observability for debugging complex systems
• Datadog - Full-stack monitoring and analytics
• New Relic - APM and observability platform
• Azure Monitor - Microsoft cloud monitoring
• AWS X-Ray - AWS distributed tracing
• Google Cloud Observability - GCP monitoring

Self-Hosted

• Metrics: Prometheus, Grafana Mimir
• Tracing: Jaeger, Grafana Tempo
• Logs: Loki
• Visualization: Grafana

Troubleshooting

No Telemetry Data

1. Verify metrics/traces are enabled:

   echo $OTEL_METRICS_ENABLED
   echo $OTEL_TRACES_ENABLED
   

2. Check the endpoint is reachable:

   curl -v http://localhost:4318/v1/traces
   

3. Review Gate logs for OpenTelemetry errors

Connection Refused

• Ensure the collector/backend is running
• Verify the endpoint URL and port
• Check firewall rules
• For Docker, ensure containers are on the same network

Authentication Errors

• Verify API keys and tokens are correct
• Check header format: key=value pairs
• Ensure credentials are base64 encoded if required

Next Steps

Metrics

Learn about available metrics and collection

Tracing

Configure distributed tracing for request flows

Logging

Set up structured logging and log levels

Further Reading

• OpenTelemetry Documentation
• OTLP Specification
• otel-config-go Library
• OpenTelemetry Collector