The Power of Structured Logging in Modern Go Applications

Introduction

In the world of software development, logging is often treated as an afterthought—a necessary evil that clutters our code with print statements or simple string messages. However, as applications grow in complexity and scale, traditional logging approaches quickly become insufficient for effective monitoring, debugging, and analysis.

Structured logging represents a paradigm shift in how we think about application observability. Rather than treating logs as unstructured text meant only for human consumption, structured logging treats each log entry as a data record with well-defined fields that can be easily parsed, queried, and analyzed by machines.

In this blog post, I'll explore the fundamentals of structured logging in Go, demonstrate implementation approaches using popular libraries, and show how structured logging can transform your application's observability posture.

The Limitations of Traditional Logging

Before diving into structured logging, let's examine why traditional text-based logging falls short in modern applications:

// Traditional unstructured logging
log.Printf("Error processing order %s for user %s: %v", orderID, userID, err)

This approach has several drawbacks:

1. Difficult to Parse: Extracting specific data elements requires parsing arbitrary text formats
2. Inconsistent Formatting: Developers may format similar events differently
3. Limited Context: Adding context requires concatenating more text
4. Costly Searches: Finding specific events requires full-text search
5. Limited Analysis: Aggregating or correlating events is cumbersome

Enter Structured Logging

Structured logging addresses these limitations by recording events as structured data, typically in JSON format:

{
  "level": "error",
  "time": "2025-06-10T14:23:45Z",
  "message": "Error processing order",
  "trace_id": "4f8d3a2b1c",
  "order_id": "ord_12345",
  "user_id": "usr_67890",
  "error": "inventory_insufficient",
  "amount": "125.50",
  "status": "rejected"
}

The advantages are immediate:

1. Machine-Readable: Logs can be easily ingested by analysis tools
2. Queryable: Specific fields can be efficiently queried
3. Consistent: Structure enforces consistency across the codebase
4. Contextual: Rich context can be added without cluttering the message
5. Analyzable: Data can be aggregated, filtered, and visualized

Implementing Structured Logging in Go

Go has several excellent structured logging libraries. We'll focus on two popular options: Zap and Zerolog.

Setting Up Zap Logger

Uber's Zap is a blazingly fast, structured logger. Here's how to set it up:

package logger

import (
	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
)

func NewLogger() (*zap.Logger, error) {
	config := zap.NewProductionConfig()

	config.Encoding = "json"

	config.EncoderConfig.TimeKey = "time"
	config.EncoderConfig.LevelKey = "level"
	config.EncoderConfig.NameKey = "logger"
	config.EncoderConfig.CallerKey = "caller"
	config.EncoderConfig.MessageKey = "message"
	config.EncoderConfig.StacktraceKey = "stacktrace"

	config.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
	config.EncoderConfig.EncodeLevel = zapcore.CapitalLevelEncoder
	config.EncoderConfig.EncodeCaller = zapcore.ShortCallerEncoder
	config.EncoderConfig.EncodeDuration = zapcore.StringDurationEncoder

	return config.Build()
}

With this configuration, Zap will produce well-structured JSON logs with consistent field names and encodings.

Enhancing Logs with Context

One of the most powerful aspects of structured logging is the ability to enrich logs with contextual information. In distributed systems, tracking a request across multiple services is essential. This is where trace IDs come in:

package logger

import (
	"context"
	"go.uber.org/zap"
)

const TraceIDKey string = "trace_id"

type ContextLogger struct {
	logger *zap.Logger
}

func NewContextLogger(log *zap.Logger) *ContextLogger {
	return &ContextLogger{logger: log}
}

func (cl *ContextLogger) GetTraceID(ctx context.Context) string {
	if v := ctx.Value(TraceIDKey); v != nil {
		if traceID, ok := v.(string); ok {
			return traceID
		}
	}
	return "unknown-trace"
}

func (cl *ContextLogger) WithContext(ctx context.Context) *zap.Logger {
	traceID := cl.GetTraceID(ctx)
	return cl.logger.With(zap.String(TraceIDKey, traceID))
}

func (cl *ContextLogger) Info(ctx context.Context, msg string, fields ...zap.Field) {
	cl.WithContext(ctx).Info(msg, fields...)
}

func (cl *ContextLogger) Error(ctx context.Context, msg string, fields ...zap.Field) {
	cl.WithContext(ctx).Error(msg, fields...)
}

This ContextLogger wrapper automatically extracts a trace ID from the context and includes it in every log message, providing crucial context for distributed tracing.

Dependency Injection with Uber's FX

To make our logger available throughout the application, we can use dependency injection. Here's how to integrate our logger with Uber's FX dependency injection framework:

package logger

import (
	"go.uber.org/fx"
	"go.uber.org/zap"
)

func Module() fx.Option {
	return fx.Options(
		fx.Provide(
			NewLogger,
			NewContextLogger,
		),
	)
}

Practical Usage Patterns

Let's explore some practical patterns for using structured logging effectively in a Go application.

Repository Layer Logging

When implementing a repository pattern, structured logging can provide valuable insights into database operations:

type Repository struct {
	db  *gorm.DB
	log *logger.ContextLogger
}

func (r *Repository) StoreOrder(ctx context.Context, order *Order) error {
	r.log.Info(ctx, "Storing order",
		zap.String("order_id", order.ID),
		zap.String("status", order.Status),
		zap.String("total", order.Total))

	result := r.db.WithContext(ctx).Create(order)
	if result.Error != nil {
		r.log.Error(ctx, "Failed to store order",
			zap.String("order_id", order.ID),
			zap.Error(result.Error))
		return fmt.Errorf("failed to store order: %w", result.Error)
	}

	r.log.Info(ctx, "Order stored successfully",
		zap.String("order_id", order.ID))
	return nil
}

Service Layer Logging

In the service layer, we can capture business logic events with rich context:

type Service struct {
	repo        Repository
	log         *logger.ContextLogger
	inventorySvc InventoryService
}

func (s *Service) ProcessOrder(ctx context.Context, req *OrderRequest) (*OrderResponse, error) {
	// Create a unique order ID
	orderID := uuid.NewString()

	// Log the beginning of the order process
	s.log.Info(ctx, "Order process initiated",
		zap.String("order_id", orderID),
		zap.String("total", req.Total),
		zap.String("user_id", req.UserID))

	// Check inventory availability
	available, err := s.inventorySvc.CheckAvailability(ctx, req.Items)
	if err != nil {
		s.log.Error(ctx, "Inventory check failed",
			zap.String("order_id", orderID),
			zap.Error(err))
		return nil, fmt.Errorf("inventory check failed: %w", err)
	}

	if !available {
		s.log.Info(ctx, "Insufficient inventory for order",
			zap.String("order_id", orderID))
		return &OrderResponse{
			OrderID: orderID,
			Status:  "REJECTED",
			Reason:  "INSUFFICIENT_INVENTORY",
		}, nil
	}

	// Store order record
	order := &Order{
		ID:        orderID,
		UserID:    req.UserID,
		Items:     req.Items,
		Total:     req.Total,
		Status:    "PENDING",
		CreatedAt: time.Now(),
	}

	if err := s.repo.StoreOrder(ctx, order); err != nil {
		// Repository already logs the error
		return nil, fmt.Errorf("failed to store order: %w", err)
	}

	// Process order fulfillment
	fulfillmentResp, err := s.fulfillOrder(ctx, order)
	if err != nil {
		s.log.Error(ctx, "Order fulfillment failed",
			zap.String("order_id", orderID),
			zap.Error(err))

		// Update order status
		order.Status = "FAILED"
		_ = s.repo.UpdateOrder(ctx, order)

		return nil, fmt.Errorf("order fulfillment failed: %w", err)
	}

	// Update order with fulfillment response
	order.Status = fulfillmentResp.Status
	order.TrackingID = fulfillmentResp.TrackingID

	if err := s.repo.UpdateOrder(ctx, order); err != nil {
		s.log.Error(ctx, "Failed to update order with fulfillment details",
			zap.String("order_id", orderID),
			zap.Error(err))
		return nil, fmt.Errorf("failed to update order: %w", err)
	}

	s.log.Info(ctx, "Order process completed",
		zap.String("order_id", orderID),
		zap.String("status", fulfillmentResp.Status))

	return &OrderResponse{
		OrderID:    orderID,
		Status:     fulfillmentResp.Status,
		TrackingID: fulfillmentResp.TrackingID,
	}, nil
}

Advanced Techniques

Log Sampling and Levels

Not all logs are equally important. We can configure Zap to sample logs at different rates based on their level:

func NewSampledLogger() (*zap.Logger, error) {
	config := zap.NewProductionConfig()

	// Sample logs: initially log 100% of logs at each level, then sample thereafter
	config.Sampling = &zap.SamplingConfig{
		Initial:    100,
		Thereafter: 100,
	}

	return config.Build()
}

This configuration will help manage log volume in high-throughput systems while ensuring important events are captured.

Custom Encoders

Zap allows for custom encoders to handle specific data types in a consistent manner:

func CustomTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder) {
	enc.AppendString(t.Format(time.RFC3339Nano))
}

func NewCustomLogger() (*zap.Logger, error) {
	config := zap.NewProductionConfig()
	config.EncoderConfig.EncodeTime = CustomTimeEncoder
	return config.Build()
}

Request/Response Logging Middleware

For HTTP services, middleware can automatically log request and response details:

func LoggingMiddleware(log *logger.ContextLogger) func(http.Handler) http.Handler {
	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			ctx := r.Context()

			// Generate trace ID if not present
			traceID := r.Header.Get("X-Trace-ID")
			if traceID == "" {
				traceID = uuid.NewString()
			}

			// Add trace ID to context
			ctx = context.WithValue(ctx, logger.TraceIDKey, traceID)

			// Add trace ID to response headers
			w.Header().Set("X-Trace-ID", traceID)

			// Create a wrapped response writer to capture status code
			ww := middleware.NewWrapResponseWriter(w, r.ProtoMajor)

			start := time.Now()

			// Extract API operation from path for better categorization
			operation := "unknown"
			if strings.Contains(r.URL.Path, "/orders/") {
				operation = "order-management"
			} else if strings.Contains(r.URL.Path, "/products/") {
				operation = "product-catalog"
			} else if strings.Contains(r.URL.Path, "/users/") {
				operation = "user-management"
			}

			// Log request
			log.Info(ctx, "Request started",
				zap.String("method", r.Method),
				zap.String("path", r.URL.Path),
				zap.String("operation", operation),
				zap.String("remote_addr", r.RemoteAddr),
				zap.String("user_agent", r.UserAgent()))

			// Process the request
			next.ServeHTTP(ww, r.WithContext(ctx))

			// Calculate duration
			duration := time.Since(start)

			// Log response
			log.Info(ctx, "Request completed",
				zap.String("method", r.Method),
				zap.String("path", r.URL.Path),
				zap.String("operation", operation),
				zap.Int("status", ww.Status()),
				zap.Duration("duration", duration),
				zap.Int("size", ww.BytesWritten()))
		})
	}
}

Zerolog: An Alternative Approach

While Zap is excellent, Zerolog is another popular structured logging library with a different API style focused on chaining:

package logger

import (
	"github.com/rs/zerolog"
	"os"
	"time"
)

func NewZerologLogger() zerolog.Logger {
	output := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: time.RFC3339}

	return zerolog.New(output).
		With().Timestamp().
		Caller().
		Logger()
}

// Usage example
func ProcessPayment(logger zerolog.Logger, paymentID string, amount float64) error {
	logger.Info().
		Str("payment_id", paymentID).
		Float64("amount", amount).
		Msg("Processing payment")

	// Process payment logic here

	return nil
}

Zerolog's chained API can lead to very clean and readable logging code.

Integrating with Log Management Systems

Structured logs shine when integrated with modern log management systems like Elasticsearch, Splunk, or Datadog. These systems can ingest JSON logs and provide powerful querying capabilities:

func NewElasticsearchLogger() (*zap.Logger, error) {
	config := zap.NewProductionConfig()

	// Add fields that help with Elasticsearch indexing
	config.InitialFields = map[string]interface{}{
		"service": "order-service",
		"environment": os.Getenv("ENVIRONMENT"),
		"version": "1.2.3",
	}

	return config.Build()
}

Best Practices for Structured Logging

1. Be Consistent with Field Names

Use consistent field names across your application to make querying easier:

// Good: Consistent field names
log.Info(ctx, "Order processed", 
	zap.String("order_id", order.ID),
	zap.String("status", "completed"))

// Later in another component
log.Error(ctx, "Failed to send order confirmation", 
	zap.String("order_id", order.ID), // Same field name!
	zap.Error(err))

2. Log at Appropriate Levels

Use the right log level for each message:

• Debug: Detailed information for debugging
• Info: Notable but expected events
• Warn: Unexpected events that don't interrupt operation
• Error: Errors that prevent normal operation
• Fatal/Panic: Critical errors that require immediate attention

3. Include Context, Not Just Messages

Structured logging excels when you include rich context:

// Poor: Just a message
log.Error(ctx, "Database error")

// Better: Include relevant context
log.Error(ctx, "Database query failed",
	zap.String("query_type", "insert"),
	zap.String("table", "orders"),
	zap.Error(err),
	zap.Int("retry_count", retryCount))

4. Don't Log Sensitive Information

Be careful not to log sensitive data:

// BAD: Logs personal identifiable information
log.Info(ctx, "Processing user registration", 
	zap.String("email", userEmail),
	zap.String("password", password))

// GOOD: Logs non-sensitive information
log.Info(ctx, "Processing user registration", 
	zap.String("user_id", userID),
	zap.Bool("has_password", password != ""))

5. Use Structured Errors

Consider using structured errors that can be included in logs:

type StructuredError struct {
	Code    string
	Message string
	Details map[string]interface{}
	Cause   error
}

func (e *StructuredError) Error() string {
	return e.Message
}

// When logging
log.Error(ctx, "Order processing failed", 
	zap.String("error_code", err.Code),
	zap.Any("error_details", err.Details),
	zap.Error(err.Cause))

Conclusion

Structured logging represents a significant advancement in application observability. By treating logs as structured data rather than mere text, we gain powerful capabilities for searching, analyzing, and monitoring our applications.

In Go, libraries like Zap and Zerolog make structured logging straightforward to implement. When combined with trace IDs for request tracking and integrated with modern log aggregation systems, structured logging becomes an essential pillar of a robust observability strategy.

The examples in this post demonstrate how to implement structured logging in various layers of a Go application, from the repository layer to HTTP middleware. By following these patterns and best practices, you can transform your application's logging from a debugging afterthought into a powerful operational tool.

As e-commerce and other digital platforms continue to scale, the ability to quickly identify and troubleshoot issues becomes increasingly critical. Structured logging provides the foundation for observability that helps engineering teams maintain reliability and performance even as systems grow more complex.

Remember: In modern distributed systems, effective logging isn't just about recording what happened—it's about providing the context necessary to understand why it happened and how components interacted during the process. Structured logging is the key to achieving this level of insight.

---