PHPMicroservicesLoggingLaravelTrace

How to keep track of logs between microservices

In an environment where micro-services are the norm, a common question arises: how do you efficiently track logs between these decoupled applications? When you have several independent services, each with their own execution flows, it can be a challenge to correlate the logs of a specific request that passes between them.

In this scenario, there are several tools on the market that solve the problem for you, such as Elastic Stack, Datadog, etc. But in a specific situation at my work, I decided to build a customized solution that would distribute the trace according to the flow of data between the services.

The Problem

Imagine that you have an authentication service and a payment processing service. A customer request can start with the authentication service and move on to the payment service. Each service has its own life cycle and its own logs. How do you know if an error in the payment service was caused by a failure in the authentication process? This is where the trace_id comes in.

The Solution

My idea was to create a PHP package that, when installed in all the micro-services, would ensure that the trace_id was propagated between them. The concept was simple: when a request came into the system, a trace_id would be generated or reused if it already existed. This trace_id would be included in the HTTP header of the requests and, in each service, would be automatically added to the log context.

To do this, I used log context sharing (usage may vary according to language or structure) to ensure that any log generated during the request lifecycle had the trace_id attached.

• In this example I use Laravel Framework:

<?php

namespace App\Http\Middleware;

use Illuminate\Http\Request;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;

class LogTraceMiddleware
{
    public function handle(Request $request, \Closure $next)
    {
        $traceId = $request->header('x-trace-id') ?? Str::uuid()->toString();

        Log::withContext(['trace_id' => $traceId]);

        return $next($request);
    }
}

Package Flow

1. The client makes a request -> The x-trace-id is generated and added to the header.
2. Each microservice that receives the request shares the trace_id in the context of the logs.
3. If the request triggers an asynchronous job, the trace_id is passed to the job.
4. All logs throughout the lifecycle, including jobs, have the trace_id.

Conclusion

This solution solved the problem of correlating logs between asynchronous services and processes. The next step was to configure monitoring tools to automatically search for this trace_id in all the logs, making it easier to identify problems.