XSCREENSAVER / 3D PIPES

[readonly] markdown buffer

Logs Are Not Observability

Jul 24, 2026 · 9 min read

A service fails in production, so somebody opens the logs.

They find thousands of lines saying that functions started, requests completed and objects were loaded. None of them answer the useful question:

What happened to this customer's operation?

The system has logging. It is not observable.

traceview://observability-problem
$ tail -f services.log
  1. gatewayrequest completed
  2. ordersloaded order
  3. paymentsretrying request
  4. ordershandler finished
which user?
unknown
same operation?
unknown
final outcome?
unknown
The log contains activity, but no shared identity connects the lines to a user or final outcome.

Logs are output

A log is one kind of evidence emitted by a running system.

Observability is the ability to use available evidence to understand its state, including failures nobody predicted in advance.

More output does not guarantee more understanding. Ten services can each print request completed while losing the identity that connects those requests into one operation.

This is not fixed by changing the log level to debug.

Start with operational questions

Before adding another line, decide what somebody must be able to answer:

  • did the operation succeed for the user?
  • where did it spend its time?
  • which dependency failed?
  • was the failure transient or permanent?
  • how many users are affected?
  • did a retry eventually complete the work?

Those questions determine the evidence the system needs.

A useful event contains stable context:

logger.info("payment.completed", {
  operationId,
  orderId,
  attempt,
  provider,
  durationMs,
  outcome: "succeeded",
});

It describes an outcome, not merely a point reached in the code.

Give the operation an identity

Requests, jobs and messages often cross process boundaries. Give the business operation a stable identifier and propagate it.

A transport request ID identifies one HTTP request. It may not identify the payment, import or fulfilment operation that survives several requests and retries.

Both are useful:

  • a trace or request ID connects one execution path
  • an operation ID connects every attempt at the same business outcome

Without that distinction, a retry looks like unrelated work and duplicate execution is difficult to recognise.

traceview://observability
$ trace operation op_7F3A
  1. gatewayrequest accepted
  2. ordersoperation claimed
  3. paymentsattempt 1 · timeout
  4. paymentsattempt 2 · declined
  5. ordersterminally failed
user outcome
failed
failed at
payments
attempts
2
duration
1.885s
With one operation identifier, the same events form a trace that answers where the operation failed, how often it tried and what the user experienced.

Use different evidence for different questions

Logs are useful for discrete events with explanatory context.

Metrics answer aggregate questions: error rate, latency distribution, queue depth and the number of exhausted retries.

Traces show where one execution travelled and where it waited.

No single signal replaces the others. A trace is poor at telling you whether failures doubled across the whole system. A metric can show that failures doubled without identifying the affected operation. A log may explain the failure once you know which operation to inspect.

Observability comes from connecting them.

traceview://observability-signals
logsWhy did op_7F3A fail?payment_declined
metricsAre failures increasing?
tracesWhere did it wait?
Logs explain one event, metrics reveal a population-level change, and traces expose where one execution spent its time.

Record outcomes at boundaries

Function-entry logs create noise because internal structure is rarely the thing an operator cares about.

Prefer events where responsibility changes:

  • an API accepted or rejected a request
  • a message was published or consumed
  • a dependency returned an outcome
  • a retry was scheduled
  • a business operation completed or became terminally failed

Include duration, attempt number and stable identifiers where they matter. Use consistent field names so tools and people can query them.

Do not put secrets, access tokens or unnecessary personal data into logs. Evidence that creates another incident is not useful evidence.

Design for the bad day

Observability is not something a logging library adds after the feature is finished.

It is part of the operation's design. The code must preserve identity, distinguish attempts from outcomes and emit evidence at the boundaries where failure matters.

The test is simple: choose a customer operation and ask whether you can reconstruct what happened without reading the source code and guessing.

If the answer is no, adding more log lines is unlikely to help.

Record less noise. Preserve more meaning.