Moving failure handling out of the program
A Socratic walk-through of moving failure handling out of the program — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why take the code that copes with failure out of every application and place it alongside them instead?
Retrying a failed call, giving up after a timeout, refusing to keep hammering a service that is clearly down, encrypting the hop, recording how long it took — all of that is code, and it plainly belongs to the thing making the call. Where else would it go?
The service mesh answer is: into a separate process running beside the application, which intercepts the application's network traffic and does that work on its behalf. The application is then written as though the network were reliable, and something else makes that nearly true. Before deciding whether that is elegant or absurd, it is worth asking what problem it is responding to — because the answer is not "applications handle failure badly" so much as "there are forty of them and they do it differently".
Reasoning it through
REASONING #Start with one service and the question does not arise. Add a shared library and it still does not: put the retry policy, the timeout defaults, the circuit breaker and the metrics in a library, have every service import it, done. That was genuinely the state of the art — Netflix's Hystrix and Ribbon, Twitter's Finagle — and for a single-language estate it works.
So find where it stops working. Ask what happens when the estate is not one language. Now the library exists in Java, and someone must write and maintain equivalents in Go, Python, Node and Rust, each with subtly different retry semantics and metric names. The behaviour of your system's failure handling is now a function of which team wrote which port. Uniformity was never declared as a goal, but it was the thing the library was quietly providing.
Then ask a second question: how do you change a timeout? With a library, you change the code, cut a release, and wait for every service to upgrade and redeploy. Across dozens of teams with their own schedules, an "immediate" policy change takes a quarter. During an incident, that is not a control at all.
Both problems have the same shape: the failure-handling logic is inside the deployment unit of the business code, so it inherits that unit's language and its release cycle. Move it out of the process and both dissolve at once. Put a proxy next to the application — Envoy is the usual one — redirect the application's inbound and outbound traffic through it, and it can retry, time out, break circuits, balance load, terminate mutual TLS, and emit consistent telemetry regardless of what the application is written in. That is the sidecar, and the set of sidecars is the mesh's data plane.
Now ask the follow-up that makes it a mesh rather than a pile of proxies: how do a hundred sidecars agree on policy? Through a control plane — Istio's istiod being the canonical example — which holds the configuration you declare, knows the service registry, issues and rotates the workload certificates that make mutual TLS automatic, and pushes updated configuration to every sidecar. Changing a timeout becomes an update to a policy object that propagates in seconds, with no rebuild and no redeploy. That is the property the library never had.
What does it cost? Be concrete. Every call now traverses two extra proxy hops, adding latency — small, typically single-digit milliseconds, but not zero and it accumulates across a call chain. Every pod carries an extra container consuming memory and CPU, which at scale is a real bill. And you have introduced a substantial distributed system whose failure modes are now yours to understand; a misconfigured mesh can break traffic that was working, and debugging gains a layer.
Those costs are exactly why the architecture has been moving. Istio's ambient mode, which reached general availability in 2024, splits the job: a per-node component handles the identity and encryption layer, with an optional per-namespace proxy added only where richer layer-7 policy is genuinely needed. That is a direct response to paying per-pod for features many workloads never used — and it suggests the durable idea is not the sidecar specifically but the separation.
The analogy
THE ANALOGY #Think of an embassy's translation and protocol staff. Every diplomat could learn each foreign language, each country's forms of address, and each set of customs rules — and then every one of them would learn it slightly differently, and a change in protocol would mean re-training all of them. Instead a specialist stands beside each diplomat, handling exactly those cross-cutting concerns identically for everyone, updated centrally when the rules change. The diplomat concentrates on what they came to say.
an interpreter understands the meaning of what is being said and can flag that a message would cause offence, whereas the proxy sees only the shape of the traffic and cannot know whether the request it is cheerfully retrying was a payment that must never be issued twice — the semantics stay with the application no matter how much of the plumbing moves out.
Clarifying the model
THE MODEL #Three refinements.
First, that last point is the sharpest one and deserves restating as a rule: a mesh can retry, but only the application knows whether an operation is safe to retry. Configuring automatic retries on a non-idempotent endpoint converts a transient failure into a duplicate charge. The mesh gives you a lever; correctness still depends on knowing which operations may be pulled.
Second, "out of the program" does not mean the concern disappeared — it moved from application code into declarative configuration, which is a different discipline with its own failure modes. The policy is still yours to get right; what changed is that it is now expressed once, versioned separately from the services, and enforced uniformly.
Third, this is not free architecture and is easy to over-adopt. Below some scale — a handful of services, one language, a shared library that works — the operational cost of a mesh exceeds what it saves. The honest trigger is heterogeneity plus scale plus a need for uniform policy and mutual TLS that you cannot get any other way.
A picture of it
THE PICTURE #How to readRead the cardinality first. One control plane configures many sidecars, but each sidecar pairs one-to-one with exactly one application — that pairing is what makes the policy language-independent. The many-to-many edge between sidecars is where traffic actually flows, and note that everything hanging off a sidecar is a cross-cutting concern, while the application retains only its own logic.
What became clearer
WHAT CLEARED #The reason to move failure handling out of the program is not that applications handle failure badly, but that failure handling is cross-cutting — it must be identical everywhere and changeable everywhere at once, and code living inside a service inherits that service's language and release cadence. Extracting it into a proxy buys uniformity and immediate central control, and charges you latency, resource overhead, and a new distributed system to operate. What cannot be extracted is meaning: the mesh moves the plumbing, never the semantics of what a call actually does.
Where to go next
ONWARD #- Ambient and eBPF approaches, and whether the per-pod sidecar was an implementation detail all along.
- Idempotency as the precondition that makes automatic retries safe.
- Where an API gateway ends and a mesh begins, given how much the two now overlap.
Key terms
TERMS #| Term | What it means |
|---|---|
| Service mesh | infrastructure that handles service-to-service communication concerns outside application code. |
| Sidecar | a proxy container deployed alongside each application instance, intercepting its network traffic. |
| Data plane / control plane | the proxies that carry traffic, versus the component that configures them and issues identities. |
| Mutual TLS | encryption in which both ends present certificates, giving each workload a verified identity. |
| Circuit breaker | a control that stops sending requests to a failing dependency until it recovers. |
| Cross-cutting concern | a requirement that applies across many components and cannot be cleanly localised in any one of them. |
Every term the collection defines is gathered in the glossary.