Compatibility has a direction
A Socratic walk-through of compatibility has a direction — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why is 'compatible' meaningless until you say which side upgrades first?
Somebody says the schema change is compatible, and everyone relaxes. Then the deploy goes out and half the consumers fail to decode — or they decode fine, and the producers fall over instead. The change really was compatible. It just was not compatible in the direction the rollout happened to take.
So the word by itself carries almost no information. Ask what it is missing, and the answer turns out to be a question nobody asked: which side of this system is going to be running the new code while the other is still running the old?
Reasoning it through
REASONING #Set up the smallest version of the problem. There are writers and readers, and a message written by one is decoded by the other. During any rollout that is not instantaneous — which is every rollout — there is a period with old writers and new readers, or new writers and old readers, and usually both at different moments.
That gives two entirely separate questions, and nothing about one answers the other:
Can new code read data written by old code? And can old code read data written by new code?
Notice these are independent. Adding a field with a sensible default satisfies both — new readers fill in the default when it is missing, old readers ignore what they do not recognize. Removing a required field satisfies only the first: new readers no longer want it, but old readers still demand it and choke. Adding a required field satisfies only the second, and renaming one satisfies neither, because a rename is a deletion and an addition wearing a single commit message.
Now the naming, which is where most of the confusion lives. Compatibility is named from the reader's point of view. A schema is backward compatible when code using it can read data written under the previous schema — it reaches backwards in time. It is forward compatible when the previous schema's code can read data written under this one — the old code copes with a future it has never seen.
And here is the consequence that makes this practical rather than pedantic. Each direction dictates a rollout order, and dictates it in the way that feels wrong at first. If your change is backward compatible only, consumers must upgrade first: they have to be able to handle the new shape before any producer emits it. If it is forward compatible only, producers go first: old consumers can already tolerate the new data, but new consumers cannot tolerate the old. Confluent's Schema Registry states exactly this pairing, and its default setting is backward — which is a bet that in most systems the readers are the easier side to move.
So "is this compatible?" is genuinely unanswerable. The answerable question is "is this compatible in the direction we intend to deploy?", and a registry enforcing one policy is really enforcing a rollout order across everybody who uses the topic.
One thing worth being honest about: these checks are structural. A registry can verify that a field's absence is survivable and that a type still decodes. It cannot see that you kept the field name and changed what the number means — reassigning amount from cents to whole units passes every compatibility check ever written and breaks every consumer. Compatibility is a property of encodings, not of meanings.
The analogy
THE ANALOGY #Think of a two-way conversation between speakers of an evolving language. One of them learns a new word this year. If the new word is a synonym for something the old speaker already knows, either can go first and both are understood. If the new speaker stops using a word the old one relies on, the listener has to learn the new form before the speaker starts using it. If the new speaker adds a word nobody else has heard, the old listener must already be the sort who shrugs at unfamiliar words and carries on.
two people share one conversation and can each notice confusion immediately, whereas a producer never learns that a consumer failed to decode — the failure happens somewhere else, minutes later, in a log the producer's team does not read. That silence is why the direction has to be decided in advance instead of discovered.
Clarifying the model
THE MODEL #The refinement that resolves most arguments: "full compatibility" is not a stricter grade of the same thing but the intersection of the two directions, and it is the only setting under which rollout order genuinely does not matter. It also permits the least — essentially adding and removing optional fields that carry defaults. If a change cannot be expressed that way, you have not failed at schema design; you have discovered that the change requires coordination, and coordination is best done explicitly with a new version of the topic rather than implicitly by hoping.
A misconception worth correcting is that the compatibility setting protects data at rest. It protects a transition. Under backward compatibility, historical data remains readable by current code — which is what lets a new consumer replay a topic from the beginning. Under forward compatibility, historical data may not be readable by anything but the code of its era, which is a very different long-term proposition for a log you intend to keep.
And a practical note on scope: a compatibility check compares against previous versions, but how many is a policy choice. Checking only the immediately preceding version is enough for a rollout, and not enough for a consumer that has been offline for a month.
A picture of it
THE PICTURE #How to readThe horizontal axis asks whether new code can read old data, the vertical whether old code can read new data. Each point is a schema change placed against both. The top-right corner is the only place rollout order is free. The bottom-right corner is backward compatibility alone, which forces consumers to upgrade before producers; the top-left is forward compatibility alone, which forces the reverse. The bottom-left holds changes that satisfy neither, where no ordering of a rolling deploy is safe and you need a second topic or a dual-write.
What became clearer
WHAT CLEARED #Compatibility is not one property but two independent ones, because a rolling deploy always produces a window with old code on one side and new code on the other. Naming which of the two you have is the same act as naming who deploys first — so a registry policy is really a rollout contract. And the check is structural only: it will let through any change that keeps the shape and quietly redefines the meaning.
Where to go next
ONWARD #- How Avro's reader and writer schemas resolve field by field, and why defaults are what make either direction possible.
- When to abandon in-place evolution and version the topic instead, and how consumers migrate across the seam.
Key terms
TERMS #| Term | What it means |
|---|---|
| Backward compatible | new code can read data written under the previous schema; upgrade consumers first. |
| Forward compatible | previous code can read data written under the new schema; upgrade producers first. |
| Full compatibility | both directions hold, so rollout order does not matter. |
| Schema Registry | a service that stores schema versions and rejects a registration that violates the configured compatibility setting. |
Every term the collection defines is gathered in the glossary.