Opaque cursors
A Socratic walk-through of opaque cursors — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why must a pagination marker be meaningless to the client that carries it?
A server hands back a page of results and, with it, a string: eyJvIjoxMDAsInMiOjd9. The client is expected to send that string back to get the next page, and is expected never to look inside it. Documentation for such fields tends to say something almost scolding — treat this as an opaque token, do not parse it, do not construct one.
That is a strange instruction. The client is carrying the value; why must it be forbidden to understand it? Nothing is hidden for secrecy here — often it is trivially decodable. So the prohibition must be doing some other kind of work.
Reasoning it through
REASONING #Begin with the simplest alternative and watch it fail. Suppose the marker is just an offset: page two means skip twenty. What happens if, between the client's first and second request, someone deletes an item near the front of the list? Everything shifts up by one, and the item that was at position twenty is now at nineteen — the client never sees it. Insert an item instead, and one row is returned twice. The client has done nothing wrong; the coordinate system moved beneath it.
So consider a marker that names a position rather than a count: "resume after the record whose sort key is X". Now a deletion in front of X does not move X, and the boundary survives. This is keyset pagination, and it is both more correct and usually much faster, because the database can seek to X on an index instead of counting past twenty rows. Notice what we just did, though — we changed the marker's internal structure while keeping its role identical. If clients had been reading the old offset, that change would have broken every one of them.
That is the first half of the answer. Opacity is a boundary around a decision the server needs to keep changing: how to resume. Today it is a sort key; tomorrow it may need to carry a shard identifier because the collection was split, or a snapshot identifier so the page comes from a consistent view of the data, or an issue time so stale cursors can be rejected, or a signature so a tampered one is detected. Each of those is a genuine engineering need, and each adds a field. If the marker is declared meaningless, all of them are free to add. If it is declared to be an offset, none of them are.
Now the second half, which is about people rather than data. What happens if a client can read the cursor? Someone will. They will decode it, see an offset, and build a page-jump feature by adding twenty. It will work — until the encoding changes, at which point the server's authors will be told they broke a contract they never made. This is the phenomenon Hyrum's law describes: with enough users, every observable behaviour of a system is depended upon by somebody, regardless of what the specification promised. Opacity is the pre-emptive answer. It does not make the value unreadable; it makes reading it unsupported, and that distinction is the entire mechanism.
Is that enforceable? Only weakly, which is worth saying plainly. Base64 of a small JSON object is the common implementation and anyone can decode it in a second. Servers that care can encrypt or sign the token, which converts a social prohibition into a technical one — and that is precisely why some do. The Model Context Protocol takes the lighter path: it specifies an opaque nextCursor string that clients must not parse or construct, and leaves the encoding entirely to the server.
One consequence follows immediately and often surprises people. Because the client cannot construct a cursor, it cannot jump to page seven, and it cannot resume a traversal it did not start. Pagination becomes strictly forward and strictly sequential. That is not an oversight; it is the honest cost of the guarantee. A numbered-page interface is offering something a changing collection cannot actually deliver.
The analogy
THE ANALOGY #Think of a cloakroom ticket. The number on it means nothing to you — it is not the peg's position, and adding one to it does not get you the next coat. It means something only to the attendant, who can rearrange every peg overnight and still return your coat tomorrow. If the ticket instead read "third peg from the left", you would hold a promise about the room's layout, and the attendant could never tidy up.
a cloakroom ticket is valid until you use it, whereas a cursor often embeds a snapshot or an issue time and may legitimately expire, so a client that pauses too long can find a perfectly well-formed cursor rejected.
Clarifying the model
THE MODEL #Three refinements are worth making.
First, opacity is not privacy. A cursor usually contains nothing sensitive, and the point is not to prevent the client from learning the internals — it is to prevent the client from depending on them. Where the token does carry something that must not be tampered with, signing it is the answer, and that is a separate design decision layered on top.
Second, opacity does not by itself give consistency. A cursor that is merely a sort key still shows a client rows inserted after the traversal began, and hides rows deleted during it. If a genuinely consistent snapshot is required, something in the cursor has to name that snapshot, and the server must keep it alive for as long as the cursor is valid — which is real cost, and the reason such cursors expire.
Third, the absence of a cursor is the terminator, and it must be distinguished from an empty page. A page with zero items but a cursor present means "keep going" — servers that filter after fetching can honestly return that. A page with no cursor means the traversal is finished. Clients that stop on an empty page rather than on a missing cursor will silently truncate results, and this is one of the most common pagination bugs in practice.
A picture of it
THE PICTURE #How to readfollow the loop between holding a cursor and presenting it — that cycle is the whole of normal pagination, and the client never leaves it knowing what the token says. The branch to the tampered state is the failure the opacity rule exists to forbid; note that it leads to rejection only later, when the server changes its encoding. The only exit is a page that arrives with no cursor attached.
What became clearer
WHAT CLEARED #The prohibition is not about secrecy but about coupling. A marker declared meaningless leaves the server free to change how resumption works — keyset, shard, snapshot, expiry, signature — without breaking anyone, and it forecloses the page-arithmetic that a shifting collection could never have honoured anyway. The client gives up the ability to jump, and gets in exchange a boundary that will not move under it.
Where to go next
ONWARD #- What it takes to offer stable, consistent pagination over a collection that is being written to throughout the traversal.
- Why signed or encrypted cursors are worth the cost in a public API, and what threat they actually close.
- How infinite-scroll interfaces conceal the loss of numbered pages, and where that concealment leaks.
Key terms
TERMS #| Term | What it means |
|---|---|
| Offset pagination | resuming by counting past a number of rows; simple, but wrong when the collection changes mid-traversal. |
| Keyset pagination | resuming after a named sort key, so insertions and deletions before that key do not shift the boundary. |
| Opaque token | a value a client must pass back unmodified and must not parse or construct. |
| Hyrum's law | the observation that with enough users, every observable behaviour of a system becomes something somebody relies on. |
| Snapshot | a consistent view of the data as of a point in time, which a cursor may reference and which costs the server something to retain. |
Every term the collection defines is gathered in the glossary.