Citation without verification
A Socratic walk-through of citation without verification — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why does an answer that cites its source still need checking?
An answer arrives with a footnote. A named report, a page number, a link. Something in us relaxes at the sight of it, and the relaxing is the interesting part — because what changed? Not the claim. The claim is the same sentence it was a moment ago. What arrived alongside it was a promise about where to look.
So let me ask the awkward question first: when you accept a cited answer without following the citation, what exactly have you accepted? You have accepted the claim, and you have accepted, on the same authority, the assertion that a document supports it. The footnote did not bring evidence. It brought a second claim, made by the same source, about the first.
Reasoning it through
REASONING #Consider what has to be true for a citation to do its work. The source must exist. It must be the source named. It must actually contain the passage. And the passage must mean what the claim says it means. Four conditions — and notice that a citation, on its own, testifies to none of them. It only asserts all four at once, compactly.
Now, why would a language model produce a citation that fails one of those conditions? Not from malice, and not really from ignorance either. Think about what the model was trained to do: continue text plausibly. In its training data, sentences of a certain kind are followed by citations of a certain shape — an author surname, a year, a journal that publishes on that topic. A well-formed citation is exactly the sort of thing that follows a scholarly claim. Producing one is the model doing its job. Whether the paper exists is a separate question that the generating process never asks.
Does this mean citations from a model are worthless? No — and this is where the reasoning has to stay careful. Models do memorise many real references, and a citation to a famous paper is usually right. The trouble is that the fluency of the citation carries no information about which case you are in. A fabricated reference and a real one are produced by the same machinery and look identical. There is nothing in the surface form to sort them.
Then there is the case people find most surprising, so hold onto it: the source exists, the link resolves, the document is real — and it still does not support the claim. The model retrieved a genuine passage on a related topic and wrote a sentence that reads as its summary but overstates it, or reverses a qualifier, or attributes to the paper's authors a position they were describing in order to reject. Verification that stops at "the link works" catches none of this. Checking that a citation resolves is a different act from checking that it supports.
Retrieval systems change the arithmetic here but do not close the gap. If the pipeline fetches real documents and shows the model only those, fabricated references become rare — a large and genuine improvement. But the last step, the one where prose is written about the passage, is still generation. The failure simply migrates from the reference to the paraphrase, where it is harder to spot, because now every link works.
There is an honest uncertainty worth naming: how often each failure mode occurs varies enormously by model, by domain, and by whether retrieval is in play, and published rates go stale within months. I would not quote you a number. The structural point does not depend on one.
The analogy
THE ANALOGY #Think of a receipt. Someone tells you they paid for lunch and hands you a slip of paper. You feel settled — but a receipt only proves anything if you read the amount, check the date, and see that it is from this restaurant rather than a different one. Handed over and glanced at, a receipt is a prop: a physical object whose shape says "an accounting happened here" without anyone having done the accounting.
a real receipt was at least produced by a till that witnessed a real transaction, whereas a generated citation is produced by a process that never observed the source at all — so the paper slip's existence is weak evidence, while the citation's existence is, on its own, none.
Clarifying the model
THE MODEL #The refinement that ties this together is the difference between evidence and a pointer to evidence. A citation is a pointer. Its value is entirely realised at the moment someone dereferences it — opens the document and reads the passage. Until then it is a stored intention to check, and an unchecked pointer confers no more support than the bare claim did.
That reframes what "citing sources" buys you. It does not make an answer more likely to be true. It makes it auditable — cheap to falsify. That is genuinely valuable, and it is a different property from accuracy. A confident uncited answer and a confident cited answer may be equally wrong; only one of them tells you where to go to find out.
Which lets us correct the misconception directly. The instinct says: citation present, therefore checked by someone. The reality is: citation present, therefore checkable by you. The work was not done upstream. It was made possible upstream and then handed to you, and the comfortable feeling arrives at the handover rather than at the completion.
One practical consequence follows. If you are going to check only some citations, check the ones doing the most load-bearing work — the specific number, the surprising finding, the claim you would act on — rather than the first one on the page. And when you check, read the passage, not just the URL.
A picture of it
THE PICTURE #How to readThe horizontal axis asks only whether the document exists and is the one named; the vertical asks whether its text actually supports the claim. Checking that a link resolves moves you rightward and tells you nothing about height. Only the top-right quadrant is a sound citation, and the bottom-right cell is the one a link-check waves through — a real, working source that does not say what the sentence claims it says.
What became clearer
WHAT CLEARED #A footnote is not evidence; it is an address where evidence may be found. What it reliably supplies is auditability — the ability to be caught — and that is worth having precisely because it is not the same as being right. The unease you should feel is not about models inventing papers, which is the easy failure to catch, but about the citation that resolves perfectly and still misrepresents its source.
Where to go next
ONWARD #- How retrieval-augmented generation shifts the failure from fabricated references to unfaithful summaries of real ones.
- What an automated citation-support check can and cannot decide, given that judging support is itself a language task.
- Why the same reasoning applies to a human research assistant's bibliography, and how scholarly norms evolved around it.
Key terms
TERMS #| Term | What it means |
|---|---|
| Citation | a pointer to a source, asserting that a specific document supports a specific claim. |
| Auditability | the property of an answer that makes it cheap to check, distinct from the answer being correct. |
| Retrieval-augmented generation | fetching real documents and conditioning generation on them, which reduces invented references without removing unfaithful paraphrase. |
| Attribution | the stricter requirement that a claim be entailed by the cited passage, not merely adjacent to it. |
Every term the collection defines is gathered in the glossary.