Hybrid search

Why does combining a search that understands meaning with one that only matches words beat either alone?

Semantic search understands that "how do I cancel" and "terminating your subscription" are the same request. Keyword search does not; it matches strings. Once you have the model that grasps meaning, keeping the one that only counts words looks like nostalgia — yet nearly every serious retrieval system in production runs both and blends the results.

So either the practitioners are hedging out of timidity, or the older method knows something the newer one structurally cannot. It is the second, and the reason is more interesting than "it is a good fallback." Two methods do not help simply by being two. They help under a specific condition, and naming that condition explains the whole design.

Start with when redundancy pays. If you ask two judges the same question and they are wrong on the same cases, the second judge adds nothing — you have paid twice for one opinion. Redundancy is only worth its cost when the failures are uncorrelated, and it is worth a great deal when the failures are close to complementary: each judge is strong exactly where the other is weak. So the question becomes whether lexical and dense retrieval fail on the same queries or on different ones.

Consider what each method actually computes. A dense retriever maps text to a point in a learned space, so similarity means "close in a space trained on what tends to co-occur and paraphrase." Ask what such a space does with a token it never learned anything about — a part number, an internal project codename, a rare surname, a version string, an error code. The model has no meaningful position for it, so the vector is dominated by the surrounding ordinary words. Query for ORA-01555 and you may get passages about database errors in general, which is the right neighbourhood and the wrong document.

Now ask what a lexical scorer does with that same query. Nothing clever at all — and that is the point. It looks up the term in an inverted index and returns documents containing it, and because the term is rare, its inverse document frequency weight is enormous, so a document containing it outranks everything else decisively. The method with no understanding is near-perfect precisely on the case where understanding has nothing to work with.

Then reverse it. A user types "my card was declined at checkout" and the document says "payment authorisation failure during order submission." Not one content word overlaps, so lexical scoring returns nothing useful — it cannot, since there is no shared term. The dense retriever handles this easily, because paraphrase is what its training data teaches.

So the failure modes are not merely uncorrelated but close to inverse: one fails on rare tokens and succeeds on paraphrase, the other the reverse. That is the condition under which redundancy is most valuable, and it follows from the mechanisms rather than from luck — which is why the effect replicates across corpora instead of being a quirk of one benchmark.

Now the practical problem: having two ranked lists, how do you merge them? The tempting move is to add the scores — but a lexical score is an unbounded sum of term weights and a cosine similarity is bounded, neither is calibrated, and their scales differ per query, so a weighted sum needs normalisation that is itself query-dependent and fragile.

Is there a way to combine them that does not need the scores at all? Yes: keep only the ranks. Reciprocal rank fusion scores each document as the sum, over the lists, of one divided by a constant plus its rank in that list — the constant conventionally 60, from the original formulation by Cormack and colleagues. Rank is comparable across any two rankers by construction, the reciprocal makes the top of each list dominate, and the sum rewards documents both methods liked without letting either score scale swamp the other. It has nothing to tune per corpus, which is why it is the usual default, though a tuned weighted blend can beat it when you have labelled data.

Three refinements.

First, "hybrid" is not a spectrum with a best mixing ratio to be found. The gain comes from coverage of different query types, so the right weighting depends on your query mix — and a system serving both product-code lookups and natural-language questions benefits from running both methods at full strength rather than compromising.

Second, fusion helps recall much more reliably than precision at rank one. Merging two lists makes sure the right document is somewhere in the candidate set; deciding which candidate is best is a different job, usually handed to a reranker. Hybrid retrieval and reranking are complements, not alternatives.

Third, an honest caveat: the improvement varies a great deal by corpus, and there are collections where a strong dense model alone is competitive with any hybrid. What is robust is the mechanism — rare tokens are a structural weakness of learned embeddings — not any published percentage.

• Whether learned sparse retrievers, which produce term weights from a model rather than from counts, collapse the two methods into one or just move the seam.
• How to detect from query logs which band of the split your own traffic falls into, and thus whether hybrid is buying you anything.

Every term the collection defines is gathered in the glossary.