Study and demonstrator

Hybrid search: when exact words matter

Christophe Cosnefroy · 13 October 2026

Needle in a haystack, part 4 of 7. What a 30-year-old keyword formula found that meaning-based search missed.

Series

1. A needle in Jules Verne
2. Big context: just read everything
3. RAG: the librarian and its catalogue
4. Hybrid search: when exact words matter ← you are here
5. Agentic search: let the model look
6. RLM: the model that writes its own search
7. The verdict: which architecture, for which haystack

Where RAG left us

In part 3, RAG made 33 novels searchable on one desk-side machine. But two facts were never found, at any size, with any number of excerpts: the name of the sunken continent (Atlantis, renamed Thalassie in my edited text) and the date Nemo claims the South Pole (25 March in my text). Two more sank out of reach as the library grew: the reward for spotting the “monster” fell from rank 3 to rank 48, and the date of the affair from 14 to 24.

The search engine in part 3 worked by meaning only. It turns the question and every excerpt into vectors and returns the closest. That works when the question and the passage talk about the same thing in the same way. It fails when they don’t.

Search engines solved a version of this long before language models. Keyword search, with a scoring formula called BM25, has ranked documents by the words they share with the query since the 1990s. It knows nothing about meaning. It just counts words, and gives more weight to rare ones.

Hybrid search runs both and merges the results. That’s the whole idea of this article.

How it works here

Library terms again: the same librarian as in part 3, with a meaning catalogue, now also has an index of every word in the library. A second reader then checks the best pages before handing them over.

1. Same excerpts as RAG. I reused the 512-token excerpts and their meaning vectors from part 3, unchanged. Any difference in the results comes from what’s added, not from a different cut of the text.

2. A keyword index. Each excerpt also gets a BM25 vector: its words, lower-cased, with common French words removed (le, de, quel…), and reduced to their stem. Rare words weigh more than common ones.

3. Two searches, one list. For each question, the meaning search and the keyword search each return their 50 best excerpts. The two lists are merged with Reciprocal Rank Fusion: an excerpt ranked high in either list ends up high in the merged one. Qdrant does this in a single query.

4. A reranker. The 50 merged excerpts go to a small model, Qwen3-Reranker-0.6B, which reads the question and each excerpt together and scores how likely the excerpt is to answer it. That’s slower than comparing vectors, but much more precise, because it reads both texts side by side instead of comparing two summaries of them. The best k excerpts go to the answering model.

I also ran one version without the reranker (steps 1 to 3 only, 8 excerpts), to see what each piece does.

Preparation cost

The keyword index is almost free. On the single novel it took 2.5 seconds (599 excerpts, 7,685 distinct word stems), against 30 seconds for the meaning index. It runs on the processor; no model is involved.

The reranker is one more model to run: 0.6 billion parameters, served by its own small vLLM instance next to the answering model, using 6% of the machine’s memory. It took three and a half minutes to start the first time, then stayed up.

Results

Accuracy	T1	T2	T3	T4
Big context	100%	95%	❌	❌
RAG, 32 excerpts (best RAG)	100%	85%	79%	71%
Hybrid, 8 excerpts	97%	93%	76%	76%
Hybrid, 16 excerpts	100%	94%	88%	83%
Hybrid, 32 excerpts	100%	94%	88%	84%
Hybrid, 8 excerpts, no reranker	100%	83%	80%	76%

Hybrid search tracks big context on one novel, then keeps going. The gap with RAG widens as the library grows.

Each hybrid figure is 80 answers (T1: 30), so about ±9 points. Read the trends, not the decimals.

1. Hybrid search matches big context, and goes further

On the single novel, hybrid search with 16 excerpts scored 94%, against 95% for big context reading the whole book. That’s the same result, within noise, from about 7,500 tokens of text per question instead of 236,000, and with no six-minute first question.

On 7 and 33 novels, where big context can’t run, it holds 88% and 83%. RAG’s best was 79% and 71%, and it needed 32 excerpts to get there.

2. Keywords found what meaning missed

For every question, I recorded where the excerpt holding the answer ranked in each search: by meaning, by keywords, and after the reranker.

33 novels. Grey circle: meaning search (what RAG used). Orange dot: keyword search. Diamond: the final order the model sees.

Fact	Meaning (RAG)	Keywords	After reranker
Pole date (25 March)	not in top 50	1	1
Tunnel (Sinaï-Tunnel)	47	1	2
Maelström → Corryvreckan	28	1	1
Date (1858)	24	4	1
First sighting (Holloway)	14	7	1
Vigo → Ferrol	10	1	1
Reward	48	29	4
Redoutable (warship)	4	3	10
Atlantis → Thalassie	not in top 50	not in top 50	—

The pole date is the clearest case. The question asks: “À quelle date le capitaine Nemo prend-il possession du pôle Sud ?” The passage says: “moi, capitaine Nemo, ce 25 mars 1860, j’ai atteint le pôle sud […] et je prends possession de cette partie du globe”. By meaning, the search engine never saw them as close, in any tier. By words, possession and pôle are rare enough to put the passage first, every time, in every tier. With the reranker, it was answered correctly 45 times out of 45.

The ranks are identical across all five runs, which is expected: search is deterministic. The variation is all in the answering model.

Across the whole test, meaning search never found a passage that keyword search missed. Every excerpt it found, keywords found too. The reverse was not true. That’s specific to this test: my questions are written in the novel’s own vocabulary, which favours keywords. On questions phrased in other words than the documents, meaning search would earn its place. Hybrid search keeps both, so you don’t have to know in advance.

With 16 excerpts, the passage holding the answer reached the model for 15 of the 16 facts, at every tier from one novel to 33. Only Atlantis was missing.

3. The reranker helps, and sometimes hurts

Compare the two versions with 8 excerpts. On one novel, the reranker is worth 10 points (93% against 83%). On 7 and 33 novels, the difference is within noise (76% against 80%, then 76% against 76%). The average hides two opposite effects.

It rescues facts the merged list buries. Without the reranker, on 33 novels, the reward passage was 46th in the merged list and the first sighting 9th: never in the prompt with 8 excerpts. The reranker moved them to 4th and 1st. The first sighting went from 0 correct answers out of 5 to 5 out of 5.

It also pushes some down. The French warship where the Nautilus stops (the Redoutable) was 5th in the merged list. The reranker moved it to 10th. With 8 excerpts on 7 and 33 novels, that was 10 correct out of 10 without the reranker and 0 out of 10 with it, all NOT FOUND. With 16 excerpts, 10 out of 10 again.

So the reranker doesn’t remove the need to choose k carefully. It changes which facts are at the edge.

4. The remaining failures are reading failures

Finding the passage is no longer the main problem. What’s left happens after the passage reaches the model.

The reward. On 7 and 33 novels, the passage is ranked 2nd to 4th, well inside the prompt. Yet the model answered NOT FOUND 16 times out of 30. On the single novel, at the same rank 2, it was right 15 times out of 15. The difference is what surrounds it: other novels bring other rewards and other bounties into the prompt. My guess is that the model, told to answer only if sure, chose to say nothing. That’s a hypothesis; the test doesn’t prove it.

The Maelström. On 7 and 33 novels, the right passage is ranked 1st. In part 3, RAG answered “the Maelström” (the original Verne name, which no longer appears in my edited novel) 5 times out of 6. Hybrid search behaves differently: at least 18 answers out of 30 gave both names, “Le Corryvreckan (Maëlstrom)”, and only one gave the original name alone. The model now sees the edited text clearly, but other novels use “maelström” for any whirlpool, and the model hedges. My scoring counts an answer with both names as wrong, as in the earlier parts. Strangely, without the reranker it did better (6 correct out of 10), perhaps because the reranker also lifts the other novels’ whirlpool passages. Again, a hypothesis.

Atlantis. Never answered, in any configuration: 0 correct out of 60. On the single novel, keyword search ranked a passage naming Thalassie 31st, too low to survive the merge; on 7 and 33 novels, neither search had it in its top 50. The question (“Quel continent disparu le professeur Aronnax découvre-t-il au fond de l’océan ?”) shares almost no rare words with the passages that name Thalassie, which are full of ancient authors’ names (Plato, Theopompus, Origen). The passage that does match the question, “ce continent immense qui sans doute reliait l’Afrique à l’Amérique”, comes a few paragraphs after the name, in a different excerpt. So the model answered with the description and no name, several times. The answer was one excerpt away.

That last case is a known limit of cutting a book into excerpts: the question matches one excerpt, the answer sits in its neighbour. A common fix is to add each excerpt’s neighbours to the prompt.

5. Wrong answers: about as many, but of a different kind

With 16 excerpts on 7 and 33 novels, hybrid search gave 19 wrong answers out of 160 (12%). RAG gave 21 out of 186 (11%). The same rate.

But every single one of those 19 came from two facts: the Maelström (both names) and Atlantis (a description without a name). No other fact got a wrong answer, with 8, 16 or 32 excerpts. RAG’s wrong answers were spread across facts, and included plain wrong values: “1859” or “1866” for the date. With hybrid search, the date passage is at rank 1, and on 7 and 33 novels the date was right every time. (On the smaller tiers, with 8 excerpts, the model still answered “1866” from memory twice in 10.)

For a business system, that’s a better kind of failure. A hedged answer that names both options can be caught by a human. A confident wrong year can’t.

Cost and speed

	Big context (1 novel)	RAG	Hybrid search
Preparation	none	30 s (1 novel) to 3 min 39 s (33 novels)	RAG’s index + 2.5 s (1 novel) for keywords
Extra model running	none	embedding model	embedding model + reranker
First question	6 min 20 s	~10 s	~10 s
Median answer time	~15 s	9–14 s	8–11 s
Text read per question	236,000 tokens	4,000–15,000 tokens	4,000–15,000 tokens
Text written per answer (median, reasoning included)	~170 tokens	~190–230 tokens	~165–185 tokens
Accuracy, 1 novel / 33 novels	95% / ❌	85% / 71%	94% / 84%

The reranker reads 50 excerpts per question, yet median answer times stayed within a second of the version without it. On this machine, it’s close to free.

What I’d tell a client

Start with hybrid, not plain RAG

• Keyword search is cheap to add: seconds of indexing, no model. Most vector databases support it natively (Qdrant, OpenSearch, Elasticsearch, Postgres with extensions).
• Business documents are full of exact words that matter: product codes, client names, clause numbers, dates, error codes. That’s where meaning-based search is weakest and keywords are strongest.
• Add a reranker, and test the number of excerpts on your own questions. The reranker fixes most ranking problems, but it can also push a good passage just past the cut.

What it won’t fix

• Questions that share no words and no meaning with the passage. Those need a different approach: rewriting the question, adding neighbouring excerpts, or letting the model search for itself (part 5).
• Lookalike documents. When many documents talk about similar things, the right passage can be in the prompt and the model still hesitates. Measure how often your system says “not found” with the answer in front of it.

Limits of this test

• Questions in the novel’s vocabulary. My questions were written in French, close to Verne’s wording. That favours keyword search. Questions written by users who don’t know the documents would favour meaning search more.
• One reranker, default settings. Qwen3-Reranker-0.6B, standard BM25 parameters, 50 candidates from each search. Larger rerankers or tuned parameters could move the numbers.
• Five answers per point. About ±9 points. The no-reranker comparison at T3–T4 is within noise.
• The “rank” diagnostic finds the first excerpt containing the edited answer, as in part 3.
• Chapter labels are still shown with each excerpt. They mattered for two facts in part 3 (Vigo, tunnel); here keyword search finds both passages at rank 1 or 2 anyway.
• Counterfactual facts only, as in the earlier parts.

Next: agentic search

Hybrid search fixed the search problem for 15 facts out of 16. The last one, Atlantis, is one excerpt away from the passage the search finds. And the reward shows that even with the right passage at rank 2, the model can give up.

Both are problems a human reader would solve by looking around: reading the page before, searching for another word, checking a second chapter. In part 5, the model gets that ability. No fixed pipeline: it searches, reads and decides for itself when it has the answer.