Study and demonstrator

RLM: the model that writes its own search

Christophe Cosnefroy · 17 October 2026

Needle in a haystack, part 6 of 7. The library is a variable; the model writes Python, and it never calls itself.

Series

1. A needle in Jules Verne
2. Big context: just read everything
3. RAG: the librarian and its catalogue
4. Hybrid search: when exact words matter
5. Agentic search: let the model look
6. RLM: the model that writes its own search ← you are here
7. The verdict: which architecture, for which haystack

Where the agent left us

In part 5, an agent with four tools (list the novels, list the chapters, grep, read) found the edited passage in 94% of questions or more, and answered 88% correctly on 33 novels. What it couldn’t do was anything I hadn’t written a tool for.

Recursive Language Models (Zhang and Khattab, 2025) remove the tool list. The model gets a Python interpreter, the whole library in one variable, and one extra function: it can hand a piece of text to another copy of itself. Whatever search it needs, it writes.

In library terms: no catalogue, no librarian, no shelves. The reader gets the entire library as one very long scroll, a programming language, and assistants they can send to read parts of it.

How it works here

The model never sees the text in its prompt. It sees the question and one line: “context holds 18,111,017 characters”. Then it writes code.

import re
pat = re.compile(r'p[ôo]le\s+[-–—]?\s*sud', re.I)
ms = list(pat.finditer(context))
print("matches", len(ms))
for m in ms[:5]:
    print(m.start(), context[m.start()-400:m.end()+400])

The code runs, the model reads what it printed, and writes the next block. When it has the answer, it writes FINAL(...).

What the model can call inside its code

Function	What it does
context	the whole tier as one string: 97,000 characters (T1) to 18.1 million (T4)
llm_query(prompt)	sends a prompt to another call of the same model, which sees only that prompt (up to ~98,000 characters), and returns its answer
llm_query_batched(prompts)	the same for a list, run 8 at a time
FINAL(answer) / FINAL_VAR(name)	ends the question

The sub-calls are the “recursive” part. They are plain model calls, without the interpreter, so they can’t call further copies themselves (recursion depth 1, as in the paper’s main experiments).

Same rules as every part: same Qwen3.8-27B model, same instruction (“answer only from the text, or say NOT FOUND”), same reasoning settings and 4,096-token output allowance per call, same 16 edited facts, same four corpus sizes, same scoring, 5 runs per question. As in part 5, one line is added: the library may have been edited, trust only the text in context.

Budgets per question: 20 turns, 60 sub-calls, about 32,000 tokens per sub-call, 12,000 characters of printed output shown per turn (the same cap as part 5’s tool results), and 30 seconds of computing per block of code. Running out of turns counts as NOT FOUND.

The loop is about 300 lines in my test harness. I didn’t use the authors’ package: I wanted every block of code logged, the same scoring as the other parts, and the interpreter in a separate container. One rule follows part 5: a turn that runs code doesn’t answer. If the model writes FINAL(...) next to a code block, before seeing its output, the answer is set aside and the model sees the output first. (That happened in a pilot; in the full run, never.)

A model that runs its own code needs a cage

The model writes arbitrary Python, so it runs in a container with no network at all. Not a restricted network: none. It talks to the harness through two local sockets, one to receive code and one to send sub-calls. The harness forwards the sub-calls to the model and counts them.

Before the run, a check tries to break out. All ten checks passed:

Check	Result
Reach the internet	blocked
Reach the model server directly	blocked
Write to the system or to the corpus	blocked (read-only)
Run as root	no: unprivileged user
Infinite loop	stopped after 30 s, the interpreter still usable
Sub-calls through the harness	working

That’s the part of RLM nobody puts on a slide. Agentic search in part 5 called four functions I wrote. RLM executes whatever the model writes. On your own data, that’s a security project before it’s an AI project.

Results

Accuracy	T1	T2	T3	T4
Big context	100%	95%	❌	❌
RAG, 32 excerpts	100%	85%	79%	71%
Hybrid, 16 excerpts	100%	94%	88%	83%
Agent, grep + read	97%	91%	90%	88%
Agent + search	100%	95%	84%	88%
RLM	100%	94%	89%	91%

Each RLM figure is 80 answers (T1: 30), so about ±9 points. RLM, the two agents and hybrid search are within noise of each other on 1 and 7 novels. On 33 novels, RLM and both agents are ahead of hybrid search and far ahead of RAG.

On 33 novels, RLM gives the most correct answers and takes the longest. The agent with search is the best compromise on speed.

1. The model never called itself

In 270 questions, the model made zero sub-calls. Not on one novel, not on 33.

It didn’t need them. Every question was solved with regular expressions, slices of context and print: a median of 3 turns on the opening chapters and on one novel, 4 on 7 and 33 novels, with 2 to 3.5 blocks of code per question. Almost no code errors, and no block ever hit the 30-second limit.

The check before the run proved the sub-calls worked. The model simply chose not to use them, for reasons I can partly explain:

• These questions are grep-shaped. Each edited fact is a short name or date near distinctive words (“prime”, “pôle”, “gouffre”). A regular expression finds the passage in a few turns, almost for free. A sub-call costs a full model call on up to 32,000 tokens and finds nothing a regex would miss. The RLM paper reports the same thing on single-needle tasks: models filter with code, and sub-calls pay off on tasks that need meaning across most of the text, such as counting or aggregating.
• My instructions point to code first. The tips in the prompt say: look at the structure, search keywords with re, and use sub-calls “if keywords aren’t enough”. They always were.
• On 33 novels, reading everything isn’t possible. 18.1 million characters in pieces of 98,000 is about 185 sub-calls. The budget is 60. On one novel, 9 would have covered everything; the model didn’t try.
• The model is a coder. Writing a regex is what it has been trained to do. llm_query is a function it has never seen.

So, as used here, RLM is an agent that writes its own grep. That’s the honest result of this part, and the main limit of it.

2. It navigates by numbers

The code shows how the model thinks about the text: as positions. A typical search in a pilot run on 33 novels:

1. Search pôle sud everywhere: 31 matches, print five with their position.
2. Notice that the matches from Vingt mille lieues sous les mers sit around character 2,050,000.
3. Slice context[2050000:2300000] and search dates inside it.
4. Narrow to context[2094000:2110000]: “Sommes-nous au pôle ? … A midi nous ferons le point.”

No chapters, no titles: offsets, like scrolling a document by its scrollbar. Part 5’s agent asked for chapter 14 of a named novel; RLM asks for characters 2,094,000 to 2,110,000.

Its very first instinct, in the check before the run, was print(repr(context)): print the entire library. Output is capped at 12,000 characters per turn, so it got the first few pages and had to change approach. A cap that looks like a technical detail is what makes the method work at all.

3. It finds the passage, and mostly believes it

	T1	T2	T3	T4
Answer passage printed by the model’s code	100%	98%	90%	95%
Correct when the passage was printed	100%	96%	99%	96%

The edited passage reached the model in 90–100% of questions. When it did, the answer was right 96–100% of the time. On the opening chapters, both were 30 out of 30.

Three facts tell the story against the earlier parts:

Correct, on 1, 7 and 33 novels	Hybrid, 16	Agent, grep + read	Agent + search	RLM
The reward	—	10/10 (7 and 33 novels)	10/10 (7 and 33 novels)	15/15
Atlantis → Thalassie	0/15	5/15	3/15	9/15
Maelström → Corryvreckan	—	11/15	4/15	11/15

Atlantis is the surprise. No search found it in part 4; the agents found it every time and then mostly added “l’Atlantide” from memory. RLM answered plain “Thalassie” 9 times out of 15. It still hedged 5 times (“La Thalassie, l’ancienne Atlantide de Platon”). Why it hedges less, I can’t prove from this test. One visible difference: the agents read snippets and paragraphs the tools chose; RLM printed slices it chose itself, often long ones.

Some answers look like hedges and aren’t: “La Thalassie, l’ancienne Méropide de Théopompe” is in Verne’s own text, and was scored correct.

4. What went wrong

On 1, 7 and 33 novels together, RLM gave 21 wrong answers out of 240 (9%). Three kinds:

• Found it, then mixed in memory (7). Atlantis with both names (5), and twice on 33 novels the Maelström: the model printed the edited passage and still answered “le Maëlström”.
• Never looked (6, all on 7 and 33 novels). The model answered on its first turn without writing any code. Three times with a name: “Farragut” and “l’Abraham Lincoln”, the original story’s names, and “l’île aux Coraux”, which is in neither version. Three times its reasoning used the whole output allowance before writing a single line. These took from 35 seconds to over 3 minutes. The first message told it to explore context before answering.
• Searched, never reached the passage (8). The pole date, three times, answered “19 mars”. The Maelström twice and Atlantis once, from memory. One question ran out of turns (20), one out of output tokens mid-search.

The “never looked” errors are new. Part 5’s agent always called a tool first. Here, given an interpreter instead of tools, the model sometimes skips the work and answers from what it knows. On the opening chapters and one novel that never happened; on 7 and 33 novels, 6 times in 160.

“19 mars” isn’t memory: it’s in the text. A few paragraphs before the pole, Verne writes “ce jour mémorable du 19 mars”, the morning the Nautilus surfaces near the pole. The model found the right chapter, read a date that sounded like the answer, and stopped before the sentence with the edited date. It’s the only plain wrong value RLM gave, and it came from the documents, not from training.

The pole date is the opposite of part 5: the agent with grep got it 15 times out of 15, often in one search. RLM got it 12 times. In the pilot trace I read, its own code got in the way: it searched month names case-insensitively, so “mai” matched every “Mais” and “maître”, and the noise filled the output before March. Writing your own search means writing your own bugs.

Cost and speed

	Hybrid search	Agent, grep + read	Agent + search	RLM
Preparation	embeddings + keywords: 30 s to 3 min 39 s	none	same as hybrid	none
Extra running	embedding model + reranker	none	embedding model + reranker	a sandbox container (CPU, 4 GB)
Steps per question (median)	—	2–3 tool calls	1 tool call	3–4 turns
Median answer time, all sizes	8–11 s	16–26 s	11–12 s	19–44 s
Median answer time, 33 novels	9 s	26 s	11 s	44 s
Slowest 10%, 33 novels	35 s	2 min 35 s	40 s	1 min 36 s
Tokens read per question, 33 novels (median, all calls)	7,700	10,600	5,600	11,000
Tokens written per question, 33 novels (median; slowest 10%)	180; 780	480; 3,500	190; 720	840; 2,000
Accuracy, 1 / 33 novels	94% / 83%	91% / 88%	95% / 88%	94% / 91%

RLM needs nothing prepared, like the agent with grep, and costs the most time per answer: a median of 44 seconds on 33 novels, against 26 for the agent with grep and 11 with search. Its slowest answers are shorter than the agent’s, though: the agent’s long tail came from reasoning that never ended.

It also writes the most: about 840 tokens per answer on 33 novels, against 480 for the agent and 180 for hybrid search. Writing code, then reasoning about its output, costs tokens at every turn, and that’s where the time goes. Its token count grows with the library: about 380 on the opening chapters, 540 on one novel, 720 on 7 novels.

It’s also the only option here that needs a security design.

What I’d tell a client

When RLM is worth it

• When the questions need code, not just search: counting, comparing, extracting from thousands of places. That’s what the method was designed for, and this test doesn’t measure it.
• When you can’t predict the tools. An agent only does what its tools allow; RLM writes what it needs.
• Not for finding one fact. On these questions it behaved like an agent writing its own grep: as accurate, slower, and with a sandbox to maintain.

What to watch

• It executes code the model writes. No network, read-only, unprivileged, with time and memory limits, and test the cage before every run.
• Cap the output. The model’s first reflex was to print everything.
• Check that it looked. Some answers came from memory in the first turn, without a line of code. Log the code, and reject an answer with no search behind it.
• Don’t assume the recursion happens. In 270 questions, the model never delegated. If your use case depends on sub-calls, measure them.

Limits of this test

• Sub-calls never ran, so this part measures RLM’s code-writing, not its recursion. A prompt without my regex-first tips, or questions that need reading everything, would test the other half.
• My prompt and my loop. The tips I gave shape the behaviour, as any system prompt would. I wrote my own loop rather than using the authors’ package, so results with their prompt may differ.
• One model, Qwen3.8-27B on one desk machine, with a 4,096-token output allowance per turn. A larger model may write better code, and may delegate.
• Questions in Verne’s vocabulary, which favours keyword search, as in parts 4 and 5.
• About ±9 points per figure. RLM, the agents and hybrid search are within noise of each other on 1 and 7 novels.
• “Printed” means the edited answer appeared in the output the model read, not that it read it carefully.

Next: the verdict

Six architectures, one model, one machine, the same 16 edited facts. Part 7 puts them side by side: which one to use, for which haystack, and what each costs to run. → Part 7: the verdict