Jev is a decision model: given a question and a set of options, it scores how well each option fits. In an earlier post we used it to pick an agent's next tool. Retrieval-augmented generation is full of the same kind of choice. Which passages answer this question? Which app holds the answer? Is what the agent found enough to answer?
We tried Jev at two of those points on a large, noisy corpus, and measured both. One of them is now a Ragwalla search option. The other lost, and the way it lost says something general about reranking.
The test
The corpus is EnterpriseRAG-Bench, Onyx's benchmark for retrieval over company documents: 511,962 synthetic internal documents of one company across nine apps (Slack, Gmail, GitHub, Google Drive, Confluence, Jira, Linear, HubSpot and Fireflies), with misfiled documents, near-duplicates and conflicting versions on purpose. Each question names the documents that answer it, its gold documents.
Every number below is a retrieval probe, with no agent in the loop: the question's own text as one Ragwalla hybrid search (vectors from voyage-4-large and a keyword index, weighted equally) over all 511,962 documents, scored by the share of gold documents among the top 10 documents returned. The questions are the 236 of the benchmark's development half that have gold documents. We tune only on that half; the other half stays unseen.
Layer 1: reranking, and how deep to read
A hybrid search returns candidates ranked by vector similarity and keyword match. Reranking hands those candidates to Jev with the question, and Jev scores each passage on whether it answers it. In Ragwalla it is one search option:
{"ranking_options": {"ragwalla": {"reranker": {"model": "typesafe/jev", "candidates": 100}}}}
candidates is how many results Jev reads from each side of the hybrid search, up to 100. The deeper it reads, the more it finds:

| no reranker | Jev, 50 a side | Jev, 100 a side | Jev 100 + fusion | |
|---|---|---|---|---|
| gold documents in the top 10 | 77.8% | 83.0% | 85.2% | 87.0% |
| mean reciprocal rank | 0.622 | 0.815 | 0.837 | |
| median seconds per search | 4.2 | 3.7 | 5.6 |
The reason depth matters at this scale: the top 10 a search returns holds 72.5% of the gold documents, but the top 50 from each side hold 83.7%. The answer is usually somewhere in the pool, just not at the top of it. A reranker can only promote what it reads.
Fusion: keep what retrieval was right about
Reranking alone has a failure mode, and it showed up by question type:

Semantic questions, which paraphrase the documents instead of quoting them, gain 19 points: this is what a reranker is for. But project_related questions need several documents from different apps together, and Jev, scoring each passage alone, fills the top 10 with the closest passages from one source. They fall 14 points.
The fix is to not throw away retrieval's order. Ragwalla now merges Jev's order and the search's order by reciprocal rank fusion (k = 10), so a passage both agree on rises, and one that only retrieval ranked well keeps a place. We chose the fusion offline on the development half and checked it on the held-out half before shipping it: 72.1% there against 68.7% for Jev alone and 67.1% for retrieval alone. Fused, project_related comes back to 75% and semantic keeps 79%. Fusion is now part of every reranked Ragwalla search.
Cost: Jev is $0.042 per million input tokens, so reading 100 candidates a side costs about $0.004 a search, plus a second or two.
Layer 2: routing to the right app, and why it lost
Half of the gold documents in this corpus sit in two small apps. Confluence and Jira are 2.2% of the documents and hold 47.5% of the development half's gold; Slack is 55.8% of the documents and 11% of the gold. In a search over everything, the chat crowds the specs and tickets out.
So we built filters that act before ranking: a Ragwalla store can declare file attributes such as source_type as filterable, and a filtered search narrows both the vector index and the keyword index before anything is ranked. If the search is told the right app, it finds a lot more: 80.8% of gold documents against 69.8% (Confluence 59.8% to 81.4%, Jira 73.7% to 89.5%).
But someone has to say which app. We asked GPT-5.4 to name the two likeliest apps for each question, given one generic line per app:

It names Confluence when the answer is in Confluence, but almost never names Jira or Linear: which tracker holds what is one company's habit, not something a question says. A router built on a model's guess would miss most of the gain.
The obvious way around routing is not to route: search every app separately, take the best few from each, and let Jev pick among them. That way small apps always get a seat at the table. We measured it on the same questions, nine filtered searches per question against one unfiltered search reranked over 90 candidates:

| gold documents in the top 10 | questions with all their gold | |
|---|---|---|
| unfiltered, no reranker | 69.8% | |
| top 10 from each app, round robin | 51.1% | 120 |
| top 10 from each app, by score | 57.7% | 133 |
| top 10 from each app, reranked by Jev | 72.8% | 171 |
| one unfiltered search, Jev over 90 | 77.7% | 194 |
| told the right app (oracle) | 80.8% |
Fan-out lost, and Jev was the reason it came close at all. Merging the apps' lists without Jev is worse than no filter: each app's top 10 brings its nine weakest results along with the strong ones, and their scores do not compare across apps. With Jev the merge beats a plain search, but one search with Jev reading 90 candidates from the whole store beats it by five points. The ten best Slack messages for a question are mostly noise, and the ten best Confluence pages are all there is from Confluence, even when the answer is the eleventh.
The lesson for Jev is the same as in layer 1: a reranker wants depth, not spread. Give it more candidates from the ranking that is already good, rather than a fixed share from every bucket.
We kept the filters. They are worth it whenever the caller actually knows the app: a user who picks Jira in a search box, or an agent that has already seen where a project's tickets live.
What this did and didn't prove
These are retrieval numbers. End to end, our best scored run (GPT-5.4 as the agent, Jev 100 with fusion) is 81.21 on the leaderboard's measure over the 251 development questions, against 90.91 for fifth place. We have not run the same agent without Jev on all 251 questions, so we can't yet claim how many of those points are Jev's: runs of 105 questions differ by more than the gain, and reruns flip about 40% of failures.
What reranking can't fix showed up in the failures. 21 questions were wrong in all three of our scored runs. In 10 of them no search ever returned a gold document: the question paraphrases the documents' jargon ("attention memory footprint" for "KV cache footprints"), and the gold never enters the 100 candidates Jev reads. In 7 the gold came back at rank 2 to 8, below a near-duplicate that also fits the question: the same kind of action item for another customer, with another date. Jev scores a passage on whether it answers the question, and both do.
Where Jev goes next
Two more places where RAG makes a decision Jev is built for, both untested:
- Is this enough to answer? After a search, score whether the passages answer the question, and search again before the model answers. Most of our remaining losses are answers from the wrong one of two look-alike documents, which this would need to catch.
- Which passages to keep. Drop reranked passages below a fit threshold instead of always passing ten, so the answering model sees fewer near-duplicates.
The reranker, fusion and filters before ranking are live in Ragwalla today: ranking_options.ragwalla.reranker on any vector store search, and filters on any declared attribute key in vector and hybrid search.