Combine RAG Search Results With Reciprocal Rank Fusion

A help-center assistant searches for account-recovery instructions. Keyword search finds pages containing the exact terms; vector search supplies a different ordering. Which documents should enter the AI’s context? Rather than add unrelated relevance scores, combine the two ranked lists with reciprocal rank fusion (RRF).

This tutorial builds and tests that merge in Python, offline. It does not run a search engine or generate embeddings: the inputs are hand-written rankings, not measured model results. In a RAG pipeline, this step sits after retrieval and before assembling the answer’s context.

1. Prepare comparable ranked lists

Use Python 3.10 or later; no packages, account, or API key are needed. Our imaginary query is “How do I recover account access?” The IDs refer to four source chunks: reset covers reset instructions, password covers password recovery, billing covers invoices, and security covers account protection.

Both retrievers must use the same stable chunk IDs and corpus version, and return best-first lists. The ModernBERT retrieval overview provides background on semantic retrieval; here, we only need the resulting order.

Do not directly add a keyword score to a vector similarity. RRF uses positions instead, so their original scales need not match. The trade-off is that it discards score margins: a narrow lead and a large lead become the same rank difference, as the OpenSearch documentation explains.

2. Fuse the ranks locally

For each occurrence, add 1 / (rank_constant + rank) to that ID’s total. Ranks start at one; an absent ID contributes nothing. This is the rule in the original RRF paper. We use its constant of 60, not an assumed universal optimum.

Save this as rrf_demo.py. Pass a list of lists containing nonempty string IDs. An ID may occur in both lists, but never twice within one list.

from collections import defaultdict
from fractions import Fraction


def rrf(rankings, rank_constant=60, limit=3):
    if type(rank_constant) is not int or rank_constant < 1:
        raise ValueError("rank_constant must be a positive integer")
    if type(limit) is not int or limit < 1:
        raise ValueError("limit must be a positive integer")
    scores = defaultdict(Fraction)
    for ranking in rankings:
        if any(not isinstance(doc_id, str) or not doc_id for doc_id in ranking):
            raise ValueError("IDs must be nonempty strings")
        if len(set(ranking)) != len(ranking):
            raise ValueError("duplicate ID within one ranking")
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] += Fraction(1, rank_constant + rank)
    return sorted(scores.items(), key=lambda item: (-item[1], item[0]))[:limit]


if __name__ == "__main__":
    keyword = ["reset", "billing", "password"]
    vector = ["password", "reset", "security"]
    for doc_id, score in rrf([keyword, vector]):
        print(f"{doc_id}: {float(score):.6f}")

Fraction keeps the arithmetic exact for this small demonstration. Sorting uses descending score, then ascending ID to make exact ties reproducible. Convert scores to floats only for display. A fusion score is a ranking value, not a probability that a document is correct.

3. Verify the arithmetic and edge cases

Save this as test_rrf_demo.py beside the first file. It checks the complete top-three ordering, one hand-calculated score, empty inputs, deterministic ties, and duplicate rejection.

from fractions import Fraction
from rrf_demo import rrf

rankings = [
    ["reset", "billing", "password"],
    ["password", "reset", "security"],
]
fused = rrf(rankings)
assert [doc_id for doc_id, _ in fused] == ["reset", "password", "billing"]
assert fused[0][1] == Fraction(1, 61) + Fraction(1, 62)
assert rrf([[], []]) == []
assert [doc_id for doc_id, _ in rrf([["z"], ["a"]])] == ["a", "z"]
try:
    rrf([["reset", "reset"]])
except ValueError as error:
    assert str(error) == "duplicate ID within one ranking"
else:
    raise AssertionError("duplicate was accepted")
print("PASS: ranking, arithmetic, empty inputs, ties, duplicate guard")

Run both files in that directory:

python3 -B rrf_demo.py
python3 -B test_rrf_demo.py

Expected standard output, reproduced in local fixture tests:

reset: 0.032522
password: 0.032266
billing: 0.016129
PASS: ranking, arithmetic, empty inputs, ties, duplicate guard

Both commands exit with status 0 and produce no standard error. reset leads because it contributes 1/61 + 1/62; password follows with 1/63 + 1/61. A duplicate within one ranking raises ValueError, which the test deliberately catches. Empty rankings return an empty result rather than invented documents.

These assertions establish implementation behavior, not retrieval quality. If human labels mark reset and password relevant, keyword Recall@2 is 0.5, vector Recall@2 is 1.0, and fused Recall@2 is 1.0. Fusion matches, rather than beats, the stronger input here. Use Recall@k and MRR on a held-out query set before changing a real system.

4. Connect the merge to real retrieval

Replace the fixture lists with IDs returned for the same query, applying the same access and metadata filters to both searches. Retrieve a candidate window from each, fuse it, then select the final context chunks. Never use fusion to bypass source permissions.

Candidate depth and final output count are separate controls. The Elasticsearch RRF reference distinguishes rank_window_size from final result size and currently defaults its rank constant to 60. Our limit controls only the final count; the supplied lists define the candidate windows. Fusion cannot recover a chunk missing from every input.

5. Troubleshoot unexpected rankings

• One source appears twice: align chunk IDs across retrievers. Do not merge solely by title; different chunks can share one.
• Results change between runs: freeze the corpus snapshot, input ordering, candidate depth, and constant. This tie-breaker cannot stabilize changing inputs.
• An irrelevant chunk ranks highly: agreement is not proof of relevance. Inspect both inputs and the human labels before adjusting settings.
• A retriever returns nothing: this code uses the remaining list. Log the missing input and distinguish no matches from a service failure.

Practical takeaway

Use RRF to combine ranked evidence without pretending different search scores are comparable. Keep stable IDs, test the merge, and compare the fused ranking against each retriever on real labeled queries before feeding it to an AI assistant.