OpenRouter releases RAG reranking tutorial and benchmark
Understand this story
OpenRouter has released a RAG reranking tutorial, stating that vector retrieval often ranks similar but incorrect chunks at the top; a cross-encoder reranking model can be added to rescore the top 30 candidate chunks returned by retrieval against the question in order to correct the ranking. In the example given in the tutorial, after adding voyageai/rerank-3-lite reranking, accuracy improved to 97%, at a cost of about $0.10 per 1,000 queries; the reranking call itself adds $0.10, so under this setup the net increase is $0.04 per 1,000 questions. OpenRouter states that this result comes from a small synthetic dataset, that actual gains depend on the user's documents, and it provides a process for measuring your own baseline; running it requires configuring the OPENROUTER_API_KEY environment variable and installing numpy and requests.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- OpenRouter AnnouncementsFeaturedReranking in RAG: How to Fix Bad Retrieval with a Reranker
This guide explains that vector search often ranks near-miss chunks first because embeddings compress whole chunks into single vectors, and shows how adding a cross-encoder reranker over the top 30 candidates raised precision@1 from 81% to 97% on a 274-question synthetic benchmark at roughly $0.10 per 1,000 queries. It compares 11 rerankers (voyageai/rerank-3-lite and rerank-3 were the most accurate dedicated models and cheapest, while LLM rerankers like openai/gpt-6-luna hit 99.6% but took about 4 seconds per call), covers candidate-count tuning, input_type settings, and silent failures such as cohere/rerank-v3.5's 4,096-token limit. It includes a Python example using the embeddings and rerank APIs and notes that real-world gains depend on your own documents and evaluation set.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 13 09:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.