Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
Yuki131 
posted an update 2 days ago
Post
2104
Test-Time Scaling for Rerankers?

Can rerankers scale at test time—not by generating longer reasoning traces, but by selectively using richer document representations?


KaLM-Reranker-V1 supports Matryoshka compression from 1× to 32×, which suggests a progressive multi-fidelity pipeline:

- Embedding retrieval → Top-100
- KaLM-Reranker @ 32× compression → Top-20
- The same reranker @ 2× compression → final ranking

The intuition is simple: cheaply screen many candidates, then allocate higher-fidelity cross-attention only to the most promising ones.

For 100@32× → 20@2×, the passage-token interaction budget is roughly 31.8% of directly running 100@2×, before fixed model overheads. The key question is whether it can retain nearly the same ranking quality.

We’re considering evaluating nDCG–latency Pareto curves.

Would you consider this a useful form of test-time scaling for retrieval?

KaLM-Embedding/KaLM-Reranker-V1-Nano

KaLM-Embedding/KaLM-Reranker-V1-Small

KaLM-Embedding/KaLM-Reranker-V1-Large

KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking (2606.22807)

https://huggingface.co/collections/KaLM-Embedding/lychee-kalm-reranker

KaLM-Embedding
alikhaters

Yes, this seems like a useful retrieval version of test-time scaling. I would be most curious about where the cascade fails: cases where the 32x pass filters out documents that only become obvious with richer interaction. Reporting recall@20 after the cheap stage alongside final nDCG/latency would make the Pareto curve much easier to interpret.

·
Yuki131

Great point—recall@20 after the 32× stage would clearly show what is lost during early filtering. We’ll include it alongside final nDCG and latency. 😏
In this post