AsteroRank-Lepton-MLX-DWQ-4bit

MLX port of AsteroRank-Lepton, a 4B SOTA open-weights reranker for STEM and general search, for Apple Silicon.

The AsteroRank blog post shows more information about AsteroRank model family which includes Lepton, Drachma which is better, and Talent which outperforms every reranker known to date. The blog post presents full evaluation of 3 models, the idea of context reranking, and how to prompt search agents to write good context.

The weights are 4-bit (group size 64) with distilled weight quantization (DWQ): the quantization scales and biases are trained to match the bf16 model's output distribution, which recovers most of the accuracy lost to plain 4-bit rounding.

Installation

In your project, install MLX-LM and fetch the reranker client; the weights download on first use:

pip install "mlx>=0.32.4" "mlx-lm>=0.32.2"   # uv project: uv add "mlx>=0.32.4" "mlx-lm>=0.32.2"
hf download Asteromorph/AsteroRank-Lepton-MLX-DWQ-4bit rerank.py --local-dir .

Usage

from rerank import MLXReranker

reranker = MLXReranker("Asteromorph/AsteroRank-Lepton-MLX-DWQ-4bit")

query = "para-selective borylation aryl sulfonate ion pair"
context = (
    "The article discusses para-selective iridium-catalyzed C–H borylation of aryl sulfonate salts "
    "using ion-pairing with bulky ammonium cations to shield ortho and meta positions, enabling high "
    "para selectivity."
)
documents = [
    "A bulky bipyridine ligand on iridium steers C–H borylation of monosubstituted arenes to the para position through steric repulsion alone.",
    "Ion pairing between aryl sulfonate anions and tetrabutylammonium cations blocks the ortho and meta sites, so iridium-catalyzed C–H borylation proceeds with high para selectivity.",
    "Suzuki–Miyaura coupling of aryl boronic esters with aryl halides under palladium catalysis.",
]

results = reranker.rerank(query, documents, context=context)
for result in results:
    print(f"Score: {result['relevance_score']:.4f}  {result['document'][:80]}...")

Returns: List of dicts sorted by relevance descending — document, relevance_score in (0, 1), and index.

reranker.rerank(
    query: str,
    documents: List[str],
    context: Optional[str] = None,   # leave out to rank on the query alone
    top_n: Optional[int] = None,
)

The score is sigmoid(logit('yes') - logit('no')) at the end of the prompt in chat_template.jinja, the same prompt and score that SGLang's /v1/rerank and vLLM's /rerank produce for AsteroRank-Lepton.

Custom Model Path

reranker = MLXReranker("/path/to/AsteroRank-Lepton-MLX-DWQ-4bit")

License

CC BY-NC 4.0, like AsteroRank-Lepton, which is fine-tuned from Qwen3.5-4B (Apache 2.0); see NOTICE.

Contact

For any inquiry about AsteroRank-Lepton contact us via [email protected].