FlowtandBriefAI · Software · ResearchMeet Flowtand

Ettin Reranker Family: Six Cross-Encoders From 17M to 1B Parameters

Announcement

Hugging Face

Browse Models topic page →

Tom Aarsen announced six Sentence Transformers CrossEncoder rerankers built on Ettin ModernBERT encoders, trained by distilling scores from a larger reranker. The publisher reports results on MTEB(eng, v2) Retrieval paired with six embedding models, a speed comparison against default loading, and an Apache 2.0 license.

What was released and how it works

Tom Aarsen published six Sentence Transformers CrossEncoder rerankers, from cross-encoder/ettin-reranker-17m-v1 up to cross-encoder/ettin-reranker-1b-v1. Each scores a (query, document) pair jointly, which the author describes as more accurate but more expensive than embedding similarity, so the documented pattern is retrieve-then-rerank: a fast embedder returns top-K candidates and the cross-encoder re-orders only those. The backbones are Ettin ModernBERT encoders with up to 8192 tokens of context; the head adds CLS pooling, a dense GELU layer, LayerNorm and a single score. Training used distillation with pointwise MSE against mixedbread-ai/mxbai-rerank-large-v2 scores over a named dataset, and the data and recipe are released. The publisher reports MTEB(eng, v2) Retrieval results, where its 1B model reaches 0.6114 mean NDCG@10 against the teacher's 0.6115, and says optimized loading with bfloat16 and Flash Attention 2 is 1.7x-8.3x faster than fp32 plus SDPA. Models are stated as Apache 2.0.

A scenario for a search relevance engineer

Consider a search relevance engineer whose product search currently uses an embedding model to return the top 100 documents and shows them unranked by query-document fit. A team could try inserting a mid-size Ettin reranker, such as the 68m or 150m variant, over those candidates. The premise is supported by the post: the reranker is a standard CrossEncoder with a rank method, so the engineer can re-order whatever the existing retriever returns without replacing the embedder. The open question is size: the author's table shows larger variants scoring higher on average across paired embedders, so a team with tight latency budgets could start small and test upward, rather than assuming the largest model is needed.

How to trial it against a baseline

A team could run a controlled experiment: record current top-100 results for a fixed sample of real queries, then rerank those same candidates with one or two sizes and compare ranking quality against the recorded baseline using the same metric throughout. Because the post presents speedups only for specific loading settings, the team should measure its own latency for each size, including the recommended bfloat16 and Flash Attention 2 configuration and the default, and check the hardware actually used. Failure checks: confirm that the reranker does not lower quality on average, as the author warns a reranker below the retriever-only line actively hurts; inspect queries where rankings shifted unexpectedly for plausible errors; and keep a human reviewer sampling reordered results before any production change.

Trade-offs and when not to use it

Reranking multiplies the cost of scoring: the post states the model runs once per query-document pair rather than once per text, so a 100-candidate funnel means 100 forward passes per query unless a smaller size is chosen. Prerequisites include a retriever already producing reasonable candidates, hardware supporting the recommended kernels, and the willingness to hold the reranker and embedder together. Unknowns remain: this brief rests on a short excerpt, so full benchmark tables, training arguments and speed details were not reviewed, and the reported figures are the author's, not independent tests. A team should not rely on the quality claim for other languages or domains without its own evaluation, and should not use a large variant where latency or throughput budgets cannot absorb per-pair scoring.

Context and limitations

  • Based on a short excerpt of the publisher's blog post; the full article, complete benchmark tables and training details were not analyzed.
  • All quality and speed figures are the publisher's own measurements; no independent evaluation is established.
  • Benchmark results are reported for MTEB(eng, v2) Retrieval pairings, which may not transfer to other domains or languages.
  • Based on bounded source text, not independent testing. Practical suggestions are editorial proposals.