Embeddings · evidence reviewed October 7, 2026
MTEB evidence for choosing embeddings.
Use embedding benchmarks to choose candidates for a corpus-specific test. This page separates a historical publisher comparison from a newly verified model release. It does not claim to be an exhaustive October leaderboard.
Current candidate · primary source checked 2026-10-07
EmbeddingGemma 2
Google lists the model card as updated October 6, 2026. The card describes 740M total parameters, with a selectively loadable 270M text component, native 768-dimensional embeddings and an 8,192-token context. It names Apache 2.0 as the license. These are publisher specifications. Google model card; dated model-card index.
| Publisher benchmark identity | Metric / configuration | Reported score |
|---|---|---|
| MTEB, multilingual, v2 | Mean(Task), mixed task metrics, 768d full precision | 61.36 |
| MTEB, English, v2 | Mean(Task), 768d full precision | 68.46 |
| MTEB, code, v1 | Mean(Task), NDCG@10, 768d full precision | 78.68 |
No comparable score for this model exists in the historical table below. These named v2/v1 results remain separate: they do not establish a rank against scores whose suite revision is unspecified. This page has no CodeSOTA corpus-specific run for EmbeddingGemma 2.
The official usage instructions specify task prefixes and normalization after dimension truncation. Validate the exact checkpoint, precision and document/query formatting used by your serving stack.
Historical source comparison · November 2025 (publisher description)
One publisher table, ten models.
Source identity: KaLM-Embedding-Gemma3-12B-2511 model-card MMTEB table. The row order is the publisher’s Borda rank; it is not descending Mean(Task). Mean(Task) and Mean(TaskType) are different aggregates. Pinned source revision, checked 2026-10-07.
| Source Borda rank | Model | Mean(Task) | Mean(TaskType) | Retrieval | STS |
|---|---|---|---|---|---|
| 1 | KaLM-Embedding-Gemma3-12B-2511 | 72.32 | 62.51 | 75.66 | 79.02 |
| 2 | llama-embed-nemotron-8b | 69.46 | 61.09 | 68.69 | 79.41 |
| 3 | Qwen3-Embedding-8B | 70.58 | 61.69 | 70.88 | 81.08 |
| 4 | gemini-embedding-001 | 68.37 | 59.59 | 67.71 | 79.40 |
| 5 | Qwen3-Embedding-4B | 69.45 | 60.86 | 69.60 | 80.86 |
| 6 | Qwen3-Embedding-0.6B | 64.34 | 56.01 | 64.65 | 76.17 |
| 7 | gte-Qwen2-7B-instruct | 62.51 | 55.93 | 60.08 | 73.98 |
| 8 | Linq-Embed-Mistral | 61.47 | 54.14 | 58.69 | 74.86 |
| 9 | multilingual-e5-large-instruct | 63.22 | 55.08 | 57.12 | 76.81 |
| 10 | embeddinggemma-300m | 61.15 | 54.31 | 62.49 | 74.73 |
The earlier page merged additional model scores from unidentified snapshots and mislabeled some task columns. Those rows have been withdrawn from this comparison. This table preserves the identities and values found together in the cited publisher report. Dataset-level uncertainty and independent reproduction remain unavailable here.
Choose for your corpus
A benchmark score starts an evaluation.
- Choose an exact MTEB suite from the official evaluator. Record its version, task revisions, split and aggregation method.
- Hold out queries and relevance labels from your own documents. Keep language mix, chunking, candidate corpus, prompts and retrieval settings fixed across models.
- Compare retrieval quality on that fixed corpus, then measure latency, memory and serving cost on the intended hardware. Do not divide an aggregate benchmark score by parameters to invent an efficiency ranking.
- Check the selected checkpoint’s code and weight licenses, context limit, query/document instructions and normalization. Model families can contain different releases with different terms.
- Evaluate a reranker separately on the same retrieved candidates. Embedding similarity scores and cross-encoder reranker results are different tasks.
The original MTEB paper and the later MMTEB paper describe different benchmark coverage. A language-filtered list of tasks from a current package is not a frozen recreation of either paper’s suite.
Reranking guide · Evidence methodology · Official leaderboard and newer submissions