Track 02 · RetrievalLesson 02 of 11
Picking embeddings on your corpus, not a leaderboard
Embeddings cap retrieval quality; measure them on the customer's own queries.
- Specialisation · Data & AI platform
- 40 min read
- 60 min hands-on
- Pro
What you'll be able to do
01- Explain what a text embedding encodes, what it does not, and why the first-stage retriever caps everything downstream.
- Choose a similarity metric, dimension and precision for a given model and corpus, and explain the memory and quality consequences.
- Implement recall@k, MRR and nDCG@k and use them to compare candidate models on the customer's own golden set, with confidence intervals and per-language slices.
- Diagnose silent embedding bugs: missing query/document input types, unnormalised vectors, truncation, mixed model versions.
- Defend a model choice to an architecture review in terms of quality on the customer's data, languages, cost, latency, residency and the cost of switching later.
Before you start
- Parsing and chunking documents that fight back: what gets embedded is a chunk, and chunk quality comes first.
- Choosing a model you can defend: the same selection discipline applies to embedding models.
- Beyond English: language, script and region: script coverage and tokenisation cost by language.
- Dot products and vector norms at the level of "I could compute one by hand".
The short version
An embedding model maps each chunk and each query to a vector so that relevant pairs land close together. Everything later in the pipeline (the vector index, hybrid fusion, the reranker, the LLM) can only work with the candidates this first stage returns, so a weak or mismatched model sets a ceiling you cannot tune past. Public leaderboards help you shortlist, but the decision has to be made on a golden set built from the customer's corpus, sliced by language and query type, with the model's cost, latency, residency and switching cost on the same page.
- Use the query/document input types or prompts the model was trained with. Getting this wrong degrades quality silently.
- Normalise consistently and use the metric the model was trained for.
- Dimensions and precision drive memory and cost linearly; Matryoshka truncation and int8 or binary quantisation trade a measured amount of quality for large savings.
- MTEB is a shortlist tool, not a decision. Your own recall@k, MRR and nDCG@k are the decision.
- Switching models means re-embedding everything. Plan for it on day one.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI