Track 07 · Data & governanceLesson 02 of 11
Is the data ready?
Warehouses, lakehouses and joins: clean data matters more than a better model.
- Specialisation · Data & AI platform
- 35 min read
- 60 min hands-on
- Pro
What you'll be able to do
01- Explain what "data readiness" means for an agent, as seven measurable dimensions: availability, access, freshness, quality, joinability, semantics and documentation.
- Choose where an agent should read from (operational store, warehouse, lakehouse) and defend the choice on freshness, load, governance and cost.
- Design an agent-ready curated view and a semantic-layer definition that encode grain, keys, business meaning and access rules, so the model never has to guess them.
- Implement a readiness scorecard that measures null rates, freshness, key uniqueness and join coverage and returns pass or fail per dimension.
- Decide for each failing dimension whether to fix upstream, fix in the curated view, label the caveat in answers, or take the source out of scope.
- Control the compute an agent consumes in a warehouse or lakehouse: dedicated compute, timeouts, quotas, attribution and budgets.
Before you start
- Finding where the data really lives: the source map is the input to this concept. Readiness is scored per row of that map.
- Golden datasets and the metrics that matter: the representative questions that define what "ready" has to cover.
- Authorization and multi-tenancy: the policy model that row filters and secure views implement.
- Keeping the knowledge base fresh: the document-side version of the freshness problem.
- Working SQL: joins,
GROUP BY, window functions, and reading aCREATE VIEWstatement.
The short version
Data readiness is the question "can this data, as it exists today, support the answers we are promising?" asked per source and per question, and answered with numbers. For an agent it breaks into seven dimensions: is the data there (availability), can the agent legally and technically reach it (access), is it recent enough (freshness), is it complete and valid (quality), does it join to the other sources it needs (joinability), does everyone agree what it means (semantics), and is that meaning written down where a model can read it (documentation). Warehouses and lakehouses are where you usually fix what fails: you build curated views at a declared grain, encode the definitions in a semantic layer, and give the agent its own governed, metered compute. You never point the agent at raw or operational tables and hope the prompt makes up for it.
- Readiness is relative to a question set. Data can be ready for "how many deposits mature this month?" and not ready for "which of my clients have one?".
- Score each dimension pass or fail. Do not average. A source with perfect quality and no access approval is not 86% ready; it is blocked.
- Fix data in data, not in prompts. Decoding, deduplication, conflict rules and freshness stamps belong in the curated view or upstream, where they are tested once.
- The semantic layer is where business meaning becomes executable. A metric defined once and signed by its owner beats a paragraph in a system prompt.
- Agent compute is a workload like any other. It needs its own warehouse, a statement timeout, a quota and a cost line.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI