Feasibility and data access, fast

• You can design a time-boxed feasibility spike on 20–50 real cases, with kill criteria written and signed before anyone sees results.
• You can separate technical, data and organisational feasibility and name the evidence that settles each one.
• You can run a six-point data-readiness check (exists, reachable, permitted, representative, labelled, fresh) and attach an owner and evidence to every line.
• You can design an access-scoped data path (least-privilege extract, sandbox copy, masked sample) and recognise when a data-sharing agreement, DPIA or BAA is triggered.
• You can treat access lead times as schedule risk and keep learning while you wait, using Wizard of Oz runs and manual baselines.
• You can record a go / reframe / no-go decision with evidence and defend it to a sponsor who wanted a yes.

Think of the spike as a pre-flight check, not a test flight. A test flight (the build plus the golden-set evaluation) costs weeks and proves the aircraft works. A pre-flight check costs days and finds the reasons it must not take off yet: no fuel, a door that will not close, nobody licensed to fly it. Most of what a spike finds is not about the model. It is about the data being absent, unlabelled, stale or off-limits, and about the organisation lacking an owner or an approval path.

flowchart TD
    Q["Promise to test<br/>(the wedge from qualification)"] --> K["Kill criteria written and signed<br/>BEFORE anyone sees data"]
    Q --> A["Access requests filed on day 1<br/>owners, dates, plan B"]
    A --> S["Access-scoped sample<br/>20-50 real cases"]
    A -->|"access is late"| W["Wizard of Oz run or<br/>manual baseline meanwhile"]
    W -->|"produces labelled cases"| S
    K --> R["Run the simplest credible solution<br/>+ readiness checklist"]
    S --> R
    R --> D{"Score against<br/>kill criteria"}
    D -->|"all criteria met"| G["GO<br/>conditions with owners and dates"]
    D -->|"a slice or a readiness item fails"| F["REFRAME<br/>narrow scope or lower autonomy,<br/>re-test, same criteria"]
    D -->|"hard stop, or even the optimistic<br/>bound misses the target"| N["NO-GO<br/>record why, park with a trigger"]

Kill criteria and access requests both start at the top, in parallel, before any data arrives. Reframing re-tests with the same criteria: you may change the scope, never the thresholds.

What a feasibility spike is, and what it is not

A feasibility spike is a time-boxed experiment that answers one question: can the data and the organisation support the specific promise we are about to make? It has a fixed duration (roughly three to ten working days is a useful heuristic), a fixed sample, pre-agreed criteria and a written decision at the end.

It is not a prototype, which is built to be shown and gets polished. It is not a vendor proof of concept, whose goal is to impress. A spike exists to find out early, cheaply and in writing whether you should stop. If it never could have produced a no, it was not a spike. The code is disposable; the sample definition, kill criteria, readiness table and access register carry straight into the golden set (Golden datasets and the metrics that matter), the baseline (Baselines and ROI you can defend) and the slice definition (Scoping a shippable slice).

Three kinds of feasibility

Teams usually test only the first of these, then get surprised by the other two.

Organisational feasibility is the one engineers skip, and the one that most often shelves a technical success. A drafting agent that needs an approval on every case is not feasible if the approving team is at capacity and nobody funds the time. Ask early: who approves, for how many minutes per case, and who pays for those minutes? The answer feeds the cost side of the ROI model in Baselines and ROI you can defend.

Designing the spike

Write the promise as a testable sentence. "An agent can classify commercial-invoice-mismatch exceptions on the Santos–Rotterdam and Jakarta–Jebel Ali lanes, from the documents in the masked Snowflake view, accurately enough to draft resolutions for operator approval." That sentence names the task, the scope, the data source and the autonomy level. Each of those is a lever you can pull if you need to reframe.

Use 20–50 real cases. Real means drawn from production, from the in-scope period and population, through the access path you will actually have. Fifty is enough to kill and not enough to certify. The table shows the 95% Wilson interval when the observed accuracy is 90%:

With 50 cases at 90% observed, the true rate could plausibly be 79%, so you cannot claim a 90% target. But if the whole interval sits below the target, even the optimistic reading fails and you should stop or reframe. That asymmetry is why spikes are good at killing ideas. Use the Wilson interval, not the textbook Wald interval; Golden datasets and the metrics that matter explains why and owns sample-size planning.

Make the sample reproducible. Draw it with a deterministic rule, such as a hash of the case ID with a fixed salt, so a reviewer can re-draw the same cases. A simple random sample keeps the production mix; a stratified sample guarantees slice counts but needs reweighting for an overall estimate. For a spike, simple random plus a slice-count check is usually enough. If a slice that matters has fewer than about five cases, top it up deliberately and say so.

Know where ground truth comes from. In operations data the label is usually a resolution code an operator picked, or a downstream outcome (customs released the shipment, the claim was paid). Both are noisy. Have the ops lead and one operator independently label ten cases. If they disagree on three, the labels set your accuracy ceiling, and that is itself a finding (see Human review and feedback loops).

Run the simplest credible solution. One prompt with a clear output schema, or even a keyword baseline. You are testing the data and the task, not your architecture. If the simplest version fails because the inputs are illegible or incomplete, a sophisticated agent will not save it.

Write the kill criteria first. Each criterion has a metric, a threshold, the action if it fails, and the date and names of the people who agreed it. A useful structure has three families:

• K1, overall quality. If the 95% upper bound is below the production target, no-go for this framing. If the lower bound is below a "viable with iteration" floor, the result is inconclusive: reframe or extend.
• K2, slices. Every slice above a volume threshold (say 5%) needs at least five cases and a minimum point accuracy. A failing slice is carved out or routed to humans, not averaged away.
• K3, readiness. Some readiness failures are hard stops (the data does not exist, or using it is not permitted). Others force a reframe. Partial results become conditions with owners.

Once agreed, the criteria are frozen. If the team wants to move a threshold after seeing results, that is a new spike with new criteria, recorded as such.

The data-readiness checklist

Is the data ready? owns readiness at platform depth: modelling, pipelines, quality tests, lakehouse table formats. At spike scale you need a fast, use-case-specific version that fits on one page. Six questions, each answered with evidence and an owner:

Permitted is broader than "we have a login": it covers purpose, location (residency, see Data residency and sovereignty), processor chain (is the model provider an approved sub-processor?) and contract. Sending case text to a model endpoint is a new processing activity even if you may already read it. Fresh is measured at serving time: a nightly extract loaded at 02:00 is up to 26 hours old by late evening the next day, and that is the number that matters.

Access-scoped data

Access-scoped data means the narrowest data you can work with that still answers the question: the fewest rows, the fewest columns, the shortest window, the least identifiable form, for the shortest time, with a named owner who can revoke it. It is least privilege applied to data access during discovery, and it is what makes customers willing to say yes quickly.

The options, from least to most exposure (direct production access is rarely justified for a spike):

The mechanics live in the customer's platform. In Snowflake, row access policies and masking policies are evaluated at query time (Snowflake docs). In Databricks Unity Catalog, row filters and column masks do the same job, with ABAC policies recommended for applying them consistently across many tables (as of September 2026, Databricks docs). Finding and masking personal data owns how masking and tokenisation are designed. Here you only need to ask for the right scope and check that the mask has not destroyed the signal.

Agreements and assessments that may be triggered. You do not decide these. You give the customer's privacy, legal and security owners an accurate description of the processing early, so their clock starts.

• Data-sharing or data-processing agreements. If data moves from the customer to you, or to a model provider acting for the customer, the contract must cover it. In US healthcare, a vendor handling PHI on a covered entity's behalf works under a business associate agreement (BAA). HHS guidance notes that a data use agreement for a limited data set can be combined with a BAA in a single agreement (HHS FAQ).
• DPIA under GDPR. Article 35 requires a data protection impact assessment where processing is likely to result in high risk. Article 35(3) names three cases that always need one, and the WP29 guidelines endorsed by the EDPB list nine indicators of high risk, including innovative technology, matching or combining datasets, and automated decisions with significant effects. The ICO's guidance notes that in most cases two indicators together point to a DPIA, though that is not a strict rule (ICO). An AI agent that combines email and ERP data about identifiable people will often meet two. Expect a screening at least.
• HIPAA minimum necessary. Covered entities must take reasonable steps to limit uses, disclosures and requests of PHI to the minimum necessary for the purpose (HHS). Your spike request should list fields and justify each.

Access lead times are schedule risk

Access is usually slower than building. A service account, a privacy screening, a security review of the model endpoint, a data-sharing agreement and a network allow-list each have a queue, an owner and a lead time you do not control. If the go-live date was set without them, it is already at risk. Manage access as a critical path:

1. File every request on day one, in parallel, even the ones you might not need.
2. Keep an access register: item, owner, requested, promised, granted, plan B. Review it in every status meeting.
3. Ask for historical lead times, not hopes. "How long did the last service account take?" beats "how long will this take?"
4. Have a plan B per item: a nightly extract instead of an API, a masked export instead of a live mailbox, a sandbox in their tenant instead of data leaving it.
5. Use the agreed channel. If an SI owns delivery, access requests usually go through their change process (Delivering through an SI). Going around it saves a day and costs the relationship.

When the data is not ready: Wizard of Oz and manual baselines

A late or missing data source does not have to stop learning. Two techniques keep the spike productive.

A Wizard of Oz run puts a human behind the interface the agent will eventually use. An operator writes the "agent draft" in the target format, through the new screen or queue, and the approving operator works with it as if it were generated. You learn whether the format helps, how long approval takes and what reviewers edit, before the model or the data feed exists, and every case becomes a labelled example. Two rules: participants know a colleague is producing the drafts, and Wizard of Oz output is never shown to external customers as AI-generated.

A manual baseline has operators do the current task with a light structured form: start time, end time, exception type, fields consulted, outcome. One effort yields the baseline Baselines and ROI you can defend needs and the labels the spike needs. Google's ML engineering guidance makes the same point: launching without ML is fine, and you should instrument the current system before building the learned one (Rules of ML).

Both techniques test organisational feasibility directly. If nobody has twenty minutes a day to approve Wizard of Oz drafts, nobody will have them for agent drafts either.

Recording the decision with evidence

The output of the spike is a short decision record, filed next to the qualification scorecard, that lets someone who was not in the room see what was promised, tested, found and decided. It holds the promise as a testable sentence; the kill criteria with date and signatories; the sample (source, window, size, how to re-draw it); results with intervals, overall and per slice; the readiness table with owners; the decision; conditions with owners and due dates; the trigger that would reopen the decision; and sign-offs from the process owner and data owner. The worked example below ends with one.

"Go with conditions" is still a go. A condition without an owner and a date is not a condition, it is a hope.

Halden Freight's wedge from Picking the wedge use case is commercial-invoice-mismatch exceptions on two lanes. The sponsor wants a board demo in four weeks. Before you commit to the acceptance target in the week brief (90% classification accuracy on a 100-case golden set), you run a five-day spike.

Day 1. You file access requests in parallel through the SI's change process: a masked read-only Snowflake view (owner: SI data lead), the SAP OData service account (SI basis team, already flagged for week 6), a DPIA screening (Halden's DPO) and approval of an EU-region model endpoint (the CISO, pending the one-page threat model). You agree kill criteria with Rafaela (the ops lead) and the SI account lead, and record them with the date.

Day 2. The view is live. The data-platform owner, not you, creates it, so the scope is theirs to approve and revoke.

-- Snowflake. Created and owned by Halden's data-platform team. Names are illustrative.
-- Assumes the platform team already has a redaction masking policy on EXCEPTION_DOCS.extracted_text.
CREATE ROLE IF NOT EXISTS FDE_SPIKE_RO;

CREATE OR REPLACE SECURE VIEW OPS_SANDBOX.SPIKE.EXCEPTIONS_2LANE_V AS
SELECT
    e.exception_id,
    e.lane_code,
    e.doc_language,
    e.doc_quality,
    e.resolution_code,
    e.created_at,
    d.extracted_text          -- base-table masking policy still applies through the view
FROM OPS.CORE.EXCEPTIONS AS e
JOIN OPS.CORE.EXCEPTION_DOCS AS d
  ON d.exception_id = e.exception_id
WHERE e.lane_code IN ('SSZ-RTM', 'JKT-JEA')
  AND e.created_at >= DATEADD(day, -90, CURRENT_DATE());

GRANT USAGE ON DATABASE OPS_SANDBOX TO ROLE FDE_SPIKE_RO;
GRANT USAGE ON SCHEMA OPS_SANDBOX.SPIKE TO ROLE FDE_SPIKE_RO;
GRANT SELECT ON VIEW OPS_SANDBOX.SPIKE.EXCEPTIONS_2LANE_V TO ROLE FDE_SPIKE_RO;

-- Reproducible sample: anyone can re-draw the same 50 cases with the same salt.
SELECT *
FROM OPS_SANDBOX.SPIKE.EXCEPTIONS_2LANE_V
ORDER BY HASH(exception_id, 'spike-2026-09-08')
LIMIT 50;

Note what is not in the view: shipper and consignee contact columns, other lanes, anything older than 90 days, and any write privilege. Leaving contact columns out does not remove names inside the document text, so extracted_text needs its own redaction policy. Snowflake applies a column's masking policy wherever the column appears, including through views; check whether the policy tests CURRENT_ROLE() or INVOKER_ROLE() (the view owner's role), because that decides what FDE_SPIKE_RO sees (as of September 2026, Snowflake docs). The production mix by language and document quality comes from the 90-day event log you mined in Shadowing the work as it's really done.

Days 3–4. You run the scorer below. The stand-in classifier is a keyword baseline so the example runs offline; in the real spike you swap stub_classifier for your model call and keep everything else. The synthetic sample deliberately mimics what you found in shadowing: older Bahasa Indonesia scans from Jakarta are much worse than everything else.

Requirements: Python 3.11+, standard library only. Runs offline with no keys.

"""Feasibility-spike scorer (offline, standard library only).

Runs a stand-in classifier over a small labelled sample, computes accuracy with a
Wilson interval and per-slice coverage, checks a data-readiness checklist, and
returns GO / REFRAME / NO-GO against kill criteria agreed *before* the run.
"""
from __future__ import annotations

import math
import random
from collections import defaultdict
from dataclasses import dataclass
from typing import Callable, Literal

Status = Literal["pass", "partial", "fail"]
Decision = Literal["GO", "REFRAME", "NO-GO"]
SEVERITY: dict[Decision, int] = {"GO": 0, "REFRAME": 1, "NO-GO": 2}
Cell = tuple[str, str]  # (language, quality)


@dataclass(frozen=True)
class Case:
    case_id: str
    language: str       # "pt" | "id" | "en"
    quality: str        # "digital" | "scanned"
    text: str           # masked document excerpt from the access-scoped view
    label: str | None   # ground truth from the operator's resolution code; None if unusable


@dataclass(frozen=True)
class ReadinessItem:
    name: str           # exists | reachable | permitted | representative | labelled | fresh
    status: Status
    evidence: str
    owner: str


@dataclass(frozen=True)
class KillCriteria:
    """Written and signed before the spike runs. Not edited after seeing results."""
    target_accuracy: float = 0.90      # production acceptance target (README AC4)
    viable_lower_bound: float = 0.75   # Wilson lower bound needed to proceed to build
    min_slice_n: int = 5               # fewer labelled cases than this = coverage gap
    min_slice_share: float = 0.05      # slices below this share of volume are reported, not gated
    min_slice_accuracy: float = 0.70   # point-accuracy floor for any gated slice
    max_mix_gap: float = 0.15          # |sample share - production share| per slice value
    min_labelled_share: float = 0.90
    max_data_age_hours: float = 24.0
    hard_stops: frozenset[str] = frozenset({"exists", "permitted"})
    agreed_on: str = "2026-09-08"
    agreed_by: str = "Halden ops lead, SI account lead, FDE"


def wilson(successes: int, n: int, z: float = 1.96) -> tuple[float, float]:
    """Wilson score interval for a binomial proportion (95% by default)."""
    if n == 0:
        return (0.0, 1.0)
    p = successes / n
    denom = 1 + z * z / n
    centre = (p + z * z / (2 * n)) / denom
    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / denom
    return (max(0.0, centre - half), min(1.0, centre + half))


# ---- stand-in model: swap for your real classifier call during the spike ----
RULES: dict[str, tuple[str, ...]] = {
    "invoice_mismatch": ("invoice", "fatura", "faktur"),
    "missing_docs": ("certificate", "certificado", "surat"),
    "hs_code": ("tariff", "ncm", "kode hs"),
    "carrier_delay": ("vessel", "navio", "kapal"),
}


def stub_classifier(text: str) -> str:
    lowered = text.lower()
    for label, keys in RULES.items():
        if any(k in lowered for k in keys):
            return label
    return "invoice_mismatch"  # majority-class fallback, as a naive model would do


# ---- synthetic sample standing in for 50 real, masked cases ----
PHRASES: dict[str, dict[str, str]] = {
    "invoice_mismatch": {"en": "invoice value does not match declared value",
                         "pt": "valor da fatura diverge do valor declarado",
                         "id": "nilai faktur tidak sesuai nilai pabean"},
    "missing_docs": {"en": "certificate of origin missing",
                     "pt": "certificado de origem ausente",
                     "id": "surat keterangan asal tidak ada"},
    "hs_code": {"en": "tariff classification query",
                "pt": "classificacao tarifaria ncm incorreta",
                "id": "kode hs salah klasifikasi"},
    "carrier_delay": {"en": "vessel rolled to next sailing",
                      "pt": "navio atrasado para proxima saida",
                      "id": "kapal terlambat jadwal berikutnya"},
}
OCR_CONFUSIONS = {"a": "o", "e": "c", "i": "l", "s": "5", "t": "f", "u": "v"}


def ocr_noise(text: str, rng: random.Random, rate: float) -> str:
    return "".join(OCR_CONFUSIONS.get(ch, ch) if rng.random() < rate else ch for ch in text)


def build_sample(prod: dict[Cell, float], n: int = 50, seed: int = 47) -> list[Case]:
    rng = random.Random(seed)
    cells, weights = list(prod), list(prod.values())
    cases: list[Case] = []
    for i in range(n):
        lang, quality = rng.choices(cells, weights=weights)[0]
        label = rng.choices(list(PHRASES), weights=[50, 20, 15, 15])[0]
        text = PHRASES[label][lang]
        if quality == "scanned":  # older Bahasa scans are the worst in the archive
            text = ocr_noise(text, rng, rate=0.35 if lang == "id" else 0.08)
        recorded: str | None = label
        if rng.random() < 0.06:            # resolution code missing or "OTHER"
            recorded = None
        elif rng.random() < 0.05:          # operator mis-coded the case
            recorded = rng.choice([x for x in PHRASES if x != label])
        cases.append(Case(f"EXC-{i + 1:03d}", lang, quality, text, recorded))
    return cases


def marginal(prod: dict[Cell, float], dims: tuple[str, ...]) -> dict[tuple[str, ...], float]:
    pos = {"language": 0, "quality": 1}
    out: dict[tuple[str, ...], float] = defaultdict(float)
    for cell, share in prod.items():
        out[tuple(cell[pos[d]] for d in dims)] += share
    return dict(out)


def grade(ok: bool, near: bool) -> Status:
    return "pass" if ok else ("partial" if near else "fail")


def score_spike(
    cases: list[Case],
    classify: Callable[[str], str],
    declared: list[ReadinessItem],
    prod: dict[Cell, float],
    data_age_hours: float,
    kc: KillCriteria,
) -> tuple[Decision, list[str], list[str]]:
    findings: list[tuple[Decision, str]] = []
    lines: list[str] = []
    labelled = [c for c in cases if c.label is not None]
    hit = {c.case_id: classify(c.text) == c.label for c in labelled}
    k, n = sum(hit.values()), len(hit)
    lo, hi = wilson(k, n)
    lines.append(f"overall  {k}/{n} = {k / n:.2f}  95% Wilson [{lo:.2f}, {hi:.2f}]")
    if hi < kc.target_accuracy:
        findings.append(("NO-GO", f"K1 upper bound {hi:.2f} < target {kc.target_accuracy:.2f}"))
    elif lo < kc.viable_lower_bound:
        findings.append(("REFRAME", f"K1 lower bound {lo:.2f} < {kc.viable_lower_bound:.2f} (inconclusive)"))

    worst_gap = 0.0
    for dims in (("language",), ("quality",), ("language", "quality")):
        for value, share in sorted(marginal(prod, dims).items()):
            members = [c for c in labelled if tuple(getattr(c, d) for d in dims) == value]
            ks, ns = sum(hit[c.case_id] for c in members), len(members)
            acc = ks / ns if ns else 0.0
            slo, shi = wilson(ks, ns)
            name = "x".join(value)
            lines.append(f"  {'x'.join(dims):<16} {name:<12} n={ns:<3} acc={acc:.2f} "
                         f"[{slo:.2f}, {shi:.2f}]  prod share {share:.2f}")
            if len(dims) == 1:
                sample_share = sum(1 for c in cases if (getattr(c, dims[0]),) == value) / len(cases)
                worst_gap = max(worst_gap, abs(sample_share - share))
            if share < kc.min_slice_share:
                continue
            if ns < kc.min_slice_n:
                findings.append(("REFRAME", f"K2 coverage gap: {name} has n={ns}"))
            elif acc < kc.min_slice_accuracy:
                findings.append(("REFRAME", f"K2 slice {name} acc {acc:.2f} < {kc.min_slice_accuracy:.2f} "
                                            f"({share:.0%} of volume)"))

    labelled_share = n / len(cases)
    computed = [
        ReadinessItem("representative", grade(worst_gap <= kc.max_mix_gap, worst_gap <= 2 * kc.max_mix_gap),
                      f"worst sample-vs-production gap {worst_gap:.2f}", "FDE"),
        ReadinessItem("labelled", grade(labelled_share >= kc.min_labelled_share,
                                        labelled_share >= kc.min_labelled_share - 0.10),
                      f"{labelled_share:.0%} of sampled cases have a usable resolution code", "ops lead"),
        ReadinessItem("fresh", grade(data_age_hours <= kc.max_data_age_hours,
                                     data_age_hours <= 2 * kc.max_data_age_hours),
                      f"worst-case data age {data_age_hours:.0f} h vs {kc.max_data_age_hours:.0f} h needed",
                      "data-platform owner"),
    ]
    conditions: list[str] = []
    for item in declared + computed:
        lines.append(f"  readiness {item.name:<14} {item.status:<7} {item.evidence}")
        if item.status == "fail":
            verdict: Decision = "NO-GO" if item.name in kc.hard_stops else "REFRAME"
            findings.append((verdict, f"K3 '{item.name}' failed: {item.evidence}"))
        elif item.status == "partial":
            conditions.append(f"condition [{item.owner}] {item.name}: {item.evidence}")

    decision: Decision = max((d for d, _ in findings), key=SEVERITY.__getitem__, default="GO")
    return decision, [why for _, why in findings], lines + conditions


def show(title: str, result: tuple[Decision, list[str], list[str]]) -> None:
    decision, reasons, lines = result
    print(f"== {title}: {decision}")
    print("\n".join(lines))
    for why in reasons:
        print(f"  -> {why}")
    print()


if __name__ == "__main__":
    kc = KillCriteria()
    declared = [
        ReadinessItem("exists", "pass", "invoices + resolution codes in Snowflake view", "SI data lead"),
        ReadinessItem("reachable", "partial", "nightly extract only; SAP OData account due week 6",
                      "SI basis team"),
        ReadinessItem("permitted", "pass", "masked read-only view, 2 lanes, 90 days; DPIA screened",
                      "Halden DPO"),
    ]
    # Volume share by (language, document quality) for the two lanes, from 90 days of event log.
    prod: dict[Cell, float] = {
        ("pt", "digital"): 0.30, ("pt", "scanned"): 0.15,
        ("id", "digital"): 0.17, ("id", "scanned"): 0.18,
        ("en", "digital"): 0.14, ("en", "scanned"): 0.06,
    }
    sample = build_sample(prod)
    show("Full scope", score_spike(sample, stub_classifier, declared, prod, data_age_hours=26, kc=kc))

    # Reframe: Bahasa scans route to operators unchanged; re-score the remaining scope
    # with production shares renormalised. Kill criteria are NOT changed.
    carve_out: Cell = ("id", "scanned")
    kept = {cell: s for cell, s in prod.items() if cell != carve_out}
    total = sum(kept.values())
    prod_reframed = {cell: s / total for cell, s in kept.items()}
    in_scope = [c for c in sample if (c.language, c.quality) != carve_out]
    print(f"carve-out {'x'.join(carve_out)} = {prod[carve_out]:.0%} of volume stays manual\n")
    show("Reframed scope", score_spike(in_scope, stub_classifier, declared, prod_reframed,
                                       data_age_hours=26, kc=kc))

What it prints (abridged).

== Full scope: REFRAME
overall  43/47 = 0.91  95% Wilson [0.80, 0.97]
  quality          scanned      n=25  acc=0.84 [0.65, 0.94]  prod share 0.39
  languagexquality enxscanned   n=7   acc=0.71 [0.36, 0.92]  prod share 0.06
  languagexquality idxscanned   n=5   acc=0.60 [0.23, 0.88]  prod share 0.18
  readiness reachable      partial nightly extract only; SAP OData account due week 6
  readiness labelled       pass    94% of sampled cases have a usable resolution code
  readiness fresh          partial worst-case data age 26 h vs 24 h needed
  -> K2 slice idxscanned acc 0.60 < 0.70 (18% of volume)
carve-out idxscanned = 18% of volume stays manual

== Reframed scope: GO
overall  40/42 = 0.95  95% Wilson [0.84, 0.99]
  readiness representative partial worst sample-vs-production gap 0.20
condition [SI basis team] reachable: nightly extract only; SAP OData account due week 6
condition [FDE] representative: worst sample-vs-production gap 0.20
condition [data-platform owner] fresh: worst-case data age 26 h vs 24 h needed

Reading the output.

1. The average hides the problem. Overall accuracy is 0.91 with an interval of [0.80, 0.97], so K1 passes. Scanned Bahasa documents score 0.60 on five cases. Even that slice's optimistic bound (0.88) is below the 0.90 target, and the slice is 18% of volume. K2 fails, so the decision is REFRAME, not GO.
2. The reframe changes scope, not thresholds. Scanned Bahasa documents stay with operators. That costs 18% of the addressable volume, which you carry into the ROI model in Baselines and ROI you can defend. The OCR problem goes on the backlog with a pointer to Scans, forms and OCR at scale.
3. GO comes with three conditions, each with an owner. The SAP account is late, so the slice runs on the nightly extract. Data can be 26 hours old, so the agent must show an "as of" time and the in/out list must exclude exception states that change intraday. The reframed sample is now heavy on scanned documents compared with the new production mix, so the 100-case golden set for the build must be re-drawn against the reframed mix.
4. English scans are a watch item. At 5 of 7 they clear the 0.70 floor; one more miss would fail them. Note it in the record.
5. The reframed result is not independent. You carved out the failing slice after seeing the same data. That is legitimate for deciding scope, but it is not proof. The proof is the held-out golden set in the build.

Day 5. You file the decision record:

# Feasibility decision record FDR-004 (filed next to the qualification scorecard)
promise: >
  Classify commercial-invoice-mismatch exceptions on SSZ-RTM and JKT-JEA from the
  masked Snowflake view, accurately enough to draft resolutions for operator approval.
kill_criteria:
  {
    agreed_on: 2026-09-08,
    agreed_by: [ops lead, SI account lead, FDE],
    ref: KC-v1,
  }
sample:
  {
    source: OPS_SANDBOX.SPIKE.EXCEPTIONS_2LANE_V,
    window_days: 90,
    n: 50,
    draw: "ORDER BY HASH(exception_id, 'spike-2026-09-08') LIMIT 50",
  }
result_full_scope:
  { decision: REFRAME, reason: "id x scanned acc 0.60 (n=5), 18% of volume" }
decision: GO_WITH_CONDITIONS
scope_change: "Scanned Bahasa Indonesia documents route to operators unchanged (18% of volume)"
conditions:
  - {
      item: reachable,
      owner: SI basis team,
      due: week 6,
      plan_b: nightly extract,
    }
  - {
      item: fresh,
      owner: data-platform owner,
      action: "show data as-of time; exclude intraday states",
    }
  - {
      item: representative,
      owner: FDE,
      action: "re-draw 100-case golden set on reframed mix",
    }
revisit_trigger: "OCR upgrade for Bahasa scans reaches 0.85 on 30 held-out scanned cases"
sign_off: [ops lead, data-platform owner, SI account lead]

Questions to ask on day one: Where is the ground truth recorded? Who owns this data, and who can say yes to this purpose? What is the worst-case age of the data at the moment the agent reads it? How long did your last service account or privacy review take? Who will approve drafts, and for how many minutes per case?

• Builds on: Picking the wedge use case, Shadowing the work as it's really done, Finding where the data really lives, Is the data ready?, Finding and masking personal data
• Used by: Baselines and ROI you can defend (the carved-out volume and the manual baseline), Running a bootcamp on live data (the access-scoped data path), Scoping a shippable slice (conditions become in/out items), Handover, run state and the next use case (sandbox deletion, revisit triggers)
• Related: Golden datasets and the metrics that matter, Data residency and sovereignty, Authorization and multi-tenancy, Delivering through an SI

• NIST/SEMATECH e-Handbook: confidence intervals for proportions: the Wilson interval formulas and why the handbook recommends it over the simple normal approximation.
• ICO: When do we need to do a DPIA?: the clearest walk-through of Article 35 triggers, the nine WP29 indicators and how AI counts as innovative technology.
• EDPB: Guidelines on DPIAs and high-risk processing: the EU-level guidance (WP248, endorsed by the EDPB) that the national lists build on.
• HHS: Minimum necessary requirement: the HIPAA standard your field list for a healthcare spike has to satisfy.
• HHS: De-identification guidance: Safe Harbor and Expert Determination, and what each removes, so you can judge whether the task survives.
• Snowflake: Understanding row access policies: how query-time row filtering works when you ask for a least-privilege view.
• Databricks: Row filters and column masks: the Unity Catalog equivalent, and when ABAC policies or dynamic views fit better.
• Google: Rules of Machine Learning: Rules 1 and 2 make the case for launching without ML and instrumenting the current system first.