Case Study: Deploying an LLM Feature at an Enterprise Customer in 6 Days as an FDE

The Monday Morning Call

Monday, 9:07 AM. A VP at a top-10 US bank is on the line. Their compliance team spends 14 hours a week manually cross-referencing internal policy documents against new regulatory filings. They bought an LLM platform license six months ago. It's still not in production.

The internal ML team estimated a 12-week timeline: data prep, fine-tuning, security review, UI. The VP is frustrated. "Can you get something working by Friday?"

This is the FDE sweet spot. Not building a perfect system. Shipping a useful one in 6 days that unblocks the relationship and proves the platform's value.

Architecture on a Whiteboard

We had three constraints:

1. Air-gapped environment. No external API calls. The model runs on-prem on the customer's GPU cluster.
2. Sensitive data. Policy docs contain PII that must be redacted before hitting the model.
3. Non-technical users. Compliance officers aren't going to write prompts. They need a simple search box.

The architecture we sketched:

No fine-tuning. No vector database they hadn't already approved. Just retrieval-augmented generation (RAG) with a redaction layer bolted on.

Day 1-2: Data Ingest and the Chunking Decision

The customer had ~8,000 policy documents spread across SharePoint and a legacy Documentum system. Formats: PDF, DOCX, and a few scanned TIFFs from the 2000s.

The first real decision: chunking strategy.

We went with section-based chunking using Python's python-docx and pymupdf to extract headings, then fell back to recursive character splitting for the scanned TIFFs. Each chunk carried metadata: source document ID, section title, page number, and a hash for deduplication.

def chunk_by_section(doc_text: str, max_chunk_tokens: int = 512) -> list[dict]:
    sections = re.split(r'(?=\n#{1,3}\s)', doc_text)
    chunks = []
    for sec in sections:
        heading = sec.split('\n')[0][:100]
        body = sec[len(heading):].strip()
        if len(tokenizer.encode(body)) > max_chunk_tokens:
            # fallback to recursive split
            subs = recursive_split(body, max_chunk_tokens)
            for i, sub in enumerate(subs):
                chunks.append({"text": sub, "heading": f"{heading} (pt {i+1})"})
        else:
            chunks.append({"text": body, "heading": heading})
    return chunks

By Tuesday evening, we had 42,000 chunks embedded with intfloat/e5-large-v2 and loaded into Milvus. The embedding model was already approved in their environment—zero procurement friction.

Day 3-4: RAG Pipeline and PII Redaction

The retrieval pipeline:

1. User query → embed with same model → cosine similarity search in Milvus → top-5 chunks.
2. Chunks pass through a PII redaction layer before hitting the LLM context window.

The PII problem: Policy documents reference real people. Names, employee IDs, internal email addresses. The compliance team cannot have those appear in LLM responses, even in a closed environment. Hallucinations plus PII equals a security incident.

We built a two-pass redaction system:

• Pass 1 (regex): Employee IDs (EMP-\d{6}), email addresses, SSN patterns.
• Pass 2 (spaCy NER): PERSON, ORG, and GPE entities. Replaced with type tokens like [PERSON_1].

def redact_pii(text: str) -> tuple[str, dict]:
    mapping = {}
    # Pass 1: regex patterns
    text = re.sub(r'EMP-\d{6}', lambda m: _replace(m.group(), 'EMPID', mapping), text)
    text = re.sub(r'[\w.-]+@[\w.-]+', lambda m: _replace(m.group(), 'EMAIL', mapping), text)
    # Pass 2: NER
    doc = nlp(text)
    for ent in reversed(doc.ents):  # reverse to preserve spans
        if ent.label_ in ('PERSON', 'ORG', 'GPE'):
            text = text[:ent.start_char] + _replace(ent.text, ent.label_, mapping) + text[ent.end_char:]
    return text, mapping

The mapping dict let us reconstruct responses if needed, but we never exposed it to the LLM. The model only saw [PERSON_1] tokens.

Thursday afternoon: we ran the first end-to-end test. A compliance officer typed "What's the policy on third-party vendor risk assessments?" and got a coherent, sourced answer in 4.2 seconds. The room went quiet. Then the VP said, "That's faster than our internal wiki."

Day 5: The UX That Almost Killed the Project

Friday morning. The pipeline works. But the UI is a Streamlit prototype that looks like a grad school project. The VP hesitates. "I can't show this to my team."

This is where FDEs earn their comp. You don't argue about MVP aesthetics. You fix it.

We had 8 hours. The customer's front-end team was booked for months. So we:

1. Forked their internal React component library (they had one, mercifully).
2. Built a 3-component UI: search bar, result card with source citations, and a feedback thumbs-up/down.
3. Added a "Show sources" toggle that expanded inline citations back to the original document section.

// The core component: 80 lines of TSX
const PolicySearch = () => {
  const [query, setQuery] = useState('');
  const [result, setResult] = useState<SearchResult | null>(null);
  const [showSources, setShowSources] = useState(false);
  
  const handleSearch = async () => {
    const res = await fetch('/api/search', {
      method: 'POST',
      body: JSON.stringify({ query, user_id: currentUser.id }),
    });
    setResult(await res.json());
  };
  
  return (
    <CorporateShell>
      <SearchBar value={query} onChange={setQuery} onSubmit={handleSearch} />
      {result && (
        <ResultCard
          answer={result.answer}
          sources={result.sources}
          showSources={showSources}
          onToggleSources={() => setShowSources(!showSources)}
          onFeedback={(type) => logFeedback(result.id, type)}
        />
      )}
    </CorporateShell>
  );
};

We deployed it to their internal Kubernetes cluster by 5 PM. The VP demoed it to her team at 5:30. Eight compliance officers used it for real work over the weekend.

Day 6: Shipping, Monitoring, and the Handoff

Monday morning (yes, we worked Saturday too—FDE life). We shipped three final pieces:

1. Monitoring dashboard. Latency percentiles (p50, p95, p99), chunk retrieval count, PII redaction hit rate, and user feedback ratio. Built with Grafana pointing at Prometheus metrics we instrumented in the FastAPI backend.

2. Feedback loop. Every thumbs-down triggered a Slack alert to the compliance team lead with the query, response, and retrieved chunks. This is how the system improves without an ML ops team.

3. Handoff doc. Not a 40-page wiki. A 3-page runbook: architecture diagram, environment variables, common failure modes, and who to call. We used the same principles from Writing Customer-Facing Technical Docs That Actually Get Read.

By noon Monday, we handed over the repo, the runbook, and a 15-minute walkthrough to their internal platform team. The feature stayed in production. Six months later, it handles 200+ queries a day.

Why Speed Matters: The FDE Comp Context

A 6-day deployment isn't just about technical skill. It's about the economic reality of enterprise AI sales.

Most LLM platform deals are $200K-$2M ACV. But they sit in "pilot purgatory" for 6-12 months. The vendor doesn't recognize revenue until the customer hits a usage threshold. The FDE is the accelerant.

When you ship in 6 days what the internal team estimated at 12 weeks, you compress the time-to-value by 14x. That directly impacts:

• Revenue recognition: The deal moves from "booked" to "recognized" faster.
• Expansion revenue: A working feature creates pull for more use cases.
• Renewal probability: Customers who see value in week 1 don't churn in month 12.

This is why FDE compensation bands scale the way they do. Senior FDEs at top firms clear $250K-$400K total comp because they're not just engineers—they're revenue accelerators who happen to write code.

For a deeper look at how FDEs operate across a full engagement cycle, see What a Forward Deployed Engineer Actually Does in a Week and How Palantir-Style FDEs Embed with Customers.

FAQ

Q: Why not fine-tune the model on the policy documents? A: Fine-tuning takes days of GPU time, requires curated training data, and creates a model drift problem every time policies update. RAG with good chunking gives fresh results instantly. For this use case—factual lookups against a known corpus—RAG beats fine-tuning on accuracy and maintainability.

Q: What if the air-gapped environment didn't have Milvus approved? A: We'd fall back to PostgreSQL with pgvector. Slower at scale, but fine for 42K chunks. The architecture pattern doesn't change—only the vector store implementation. FDEs always carry a compatibility matrix: what's approved, what's not, and what's the closest approved alternative.

Q: How do you handle document updates after deployment? A: We shipped a nightly cron job that hashes all source documents, re-processes changed ones, and upserts chunks into Milvus. Chunks carry source hashes for idempotent updates. The compliance team adds ~15 new docs a week—the cron handles it in under 3 minutes.

Q: Is 6 days realistic for most FDE engagements? A: It depends on the scope and the customer's infrastructure readiness. This engagement had GPUs provisioned and a Kubernetes cluster running before we arrived—that's rare. But the pattern of shipping a narrow, high-value feature in 1-2 weeks is standard. The skill is in scoping ruthlessly: what's the smallest thing that proves value?

Q: How do you think about extensibility for features like this? A: The redaction layer and chunking pipeline were built as composable modules, not a monolith. When the customer later wanted to add a second document corpus (HR policies), it took 2 days instead of 6. The principle: design for extension, not modification.