Case Study: Deploying a RAG-Based LLM Feature at an Enterprise Customer in 10 Days

The 10-Day Constraint and the 'Trust Battery'

A logistics customer with 15,000 employees had a simple problem: their claims adjusters spent 40% of their day searching a fragmented knowledge base of 12,000 PDFs. The ask was an internal Q&A bot. The timeline was two weeks. We had 10 working days.

This is the standard Forward Deployed Engineer (FDE) tempo. You aren’t just shipping code; you are shipping confidence. The first 24 hours aren't for coding—they are for charging the 'trust battery.' We sat with the claims team lead, not the IT VP, and mapped their actual workflow. The critical insight wasn't technical; it was behavioral. Adjusters didn't want a chat window. They wanted an answer injected directly into the claim form they already had open.

As an FDE, your value is compressing the time between a customer’s vague pain point and a concrete, deployed artifact. This often means saying 'no' to elegant engineering in favor of high-reliability glue. For more on this weekly rhythm, see What a Forward Deployed Engineer Actually Does in a Week.

Architecture: Why We Killed the Vector DB (For Now)

Enterprise environments are graveyards for novel infrastructure. The customer’s security team had a three-month review cycle for new data stores. Postgres, however, was pre-approved. We chose pgvector over Pinecone or Weaviate not because it's technically superior, but because it was boring and already inside the trust boundary.

We used an off-the-shelf embedding model (text-embedding-3-small) because fine-tuning a custom model would have blown the timeline. The retrieval logic was a simple hybrid search: keyword BM25 for exact policy numbers, cosine similarity for semantic meaning.

The Data Pipeline: Unstructured PDFs to Structured Context

The 12,000 PDFs were a nightmare: scanned images, no OCR, inconsistent headers. We built a three-step ingestion pipeline using Python and unstructured.io.

Step 1: Extraction. We ran OCR locally (no external API calls allowed) using Tesseract inside a Docker container. Step 2: Chunking. We used semantic chunking based on section headers. This was the highest-leverage decision. Fixed-size token chunks broke policy clauses in half, causing the LLM to hallucinate deductibles. Step 3: Metadata Tagging. We extracted policy_type and effective_date using regex and stored them in Postgres columns for pre-filtering. This reduced noise drastically.

# Critical chunking logic snippet
def semantic_chunk(text: str, max_chars: int = 1000) -> list[str]:
    splitter = RecursiveCharacterTextSplitter(
        separators=["\n## ", "\n# ", "\n### ", "\n", ". ", " "],
        chunk_size=max_chars,
        chunk_overlap=100
    )
    return splitter.split_text(text)

The Security Gauntlet: VPC Peering and Guardrails

This was the part that almost killed the project. The customer’s InfoSec team flagged our initial design because it sent data to OpenAI’s public endpoint. The fix was a two-hour fire drill:

1. VPC Peering: We routed traffic through a private link to Azure OpenAI Service inside their tenant.
2. Prompt Injection Guard: We implemented a lightweight input sanitizer that rejected prompts containing more than 3 consecutive special characters or obvious jailbreak strings. It wasn't perfect, but it satisfied the "reasonable effort" clause in the security review.
3. Logging: We zeroed out PII in logs using Microsoft Presidio.

If you want to dive deeper into the specific prompt engineering techniques that prevent hallucinations in these high-stakes environments, Claude Opus 5: What Extended Thinking Means for Complex Engineering Work is highly relevant.

The 'Last Mile' UX Hack

We didn't build a web app. The adjusters lived in a legacy Java Swing application. We built a lightweight Chrome Extension that injected an answer panel directly into the DOM.

When an adjuster highlighted a claim description, the extension called our FastAPI backend. The backend retrieved the top 3 policy chunks, stuffed them into a strict JSON-mode prompt, and returned a structured object: { "summary": "...", "coverage_decision": "Covered/Denied/Review", "citation": "Policy 45-A, Page 2" }.

This is the FDE "last mile." The model’s output is worthless if the user has to copy-paste it. The integration must be native. For another example of this injection pattern, check out Build a Browser Extension Autofill Agent for Job Applications Using Playwright and Gemini.

Comp and Career Context for the FDE

Why does an FDE take a $180k-$250k base salary (plus significant equity) to do this instead of a standard solutions engineer? Because an FDE writes production code that lives inside the customer’s stack, not just slide decks.

In this 10-day sprint, the FDE role involved:

• Backend Engineering: Python, FastAPI, SQL.
• ML Engineering: Embedding strategies, RAG evaluation metrics.
• Security Engineering: VPC networking, PII redaction.
• Product Management: Identifying the "highlight-to-answer" workflow.

The market is shifting. Companies are realizing that shipping AI features requires this hybrid profile. If you're preparing for loops that test this breadth, avoid grinding abstract algorithms. Focus on execution scenarios. The FDE Interview Loop: How to Prepare for Execution, Not LeetCode Crimes maps out the exact signal you need to send.

FAQ

Q: Why not just fine-tune a model on the 12,000 PDFs? A: Time and maintainability. Policies change monthly. A RAG architecture lets us update the vector index without retraining. For a 10-day timeline, fine-tuning is a trap.

Q: How did you handle hallucinations in citations? A: We forced the LLM to output verbatim strings from the retrieved chunks. If the output string didn't fuzzy-match the source text, we threw a "Source Unavailable" error to the user rather than guess.

Q: Is this just professional services? A: No. Professional services hand over code and leave. An FDE maintains a persistent, on-call relationship with the customer’s engineering team to keep the feature alive. It's a product engineering role, just deployed forward.

Q: What was the actual business impact? A: Average claim processing time dropped from 22 minutes to 4 minutes for standard queries. The customer expanded the contract from a 10-day POC to a 12-month enterprise license.