5-Day LLM Prototype: An FDE's Playbook for Messy Enterprise Problems

The call came in on a Tuesday at 4 PM. A logistics enterprise customer had a “small” problem. Their claims adjusters were spending 45 minutes per case manually cross-referencing damage photos against a 600-page internal PDF of coverage rules. They wanted “AI” to do it. They had no API, no clean dataset, and a deadline of “yesterday.”

This is the standard Forward Deployed Engineer (FDE) starting point. It’s not a Kaggle competition. It’s a messy, high-stakes trust exercise. Here is the exact 5-day playbook for turning that chaos into a shipped LLM prototype, and why this workflow defines the modern ai engineer job description for resume bullet points.

Day 0: The Inbound Mess

Before writing a line of code, you have to triage the organizational scar tissue. The customer’s CTO promised the board an “AI autopilot.” The claims manager just wants to reduce overtime. The individual adjusters are terrified of being automated.

The FDE Move: Ignore the CTO’s grand vision and embed with the adjusters. Watch them work for 2 hours. You’ll notice they don’t read the whole 600-page PDF. They use Ctrl+F for specific damage codes (e.g., “WET-ROT-3B”) and flip to dog-eared pages. The AI doesn’t need to understand insurance; it needs to replicate the Ctrl+F + visual pattern matching of a tired human at 5 PM.

The hard reset: Tell the customer you are not building an autopilot. You are building a “second set of eyes” that highlights the relevant page in <5 seconds. This manages expectations and saves you from building a skynet agent that hallucinates liability.

Day 1: Scoping Down to the Atomic Pain Point

Enterprise prototypes die when they try to ingest 600 pages perfectly. The FDE scopes down to the atomic unit of value.

We identified that 80% of the adjusters’ time was spent on just 3 types of water damage claims. The PDF section for this was only 40 pages.

Technical Scoping:

• Input: A mobile photo of a water-stained ceiling.
• Process: Extract visual descriptors (not just OCR) -> Match against a vectorized subset of the PDF.
• Output: The single most relevant clause (not a chat response).

Tooling Decision: For a 5-day prototype, fine-tuning is a trap. We need retrieval-augmented generation (RAG) with a strong vision model. The stack: Python, GPT-4o (for vision), and a local ChromaDB instance to avoid enterprise network egress headaches.

Day 2: Building the Dirty Pipeline (RAG on a Laptop)

Enterprise data is never clean. The PDF was scanned at a slight rotation, with handwriting in the margins. Standard OCR libraries (Tesseract/PyPDF) produced gibberish.

The Chunking Strategy: We bypassed traditional OCR entirely. We converted every page of the PDF to a high-resolution PNG. For the ingestion pipeline, we used a loop:

import base64
from openai import OpenAI

client = OpenAI()

def describe_page(image_path):
    with open(image_path, "rb") as f:
        base64_image = base64.b64encode(f.read()).decode("utf-8")
    
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Extract all text verbatim. Describe any diagrams or handwritten notes."},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{base64_image}"}}
                ]
            }
        ]
    )
    return response.choices[0].message.content

This “vision-first” extraction was a revelation. It perfectly captured the marginalia (often containing the crucial edge-case rules). We chunked the text output by logical sections, embedded them using text-embedding-3-small, and stored them in ChromaDB.

The FDE Career Context: This is the reality behind an ai engineer job description for resume bullets. You aren’t just calling APIs; you are engineering around dirty data. This skill set—building deterministic guardrails around stochastic outputs—commands a $200k–$350k+ total comp range in high-end FDE roles, precisely because it requires production grit, not just notebook tinkering.

Day 3: Prompt Engineering the “Subject Matter Expert”

With the vector store live, the naive approach failed. A query for “mold on ceiling” returned a clause about “roof replacement deductible.” The vector similarity was high (both mention roofs), but the context was wrong.

We implemented a two-stage retrieval pattern with a re-ranker. We fed the top-5 chunks and the original image description to GPT-4o with strict instructions:

You are a claims coverage auditor. 
Given the damage description: {description}
And the following policy clauses: {chunks}

Identify the single most applicable clause. 
If no clause explicitly covers this damage, say "NO_COVERAGE_FOUND." 
Do not infer. Do not assume.

The NO_COVERAGE_FOUND escape hatch was critical. The customer trusted the prototype because it admitted ignorance rather than hallucinating a payout. This is the difference between a demo and a tool an adjuster will actually use. For more on building trust through technical honesty, see our breakdown of What a Forward Deployed Engineer Actually Does in a Week: Trust, Code, and Customer Obsession.

Day 4: The UX Hack That Made It Real

No enterprise user wants a chat window. They want a tool that fits into their existing Alt+Tab workflow. We built a single-file Streamlit app that ran locally on the adjuster’s machine.

The UI was brutalist: an image upload box, a “Check Coverage” button, and a text box showing the clause. The killer feature: a confidence score slider.

We visualized the cosine similarity scores from the vector search as a red/yellow/green badge. If the top result had a similarity < 0.75, the badge was red, and the app explicitly said, “Low confidence—please review manually.”

This transparency turned the adjusters from skeptics into collaborators. They started trying to break it, feeding it edge cases. This adversarial testing in the wild is worth more than any synthetic eval. The FDE embeds these feedback loops directly into the prototype, a practice we detail further in How Palantir-Style FDEs Embed with Customers: Weekly Rituals, Artifacts, and Trust.

Day 5: Shipping and the Art of the Hard Reset

We didn’t deploy to the cloud. We handed the adjuster a .bat script that launched the Streamlit server on localhost. The “shipping” was a 30-minute training session with the 3-person claims team.

The Outcome: Average coverage lookup time dropped from 45 minutes to 90 seconds for the targeted water damage subset. The prototype wasn’t perfect, but it was real. The customer signed a $150k expansion contract to productionize the full 600-page manual.

The FDE Lesson: You are not paid for the code. You are paid for the de-risked path to production. The prototype proved the value without requiring a $2M infrastructure overhaul. The messy, local-first Python script is often the most honest salesperson the enterprise has.

For those building out an ai engineer job description for resume, this is the narrative to highlight: the ability to collapse a 6-month enterprise software cycle into a 5-day trust-building artifact. It’s not about the LLM; it’s about the context engineering around it. As we explore in Why AI Coding Agents Stall: The Context Engineering Gap No One Talks About, the bottleneck is never the model’s IQ—it’s the messy human and data context it’s dropped into.

FAQ: 5-Day LLM Prototyping

Q: Why not fine-tune a model on the PDF? A: In a 5-day sprint, you lack the time to curate a clean Q&A dataset. Fine-tuning on noisy OCR data would amplify errors. RAG with a strong vision model is faster to iterate and easier to debug when it fails.

Q: How do you handle PII in the images? A: For the prototype, we used a local ChromaDB instance and ensured the Streamlit app ran entirely on the adjuster’s machine. No data left the device except the API call to the vision model, for which we used an enterprise account with a zero-data-retention agreement. In production, you’d swap in a self-hosted model like Llama 3.2 Vision.

Q: Is this really an “AI Engineer” role or just prompt engineering? A: The market is rapidly converging. The modern AI Engineer job description blends backend engineering, data engineering (chunking/parsing), and product sense. Prompt engineering is the thin UI layer on top of a deep stack of parsing, retrieval, and evaluation logic.

Q: What if the prototype fails on Day 5? A: Ship the failure analysis. If the model can’t distinguish between mold and soot, present that exact boundary as the deliverable. “Here is the specific technical hurdle to solve in Phase 2.” An FDE turns failure into a precise, billable workstream.