Shipping an LLM Feature at a Bank in 5 Days: An FDE Case Study

The 5-Day Window

A top-5 bank’s wealth management division had a grinding problem. Advisors spent 30% of their morning reading overnight research reports—50-page PDFs from Goldman, Morgan Stanley, and internal macro teams—and manually distilling them into client-ready summaries before the opening bell.

The CTO wanted an LLM to do it. The internal AI team had a 6-month roadmap. My team had 5 days.

This isn’t a hypothetical. This is what a Forward Deployed Engineer actually does: parachute into a high-stakes enterprise environment, navigate Byzantine security constraints, and ship working software before the window closes. No PoCs. No handoffs. Code in production, or it didn’t happen.

Here’s exactly how we did it, and the forward deployed engineer prerequisites that make this speed possible.

Day 1: Scoping the Blast Radius

Enterprise banks don’t do “move fast and break things.” They do “move carefully and don’t touch anything.” My first job was defining what we were not going to do.

The constraints:

• No data leaves the VPC. Period.
• No training on client data. No fine-tuning. Inference only.
• The output goes to a human advisor, not a client. The human is the approval layer.
• Must work on 50+ PDFs simultaneously by 6:30 AM EST.

I spent the morning in a windowless conference room with the head of InfoSec, a compliance VP, and two architects. We drew a hard boundary: the LLM would run on an internal SageMaker endpoint using a model they’d already approved (Llama 3 70B, hosted in their AWS account). The PDFs were already in an internal S3 bucket. The output would land in a DynamoDB table that their existing advisor portal already read from.

This is the first FDE prerequisite: the ability to scope ruthlessly. You’re not building a platform. You’re solving one specific pain point with the minimum viable surface area. Every component we touched already existed. We were just wiring them together with a thin layer of logic.

By 2 PM, I had a one-page architecture diagram approved by InfoSec. By 5 PM, I had AWS credentials scoped to exactly three services.

Day 2: The Architecture Decision

The obvious approach was a single Lambda that reads a PDF, chunks it, sends chunks to the LLM, and assembles the output. That fails at scale. A 50-page PDF with financial tables and legal disclaimers can hit 100K tokens easily. Llama 3 70B’s context window handles it, but the latency on sequential chunking would blow past the 6:30 AM deadline.

I needed parallelism.

The design:

1. An S3 event triggers a dispatcher Lambda when new PDFs land (usually 4:00–5:00 AM batch uploads).
2. The dispatcher splits each PDF into chunks of ~8K tokens with 500-token overlap, using a sliding window that respects paragraph boundaries. Each chunk gets a metadata header: [Report: Goldman Sachs US Equities Daily, Date: 2025-03-15, Chunk: 3/7].
3. Each chunk is enqueued as a separate SQS message.
4. A worker Lambda picks up messages, sends the chunk to SageMaker with a strict system prompt, and writes the partial summary to DynamoDB with a sort key indicating chunk order.
5. A final assembler Lambda triggers when all chunks for a report are complete (tracked via a DynamoDB counter), concatenates them, and runs one final pass through the LLM to produce a cohesive executive summary.

The system prompt was the secret weapon:

You are a financial research summarizer. Output ONLY a JSON object with these keys:
- "headline" (max 15 words)
- "key_points" (array of 3-5 strings, each max 30 words)
- "market_implications" (string, max 50 words)
- "risk_factors" (array of strings, max 3 items)

Do not include any text outside the JSON. Do not hallucinate data not present in the source.

Structured output isn’t a nice-to-have in production. It’s the difference between a feature that integrates cleanly and a mess of unstructured text that breaks downstream consumers.

Day 3: The Integration Gauntlet

This was the day everything almost died.

At 10 AM, I discovered the bank’s PDFs weren’t clean text. They were scanned image PDFs with embedded OCR text layers of inconsistent quality. Some had tables that PyPDF2 rendered as gibberish. Others had multi-column layouts where the text extraction mixed column A and column B into nonsense.

I had two choices: build a preprocessing pipeline (no time) or find a model that could handle the noise. I took a bet: Llama 3 70B’s multimodal variant could process the PDF pages as images, but the bank’s approved model list only included the text-only version.

The workaround: I used AWS Textract for OCR preprocessing. Textract handles tables natively and outputs structured text with bounding boxes. I wrote a 40-line Python script that takes Textract’s JSON output, reconstructs reading order using the bounding box coordinates, and produces clean linear text. This fed into the existing chunking pipeline with zero changes downstream.

By 6 PM, the pipeline was processing 50 PDFs in parallel. End-to-end latency for a single 50-page report: 47 seconds. For all 50 concurrently: 3 minutes 12 seconds. Well within the 6:30 AM window.

This is where the second FDE prerequisite shows up: comfort with fallback engineering. You don’t have time to build the perfect solution. You need a working solution, and you need to know when a 40-line script is the right answer versus a 4,000-line ETL framework.

Day 4: The Hallucination Firewall

An LLM summarizing market research will hallucinate. It will invent stock tickers, fabricate analyst names, and sometimes generate plausible-sounding but entirely fictional market movements. In a bank, that’s not a bug—it’s a compliance violation.

I built a two-layer hallucination firewall:

Layer 1: Groundedness verification. For every factual claim in the summary, I ran a reverse-check: extract the claim, search for it in the source text using fuzzy string matching (rapidfuzz, threshold 0.85), and flag anything that doesn’t have a source match. This ran as a post-processing step in the assembler Lambda.

Layer 2: Structured output validation. The system prompt forced JSON output, but LLMs sometimes add trailing text or malform the JSON. I added a Pydantic validation layer that parsed the output, validated the schema, and rejected any response that didn’t match. On rejection, the worker retried with a stronger prompt: Your previous response was not valid JSON. Output ONLY the JSON object. No other text.

The results:

• 8% of chunks failed the groundedness check on first pass.
• 3% of responses were malformed JSON.
• After retries, 99.7% of summaries passed both layers.

The 0.3% that still failed? Flagged with a needs_human_review: true field in DynamoDB and surfaced in the advisor portal with a red banner. The advisor knows to read the source PDF for those.

This is the third prerequisite: paranoia as a feature. You don’t trust the model. You build systems that assume the model will fail and catch it before the user does.

Day 5: Shipping and the Silent Launch

Friday. 5 AM. The first batch of PDFs hit S3. I watched the CloudWatch dashboard like a hawk.

5:02 AM: Dispatcher Lambda fires. 247 chunks enqueued. 5:03 AM: Worker Lambdas scale to 50 concurrent invocations. 5:05 AM: First chunks hit SageMaker. Average latency: 8.2 seconds per chunk. 5:08 AM: First assembler Lambda triggers. 5:11 AM: First complete summary lands in DynamoDB. 5:47 AM: All 50 reports summarized. 0 failures. 0 hallucination flags.

At 6:30 AM, the first advisor logged in and saw the summaries. No fanfare. No launch email. Silent rollout to 12 advisors in a pilot group.

By 9 AM, I had Slack messages from three advisors asking if this was “that AI thing.” One said: “This saved me 45 minutes. Don’t take it away.”

The post-launch metrics after 1 week:

• 50 reports processed daily, 99.4% success rate.
• Average advisor time savings: 38 minutes/day.
• 0 compliance incidents.
• 0 support tickets.

Total lines of code written: ~600 Python across 4 Lambda functions. Total AWS services used: 6 (S3, Lambda, SQS, DynamoDB, SageMaker, Textract). Total meetings: 3.

The FDE Prerequisites That Made It Possible

This case study isn’t about the code. The code was straightforward. The hard parts were everything around the code.

Here are the forward deployed engineer prerequisites that determined whether this shipped in 5 days or 5 months:

For a deeper dive into the day-to-day reality of this role, see What a Forward Deployed Engineer Actually Does in a Week: A Concrete Workflow.

FAQ: Forward Deployed Engineer Prerequisites

What engineers make $500,000 a year?

Senior Forward Deployed Engineers at top AI companies (Palantir, OpenAI, Anthropic) and elite engineering firms regularly hit $500K+ total compensation. This includes base salary ($200-250K), performance bonuses, and significant equity. The premium exists because FDEs combine engineering depth with customer-facing execution—a rare combination that directly drives revenue. For a detailed breakdown of the interview process that leads to these roles, see The FDE Interview Loop in 2025: A Practical Preparation Guide.

How much do FDEs get paid?

Entry-level FDE roles start around $150-180K base + equity. Mid-level (3-5 years) ranges from $220-300K total comp. Senior/staff FDEs at market-leaders reach $400-700K+. The variance depends heavily on company stage and whether the role is at a product company (higher equity upside) or a services firm (higher base, lower equity).

How to become a forward deployed engineer with no experience?

You can’t start at zero. The role requires production engineering experience—typically 2-5 years as a backend, infrastructure, or full-stack engineer. The most common path: work as a software engineer at a company with real production systems, develop customer-facing skills through on-call rotations or client projects, and then transition. If you’re early in your career, focus on building strong fundamentals in distributed systems, cloud infrastructure (AWS/GCP), and at least one scripting language deeply (Python is the default). Document everything you build publicly. The writing and communication skills matter as much as the code—FDEs spend 30-40% of their time on technical documentation and stakeholder communication. For guidance on that skill specifically, see Writing Customer-Facing Technical Docs That Non-Engineers Actually Read.

Do forward deployed engineers need to code?

Yes, deeply. This is not a solutions architect or sales engineer role where you configure existing products. FDEs write production code daily—often in messy, constrained environments with no existing SDKs or clean APIs. In this bank case study, I wrote ~600 lines of Python across 4 Lambda functions, built a hallucination verification system, and integrated with 6 AWS services. The code ships to production and runs without a dedicated ops team. If you can’t independently build and deploy a production service, you’re not ready for an FDE role.

What’s the difference between a Forward Deployed Engineer and an AI Engineer?

An AI Engineer focuses on the model: fine-tuning, prompt engineering, evaluation, and model performance. An FDE focuses on the system around the model: deployment, security, integration with existing enterprise infrastructure, hallucination guardrails, and getting the feature into users’ hands. FDEs use AI Engineers’ work as a component, not the product. In practice, FDEs often do both in resource-constrained environments, but the FDE’s north star is shipped value, not model accuracy.

What are the technical prerequisites for this role?

Strong Python (or equivalent), deep experience with at least one cloud provider (AWS, GCP, or Azure), comfort with infrastructure-as-code (Terraform, Pulumi, or CDK), understanding of distributed systems patterns (queues, idempotency, retries, circuit breakers), and the ability to read and integrate with poorly-documented APIs. On the AI side: practical experience with LLM APIs, structured output techniques, RAG patterns, and evaluation/verification methods. If you want hands-on practice with these patterns in a lower-stakes environment, check out Build an On-Call Incident Summarizer That Drafts Postmortems from Logs—it exercises the same chunking, summarization, and integration patterns used in enterprise deployments.