Case Study: Deploying an LLM Feature That Survived an F100 Security Review

The Scenario: $1.2M ACV on the Line

A Fortune 100 commercial bank signed a $1.2M ACV deal for an internal tool that summarizes credit-risk memos. The catch: the model had to run inside their VPC, pass a 120-point security questionnaire, and never send a single byte of customer data to an external API.

I was the Forward Deployed Engineer (FDE) dropped into the account post-signature to turn a slick demo into a production feature that actually survived the InfoSec review. The demo used gpt-4 via the public Azure OpenAI service. The production reality required a self-hosted open-weight model, a custom retrieval pipeline, and a six-week review cycle that nearly killed the timeline.

This case study is the unvarnished playbook: architecture decisions, the security controls that mattered, and how closing this gate unlocked a promotion to Senior FDE.

The Architecture: Air-Gapped Inference

The bank’s red line was simple: no egress to the public internet from the inference path. That ruled out every managed LLM API. We landed on a design that kept the model inside their Kubernetes cluster on AKS, pulling documents from an on-prem SharePoint farm via a private link.

Model Selection Under Constraints

The bank’s infrastructure team gave us a single Standard_NC24ads_A100_v4 node (1x A100 80GB). That’s generous by startup standards, but tight for a production service expecting 50 concurrent analysts. We benchmarked three open-weight models with the following criteria:

We chose Mistral-7B-Instruct-v0.2 served via vLLM with continuous batching. The summarization task was extractive—pulling key figures, obligor names, and risk ratings from 20-page memos—and the 7B model handled it reliably after fine-tuning on 1,200 internal credit memos (anonymized, approved by compliance).

Why Not RAG Alone?

Pure retrieval-augmented generation failed on two fronts. First, credit memos contain tables with financial covenants that chunking destroys. Second, analysts needed citations to exact page and paragraph numbers. We built a hybrid pipeline:

1. Azure Form Recognizer extracted tables as structured JSON before chunking.
2. Weaviate stored dense embeddings (text chunks) and sparse BM25 vectors (keywords like “Debt Service Coverage Ratio”).
3. The orchestrator appended a strict instruction: “If you reference a financial figure, cite the source paragraph number in brackets [§4.2].”

This hybrid approach became the single biggest factor in the security review passing—because every output was auditable back to a source document, the compliance team could validate outputs without re-reading 20 pages.

The 6-Week Security Gauntlet

Enterprise security reviews aren’t a checkbox. They’re a negotiation. The bank’s questionnaire had 120 items across network security, data handling, model integrity, and access control. I’ll focus on the five that nearly blocked us.

1. Model Provenance & Supply Chain

Their ask: “Provide an SBOM for every artifact in the inference path.”

vLLM’s Docker image pulls ~200 Python packages. We generated a Software Bill of Materials using syft, then mapped every CVE to the bank’s risk matrix. One critical CVE in transformers (CVE-2024-3568, a pickle deserialization vector) required us to backport a patch and rebuild the image. The FDE skill here wasn’t deep security research—it was translating between the bank’s AppSec team and our internal infra engineers to ship a patched image in 48 hours.

2. Prompt Injection & Indirect Attacks

Their ask: “Demonstrate resilience against prompt injection from document content.”

Credit memos are adversarial by nature—an obligor might embed language designed to influence a risk rating. We implemented three controls:

• Input sanitization: All retrieved chunks were wrapped in XML-style <source> tags. The system prompt explicitly instructed the model to treat content inside <source> as data, not instruction.
• Output validation: A lightweight regex-based guardrail scanned summaries for prohibited phrases (“no risk,” “guaranteed approval”) and flagged them for human review.
• Structured output: We forced the model to output JSON with a fixed schema ({"obligor": "...", "risk_rating": "...", "key_covenants": [...]}). If the JSON failed to parse, the request was rejected.

This defense-in-depth approach satisfied their red team’s internal pen test. For a deeper dive on prompt injection vectors, see our breakdown of The Memory Heist: How Prompt Injection Can Leak Claude's Persistent Memory.

3. Data Isolation & Tenant Boundaries

Their ask: “Prove that one business unit’s documents cannot leak into another’s summaries.”

Weaviate’s multi-tenancy feature was the answer. Each business unit (Commercial Lending, Wealth Management, etc.) got a separate Weaviate tenant with its own vector index. The orchestrator resolved the tenant from the user’s JWT claim and set a tenant= parameter on every query. We verified isolation with a chaos test: inserting a poisoned document into the Commercial Lending tenant and confirming that Wealth Management queries never returned it.

4. Audit Logging

Their ask: “Every inference must be logged immutably for 7 years.”

We shipped structured logs to Azure Log Analytics with a 7-year retention policy. Each log entry contained:

• User principal name (from SSO)
• Timestamp
• Prompt text (the assembled prompt, not just the user query)
• Model response
• Retrieved source document IDs and paragraph numbers
• Latency breakdown (embedding, search, generation)

This logging volume was non-trivial—roughly 2GB/day at 500 inferences/day. We negotiated down from “log the full prompt” to “log a hash of the prompt + the full response” after proving we could reconstruct the prompt from the source document IDs. This is classic FDE work: finding the compromise that satisfies compliance without blowing up the infrastructure budget.

5. Fail-Open vs. Fail-Closed

Their ask: “If the model returns malformed output, does the system fail open or closed?”

We argued for fail-closed: if the JSON schema validation failed, the system returned an error to the analyst rather than showing raw, potentially misleading text. The bank’s risk team agreed. We added a fallback that, after three retries, escalated the memo to a human reviewer queue. This decision added a week of implementation but was critical for the security sign-off.

The Rollout: Shadow Mode to Production

Even after the security review passed, we didn’t flip a switch. We ran in shadow mode for two weeks: the model generated summaries silently while analysts continued writing them manually. We compared model outputs against human-written summaries on 200 memos, measuring:

• Factual accuracy: Did the obligor name, amount, and risk rating match? (97.3%)
• Completeness: Were all required sections present? (94.1%)
• Citation correctness: Did bracketed paragraph numbers point to the right source? (91.8%)

The 8.2% citation error rate was unacceptable. We traced failures to the chunking strategy—paragraphs spanning page breaks were split incorrectly. A fix to the chunking logic (overlapping windows of 512 tokens with 64-token stride) brought citations to 96.5%.

We also instrumented the UI to collect implicit feedback: every time an analyst edited a model-generated summary, we logged the diff. After 90 days, the edit distance dropped 40%, indicating the model was improving (or analysts were trusting it more—we tracked both interpretations).

FDE Comp & Career Context

This project was the anchor of my promotion case to Senior FDE. Here’s the comp context most engineers don’t get:

The jump from FDE to Senior FDE is rarely about code volume. It’s about proving you can navigate a Fortune 100 security review, translate between AppSec and engineering, and ship a feature that unlocks revenue. This single project de-risked $1.2M in ACV and became the reference architecture for three subsequent bank deals.

For a ground-level view of how these projects fit into the broader FDE rhythm, read A Week in the Life of a Forward Deployed Engineer: Demos, Debugging, and Deadlines. And if you’re wondering how FDEs are measured beyond ship velocity, Metrics an FDE Actually Owns: Time-to-Value, Adoption, and Expansion Revenue breaks down the numbers that matter.

FAQ

What is a key challenge of deploying LLMs in customer service? Data isolation and tenant boundaries dominate. In customer service, one client’s support tickets must never leak into another client’s responses. Multi-tenant vector databases and strict JWT-scoped queries are the standard fix, but they must be verified with chaos testing—inserting poisoned data into one tenant and proving it never surfaces elsewhere.

What are the use cases for LLM in Enterprise? The highest-ROI enterprise use cases we see are: (1) internal document summarization (credit memos, RFP responses, legal contracts), (2) customer-support agent assist (retrieval over knowledge bases with draft replies), (3) code migration and legacy system modernization, and (4) compliance monitoring (flagging regulatory gaps in communications). The common thread is high-volume, structured-input tasks where a human reviewer remains in the loop.

How do you handle model updates without breaking the security review? Treat model weights as versioned artifacts with a full CI/CD pipeline. Every new model version triggers a lightweight re-review (SBOM diff, benchmark accuracy on a held-out eval set, and a subset of the original pen tests). The first review is the heavy lift; subsequent updates should be 80% automated.

Is fine-tuning worth it for enterprise deployments? For narrow, high-value domains (credit analysis, medical coding, legal document review), yes—a fine-tuned 7B model often outperforms a generic 70B model on the specific task and runs cheaper. But fine-tuning introduces a new attack surface (data poisoning, model inversion). Budget an extra two weeks in the security review for the fine-tuning pipeline.

What’s the biggest mistake FDEs make in enterprise LLM projects? Assuming the demo architecture will survive the security review. The demo uses managed APIs; production requires self-hosted models, private links, and audit logging. Start the security conversation in week one, not week ten. The FDE who ships is the one who treats the security questionnaire as a design document, not an afterthought.