AI Engineer vs Forward Deployed Engineer: 4-Week LLM Deployment at a Regulated Bank

The 4-Week Constraint: Scope and Stack

A top-5 US bank needed an internal "Policy Q&A" tool for their wealth management division. The ask was deceptively simple: “Let advisors ask natural language questions about our 1,200-page operational manual and get instant, cited answers.”

The constraints were not simple:

• Zero egress: Data must never leave the bank’s VPC.
• Role-Based Access Control (RBAC): A junior advisor cannot see M&A policies intended for VPs.
• Auditability: Every query and response logged immutably for FINRA compliance.
• Latency: Sub-3-second responses to avoid disrupting client meetings.

This is the exact environment where the distinction between an AI Engineer and a Forward Deployed Engineer (FDE) collapses into sharp relief. The AI Engineer builds the RAG pipeline. The FDE ships it inside a fortress.

Week 1: The Discovery Minefield (RBAC & PII)

We didn’t start with code. We started with the IAM team’s whiteboard. The bank’s document repository wasn’t a clean vector store—it was a labyrinth of SharePoint folders with deeply nested ACLs inherited from Active Directory.

The core technical discovery: The embedding strategy had to be multi-tenant at the chunk level. A naive single-collection vector store was a non-starter. If we embedded all 1,200 pages together, a VP’s sensitive document chunk would sit in the same index as a generic onboarding doc. A similarity search for “acquisition thresholds” could leak data via the vector distance alone.

Decision: We partitioned the vector store by security group. We used a metadata filtering approach with pgvector, attaching an allowed_groups array to every chunk metadata.

-- Simplified metadata filtering to enforce RBAC at retrieval
SELECT chunk_text, document_id
FROM policy_chunks
WHERE embedding <-> query_embedding < 0.3
  AND allowed_groups && ARRAY['wm_vp', 'compliance_officer']
ORDER BY embedding <-> query_embedding
LIMIT 5;

PII Redaction Pipeline: We discovered that the “static” manuals contained residual PII in footnotes (client names from redacted case studies). We implemented a pre-embedding sweep using a fine-tuned spaCy model running locally, replacing entities with typed tokens ([PERSON], [ACCOUNT]) before they hit the text splitter.

The FDE distinction here is ownership. An AI Engineer might hand off a chunking strategy. The FDE sits in the IAM meeting, maps the group hierarchy, and writes the authorization middleware before the LLM is even selected.

Week 2: Building the Secure Retrieval Core

With the data pipeline defined, we moved to the retrieval logic. The bank’s security team approved a self-hosted Llama-3-70B instance on their internal GPU cluster. No API calls to the outside world.

Architecture Flow:

We spent most of Week 2 on the Query Rewriter, not the LLM. Advisors type in shorthand: “whats the rule on options for clients over 70”. That query, embedded directly, has low cosine similarity to the formal language in the manual (“Derivatives trading authorization for clients exceeding the age threshold defined in Section 4.2…”).

We used a lightweight local model (Llama-3-8B) for HyDE (Hypothetical Document Embeddings)—generating a synthetic paragraph of formal policy language from the user’s slang before embedding it. This bridged the semantic gap without touching an external API.

Week 3: The Air-Gapped Evaluation Gauglet

This is where the AI Engineer vs FDE salary gap becomes visible. An AI Engineer might evaluate RAGAS scores in a notebook. An FDE has to build an evaluation framework that the bank’s Model Risk Management (MRM) team can sign off on.

We couldn’t use GPT-4 as a judge (no egress). We built a custom eval harness using the bank’s internal Llama-3-70B as a critic, but the MRM team required human validation. We created a Streamlit app where three compliance officers rated 150 query-response pairs on:

• Faithfulness (did the answer stay within the provided context?)
• Relevance (did we answer the advisor’s actual intent?)
• Safety (did we refuse to answer when uncertainty was high?)

We defined a strict refusal policy: if the top-3 chunks had a cosine similarity below 0.25, the system responded with “I could not find sufficient policy documentation to answer this question. Please contact the compliance desk.”

The Hallucination Firewall: We implemented a post-generation verification step. The LLM output was chunked into claims, and each claim was checked against the retrieved context using NLI (Natural Language Inference). If a claim was classified as “contradiction,” the entire response was scrapped and the refusal message triggered.

Week 4: Silent Launch and the Feedback Loop

We deployed to 50 advisors in a single branch with a silent launch (no announcement). We instrumented the app with implicit feedback signals: if an advisor copied the response and pasted it into an email, that was a positive signal. If they immediately rephrased the query, that was a negative signal.

Critical Bug on Day 2: An advisor asked about “penny stock restrictions” and received a response citing a policy that had been updated 48 hours prior. The vector store was stale. We hadn’t built a real-time sync from SharePoint. The FDE fix: we didn’t rebuild the pipeline. We implemented a lightweight freshness layer—a separate SQL query that checked the last_modified timestamp in SharePoint for any chunk returned, and if older than the source, appended a warning: “Note: This policy may have been recently updated. Verify with the latest manual.”

This is the core FDE instinct: don’t over-engineer the happy path; harden the failure modes.

AI Engineer vs Forward Deployed Engineer: The Execution Gap

After this case study, the distinction between these roles should be concrete. The market data backs it up.

Comparison Table

The salary premium for FDEs comes from scarcity. It’s easier to find someone who can fine-tune a model on a clean dataset than someone who can deploy that model inside a bank while keeping a straight face in a 3-hour security architecture review.

If you’re weighing the AI Engineer vs Forward Deployed Engineer path on Reddit or elsewhere, the fork is simple: do you want to push the boundary of what models can do, or do you want to push the boundary of where models can survive? Both are hard. One happens in a lab coat, the other in a flak jacket.

To build the deployment velocity that FDE roles demand, your portfolio needs to show you can navigate constraints, not just build demos. For concrete project ideas that demonstrate this skillset, see The FDE Portfolio: What to Build to Demonstrate Deployment Velocity and Get Hired.

For those looking to automate enterprise workflows, patterns like the one in Build a Calendar Negotiation Agent That Schedules Meetings Over Email Using Groq and n8n translate directly to the kind of internal tooling FDEs ship.

FAQ

What is the salary difference between an AI Engineer and a Forward Deployed Engineer? In 2026, AI Engineer roles typically range from $180K–$280K base, while FDE roles command $200K–$320K base. The premium reflects the FDE’s requirement to manage customer security reviews, on-call rotations, and deployment risk. FDE packages often include larger cash bonuses tied to customer go-live milestones.

Is a Forward Deployed Engineer the same as a Solutions Engineer? No. A Solutions Engineer (SE) typically demonstrates the product and designs the architecture during the sales cycle. An FDE stays with the customer post-signature to write integration code, build custom features, and ensure the deployment survives the customer’s unique infrastructure constraints. The SE sells it; the FDE makes it work.

Do I need a certification to become a Forward Deployed AI Engineer? There is no formal certification that matters. FDE hiring managers look for a portfolio of shipped projects under real constraints (auth, rate limits, air-gapped data). The FDE Interview Loop Decoded covers the practical demonstrations you’ll face instead of trivia questions.

What’s the hardest part of deploying LLMs in regulated environments? It’s rarely the model itself. The hardest parts are RBAC enforcement at the retrieval layer, immutable audit logging, and building an evaluation framework that satisfies Model Risk Management teams who are accustomed to traditional ML models, not generative text.