Deploying an LLM Feature at an Enterprise in 6 Days: An FDE Case Study

The Enterprise LLM Sprint: Context and Constraints

The Forward Deployed Engineer (FDE) role sits at the intersection of a SWAT team engineer and a solutions architect. You aren't just writing code; you are the tip of the spear for closing a $500K+ annual contract. The scenario: a multinational logistics customer with 15,000 employees needs an internal knowledge assistant to prevent $2M/year in lost productivity from searching fragmented SOPs.

The technical ask: deploy an LLM-powered "Ask SOP" feature on their legacy on-premise SharePoint. The catch? We have six days to go from a signed pilot agreement to a live, compliant feature on their VPN. No vaporware. No "it works on my machine." This playbook breaks down the exact architecture, code, and stakeholder management required to win.

The Architecture Blueprint

We can't just drop an OpenAI API key into their network. The enterprise environment required a hybrid architecture with a local embedding model to keep sensitive data in-house and a secure proxy to a managed LLM.

This flow ensures zero raw data leaves the perimeter. We used all-MiniLM-L6-v2 for local embeddings and Qdrant as a lightweight Rust vector store that could run on their existing Windows Server without Docker.

Day 0: The Scoping Call and the "No-Training-Data" Trap

Before writing a single line of Python, the FDE must disarm the enterprise stakeholder. The logistics VP asked, "How long to train the AI on our SOPs?"

This is a classic trap. Training implies fine-tuning, which requires high-quality labeled data they don't have and a timeline we can't meet. The pivot: Retrieval-Augmented Generation (RAG) . We explained we aren't retraining a brain; we are giving a smart agent a flashlight to read their library in real-time. This reset the expectation from "months of ML training" to "days of data plumbing."

Tooling and the Tech Spec

We committed to a technical spec on the call:

• Embedding: sentence-transformers running on CPU (to avoid GPU procurement hell).
• Database: Qdrant, deployed via a single binary on their server.
• Ingestion: A Python script to parse SharePoint exports (messy HTML and docx files) into markdown chunks.
• Interface: A Word Add-in (JavaScript) because that's where their operators live.

Days 1-2: Architecture, RAG, and the Data Silo Problem

Day 1 is about data extraction. The customer provided a 4GB dump of SharePoint files. We discovered 60% were scanned PDFs of physical paperwork. We needed a rapid OCR pipeline. We used pdf2image and pytesseract to convert scans to text on the fly during ingestion.

Chunking Strategy

Enterprise SOPs have massive tables. Naive recursive splitting breaks tables into nonsense. We wrote a custom MarkdownTextSplitter that respected table boundaries:

import re
from typing import List

def split_markdown_aware(text: str, max_chunk: int = 512) -> List[str]:
    # Split by markdown headers while preserving tables
    sections = re.split(r'(^#{1,6} )', text, flags=re.MULTILINE)
    chunks = []
    current_chunk = ""
    for part in sections:
        if len(current_chunk) + len(part) > max_chunk:
            chunks.append(current_chunk.strip())
            current_chunk = part
        else:
            current_chunk += part
    chunks.append(current_chunk.strip())
    return [c for c in chunks if c]

Stakeholder Check-in

At the end of Day 2, we demoed the ingestion pipeline on a Zoom call. We showed a raw SharePoint PDF and the resulting parsed markdown in Qdrant. The IT director nodded. The VP asked, "Will it hallucinate?" We promised to tackle that next with a guardrail layer.

Days 3-4: The Guardrail Gauntlet and Stakeholder Demos

This is where most LLM pilots fail. The model knows the SOPs but invents a procedure for "hazmat spill" that skips a legal step. We built a three-tier guardrail system:

1. Retrieval Score Threshold: If the cosine similarity of the retrieved context is below 0.75, the system forces a "I don't know, please consult the safety manual" response.
2. Output Validation: A regex layer to ensure critical safety codes (e.g., "SOP-101-B") are present in the answer if they were in the context.
3. Prompt Engineering: The system message strictly pinned the model to the context: You are a logistics assistant. Answer ONLY based on the provided SOP context. If the context lacks the answer, state that you cannot find the relevant procedure. Never guess.

The "Vomit" Sanitizer

We encountered a bizarre bug: the Azure OpenAI proxy occasionally returned malformed token streams with repeated unicode blocks when parsing complex tables. We deployed a lightweight output sanitizer to clean the token stream before it reached the UI. The concept is similar to how we handle sanitizing LLM code output in code-generation tasks, but adapted for prose and tables.

By Day 4, we had a working Word Add-in sidebar. A user could highlight a phrase, click "Ask SOP," and get a verified answer.

Days 5-6: Bare-Metal Integration and the Go-Live

The final hurdle was deployment. The IT security team blocked our initial Python requests library because of their custom SSL inspection certificate. We had to bundle the custom .pem file and modify the httpx client trust store.

import httpx
client = httpx.Client(verify="/path/to/corp-cert.pem")

We wrapped the FastAPI retriever as a Windows Service using pywin32 so it would survive server reboots. The go-live wasn't a splashy launch; it was a silent deployment to a pilot group of 50 warehouse managers. By the end of Day 6, we had logged 200 queries. The VP searched for "cold storage packing" and got the exact SOP paragraph in 2 seconds. The pilot was approved to expand to 500 users, triggering the contract's first expansion milestone.

The FDE Career Context: Why This Role Is Booming

This case study isn't just about code; it's about the business value of the FDE role. We didn't just build a feature; we navigated a procurement process, debugged SSL certs, and managed a VP's expectations. This blend of high-stakes execution is why the demand for Forward Deployed Engineers is skyrocketing. Companies need engineers who can close the gap between a generic API and a specific enterprise reality.

For engineers eyeing AI engineer jobs in Nepal or similar emerging markets, the FDE track is a high-leverage path. It offers exposure to global clients and complex problem-solving without necessarily requiring a PhD. The key is mastering the "last mile" of AI: the integration, the guardrails, and the stakeholder communication.

If you are preparing for this role, the interview process is unique. It tests not just your Python but your ability to whiteboard a customer architecture under pressure. To understand the specific breakdown of technical and stakeholder rounds, review the FDE interview loop and preparation guide.

FAQ: AI Engineer Jobs and Career Paths in Nepal

What is the average salary for an AI engineer in Nepal?

Salaries for AI engineers in Nepal vary widely based on experience and whether the role is local or remote for a foreign company. A junior AI engineer at a local firm might earn NPR 40,000–80,000 per month, while senior engineers or those working remotely for US/European startups can command $30,000–$60,000+ USD annually. Forward Deployed roles, which require client-facing skills and travel, often sit at the higher end of this range due to their direct revenue impact.

Which country is best for AI engineers?

The "best" country depends on your goal. For raw compensation and research depth, the United States (San Francisco, New York) leads. For a balance of high quality of life and strong engineering culture, Switzerland and the UK are excellent. However, with the rise of remote work, many engineers in Nepal are accessing US-level compensation without leaving home, making the remote-first FDE path extremely lucrative.

Which engineer has the highest salary in Nepal?

Within software engineering in Nepal, AI/ML specialists and Forward Deployed Engineers working for international product companies generally command the highest salaries. The FDE role often outpaces standard software engineering because the compensation is tied directly to revenue retention and expansion, not just headcount budgeting.

Which job is highly paid in Nepal?

Beyond engineering, high-paying jobs in Nepal include senior management in multinational corporations, specialized medical professionals, and development sector advisors. However, in the technology sector, roles like AI Engineer, DevOps Architect, and Forward Deployed Engineer are currently the most highly compensated due to the global demand/supply gap in these skills.