Deja Vu: SSH-Synced Agent Memory for Self-Hosted Coding Workflows

The Amnesia Problem in Coding Agents

You’ve felt it. You spin up a coding agent—Aider, Claude Code, a custom LangChain loop—point it at a repo, and watch it generate a patch. It works. Then you fire up a new session, ask it to extend yesterday’s feature, and it stares at you blankly. No context. No memory of the architectural decisions it made three hours ago. You’re back to typing /init and pasting a 200-line prompt full of constraints you already explained.

This isn’t just annoying. It’s a material drag on throughput. Every context reset costs you tokens, time, and the cognitive load of reconstructing state. For a single engineer hacking on a side project, it’s a papercut. For a Forward Deployed Engineer (FDE) juggling three customer integrations, each with its own bespoke codebase and a running dialogue with a non-deterministic LLM, it’s a hemorrhage.

The core issue is architectural: most coding agents treat state as ephemeral. They stuff context into the prompt window or a vector store that gets rebuilt. They don’t have a durable, queryable memory that survives process restarts and machine reboots.

What Deja Vu Actually Does

Deja Vu is an open-source memory layer designed explicitly for coding agents. It’s not a vector database. It’s not a prompt cache. It’s a relational memory store that records the agent’s decisions, file edits, tool calls, and reasoning traces in a structured format, then makes them queryable across sessions.

Here’s the plain-English breakdown:

• Records session history: Every agent interaction—user prompts, LLM responses, tool outputs, file diffs—gets logged as structured events.
• SQLite under the hood: The storage engine is a single SQLite file. No external database server. No Docker container. Just a file you can scp around.
• Syncs over SSH: This is the killer feature. Deja Vu synchronizes that SQLite database between your local machine and a remote server (or between multiple machines) using SSH. No cloud dependency. No API keys. Just standard Unix primitives.
• Agent-agnostic: It ships with integrations for popular open-source coding agents (Aider is the first-class citizen), but the event schema is generic enough to wrap around any tool-calling loop.

Think of it as git for your agent’s brain. Your code history lives in the repo; your agent’s decision history lives in a Deja Vu database that follows you across environments.

The Architecture: SSH as a State Bus

Most memory solutions for AI agents reach for a cloud database—Pinecone, Supabase, a self-hosted Postgres instance. Deja Vu takes a different bet: if you’re already managing SSH keys to access your dev servers, why add another network dependency?

The data flow is surprisingly simple:

Here’s what happens step-by-step:

1. Local agent runs a session. You ask Aider to refactor a module. Aider calls the Deja Vu plugin, which writes structured events—session_start, user_message, tool_call, file_edit, llm_response—to a local SQLite file.
2. Session ends. Deja Vu’s sync engine fires. It uses rsync (or a similar SSH-based file transfer) to push the updated SQLite file to a remote host. The remote path is configurable—could be your homedir on a dev server, a dedicated VM, or even a Raspberry Pi in your closet.
3. Remote agent picks up. You SSH into that server, launch Aider, and ask a follow-up question: “Why did we choose the visitor pattern for that parser?” The remote Deja Vu instance reads the synced database, retrieves the relevant session events, and injects them into the agent’s context. The agent answers with full knowledge of the earlier discussion.
4. Bidirectional sync. Changes made on the remote side sync back to local. The SQLite file is the single source of truth; SSH is the transport. Conflict resolution is currently last-write-wins, but the schema includes timestamps that make merging feasible.

The elegance here is in what’s not built. No message broker. No WebSocket server. No authentication service. Just SSH, which every engineer already has configured and which security teams already understand. For an FDE working in a customer environment where opening new ports is a multi-week approval process, this is a feature, not a limitation.

Why This Matters for Forward Deployed Engineers

Forward Deployed Engineers live in the gap between product and customer reality. You’re not just writing code; you’re building context. You learn a customer’s data model, their weird authentication layer, the undocumented API quirks. You have multi-hour debugging sessions where the agent proposes five different fixes and you guide it through trade-offs. That context is gold—and it evaporates the moment your terminal closes.

Deja Vu changes the calculus in three concrete ways:

1. Persistent Context Across Customer Environments

Picture this: you’re debugging a customer’s on-prem deployment. You have an agent session running locally with 40 messages of context about their LDAP configuration. You need to move to their staging server to test a fix. With Deja Vu, you sync the memory file to staging, launch the agent there, and it immediately knows the full history. No copy-pasting. No re-explaining.

This is the same muscle memory you use with git push and git pull. The agent’s state becomes as portable as your code. For more on debugging in constrained customer environments, see our playbook on debugging without direct access.

2. Audit Trails That Security Teams Actually Like

Enterprise security reviews hate black-box AI. They want to know what the agent did, what files it touched, and why. Deja Vu’s structured event log is an audit trail by construction. Every file edit is recorded with a timestamp, the agent’s stated reasoning, and the diff. When you’re defending an LLM feature to a security architect, pointing to a queryable SQLite database of agent actions is far more convincing than saying “trust me, the prompts were safe.” We’ve written about this dynamic in our case study on surviving enterprise security review.

3. Multi-Session Workflows Without Token Bloat

The naive approach to memory is stuffing the full conversation history into the prompt. That works for 10 messages. At 100, you’re burning tokens on irrelevant tangents and hitting context window limits. Deja Vu stores everything but retrieves selectively. You query for events related to a specific file, a specific decision, or a time range. The agent gets relevant history without the bloat. This is how AI-native startups keep their FDEs productive across long-running engagements—a pattern we explore in how FDEs win enterprise deals.

Getting It Running: A Practical Engineer’s Guide

Deja Vu is a Python package with a plugin interface for Aider. Here’s the fastest path from zero to synced memory, assuming you have Python 3.10+ and SSH keys configured.

Step 1: Install

pip install deja-vu-memory

Or clone the repo and install from source if you want to poke at the internals:

git clone https://github.com/vshulcz/deja-vu.git
cd deja-vu
pip install -e .

Step 2: Initialize a Memory Store

deja-vu init --path ~/.deja-vu/memory.db

This creates a SQLite database at the specified path. You can inspect it anytime with sqlite3 ~/.deja-vu/memory.db. The schema is straightforward: sessions, events, files, decisions.

Step 3: Configure the Remote Sync Target

Edit ~/.deja-vu/config.yaml:

sync:
  enabled: true
  remote_host: "dev-server.customer.com"
  remote_path: "~/.deja-vu/memory.db"
  remote_user: "fde-user"
  # Optional: use a specific SSH key
  ssh_key: "~/.ssh/customer_rsa"
  # Optional: sync interval in seconds (0 = sync on session end)
  interval: 0

Deja Vu will use your existing SSH config (~/.ssh/config) if the host matches an entry there. No need to duplicate connection settings.

Step 4: Wire It Into Aider

Aider supports plugins via its --load flag. Deja Vu ships with an Aider plugin:

aider --load deja_vu.plugins.aider

Start a coding session. Make a few changes. End the session. Deja Vu will sync the memory file to your remote host. On the remote side, launch Aider the same way, and it will read from the synced database.

Step 5: Query Memory Directly

You can query the memory store without an agent, which is useful for debugging or building custom tooling:

deja-vu query --session latest --type decisions

This returns the most recent session’s decision events as JSON. Pipe it to jq for filtering. The CLI also supports --file <path> to get all events related to a specific file and --since "2025-01-01" for time-range queries.

Step 6: (Optional) Wire Into a Custom Agent

If you’re not using Aider, the Deja Vu Python API is trivial:

from deja_vu import MemoryStore, Event

store = MemoryStore("~/.deja-vu/memory.db")

# Start a session
session_id = store.start_session(project="customer-x", tags=["debugging", "ldap"])

# Log an event
store.log_event(Event(
    session_id=session_id,
    type="file_edit",
    payload={
        "file": "auth/ldap.py",
        "diff": "...",
        "reasoning": "Switched to simple_bind_s for compatibility"
    }
))

# End session (triggers sync if configured)
store.end_session(session_id)

That’s it. Five lines of integration code. The SSH sync happens transparently when the session ends.

The Trade-offs: A Balanced Assessment

Deja Vu is a sharp tool with a clear opinion. That opinion won’t fit every workflow. Here’s an honest breakdown.

The SSH sync model is clever but introduces a subtle failure mode: stale reads. If you sync from local, work on remote, forget to sync back, then work on local again, you’ve diverged. The tool doesn’t do merge conflict resolution yet—it’s last-write-wins. For a solo engineer, this is manageable with discipline. For a team, it’s a sharp edge.

My take: Deja Vu nails the 80% use case for solo developers and FDEs who move between a local machine and a customer environment. The SSH-based architecture is a refreshing rejection of cloud sprawl. But if you need real-time multi-agent memory or semantic retrieval, you’ll want to layer something else on top—or wait for the project to mature.

For engineers building on local models, this pairs naturally with workflows like the screenshot-to-code agent using LLaVA via Ollama, where the agent’s visual understanding decisions are worth preserving across sessions. And if you’re thinking about how persistent agent memory changes the game for enterprise deployments, the pattern aligns with what we teach in FDE Coach’s training on production-grade AI integration.

FAQ

Q: Can I use Deja Vu with multiple remote hosts? A: The config supports a single remote target per instance. For multi-host setups, you can run multiple Deja Vu instances with different config files, or sync to a central jumpbox that other hosts pull from. The project’s README hints at multi-remote support as a roadmap item.

Q: How is this different from just saving chat logs? A: Chat logs are unstructured text. Deja Vu stores structured events with typed fields (tool_call, file_edit, decision). You can query by type, file, session, or time range. A chat log tells you what was said; Deja Vu tells you what was decided and done.

Q: Does the remote host need Deja Vu installed? A: Yes. The remote host needs the Deja Vu package to read the synced SQLite database and inject context into the agent. The sync itself only requires SSH and rsync, but the consumption of memory requires the library.

Q: What happens if the SSH connection drops mid-sync? A: Deja Vu uses rsync under the hood, which handles partial transfers gracefully. A failed sync will retry on the next session end. The local database is never corrupted by a failed remote push.

Q: Can I use this for non-coding agents? A: The event schema is generic enough for any tool-calling agent. You’d need to write a thin integration layer (similar to the Aider plugin) that maps your agent’s events to Deja Vu’s event types. The core library doesn’t care whether the agent writes code or schedules meetings.

Q: Is this production-ready for team use? A: It’s early-stage open source (check the repo for current activity). For a solo engineer or an FDE managing their own context across machines, it’s immediately useful. For a team sharing a memory store, the lack of merge conflict resolution and concurrent-write handling makes it risky. Treat it as a powerful personal tool that will likely grow team features.