Constraint Injection: The Engineering Secret to Hallucination-Free Financial AI

The MIT Finding: Why 'The Right Questions' Matter

Researchers at MIT Sloan recently dropped a paper that should make every engineer sit up. The headline finding: AI financial advice is surprisingly good, but only if you ask the right questions. (MIT Sloan, March 2025)

The study tested how consumers interact with LLMs for financial guidance. The raw output from a naive prompt—"How should I invest $10,000?"—was mediocre at best, hallucinatory at worst. But when researchers structured the prompt to constrain the model's reasoning space, the quality of advice jumped dramatically.

This isn't magic. It's constraint injection, and it's a pattern every FDE and forward-deployed engineer already uses when shipping reliable LLM-powered features. The financial domain just makes the stakes—and the technique—crystal clear.

The Core Problem: LLMs Are Not Financial Oracles

An LLM is a next-token predictor trained on a vast corpus that includes everything from SEC filings to Reddit's r/wallstreetbets. Ask it an open-ended financial question, and it will happily generate plausible-sounding advice that blends CFA-level rigor with meme-stock energy. The model doesn't "know" which source to weight. It just completes the pattern.

Three failure modes dominate:

1. Source conflation: The model mixes legitimate financial planning principles with speculative forum chatter because the distance between those tokens in embedding space is smaller than you'd hope.
2. Temporal blindness: A model with a knowledge cutoff in early 2024 can't know the current Fed funds rate, yet it will confidently state a number.
3. Missing personalization: Without explicit constraints about risk tolerance, time horizon, tax bracket, and jurisdiction, the model defaults to generic advice that fits no one.

The MIT study effectively proved that these failure modes aren't inherent to the model—they're a function of the prompt's lack of structure. Engineers who build production LLM systems already know this intuitively. The prompt is the interface, and an underspecified interface produces undefined behavior.

Constraint Injection: An Engineer's Definition

Constraint injection is the practice of embedding hard boundaries, role definitions, and output schemas directly into the system prompt or user message to shrink the model's solution space to a safe, relevant subset.

Think of it like this: an unconstrained LLM is a function with a massive domain and an unpredictable range. Constraint injection turns it into a narrower, well-typed function with guardrails on both input interpretation and output format.

In the financial context, constraint injection means telling the model:

• Who it is: "You are a fiduciary financial planner bound by the CFP Board's Code of Ethics."
• What it knows: "You have access only to generally accepted financial planning principles as of your training cutoff. You must flag any assumption you're making."
• What it must ask: "Before providing any recommendation, you must elicit and confirm: time horizon, risk tolerance, tax filing status, and jurisdiction."
• What it must not do: "Do not recommend specific securities. Do not predict market movements. Do not provide tax or legal advice."
• How it outputs: "Structure your response as: (1) Assumptions Made, (2) Principles Applied, (3) Options with Trade-offs, (4) Questions You Should Ask a Human Advisor."

This isn't prompt engineering fluff. It's defensive programming for a non-deterministic system. You're reducing the attack surface of hallucination by explicitly removing the paths that lead to it.

The Five-Part Financial Prompt Scaffold

After tearing down dozens of financial prompts that shipped in production features, here's the scaffold that consistently produces grounded, useful output. It's opinionated and battle-tested:

[ROLE BOUNDARY]
You are {specific persona} operating under {explicit constraint set}.

[KNOWLEDGE SCOPE]
Your advice must be grounded in {domain}. You do not have access to real-time data.
Flag any assumption with [ASSUMPTION: ...].

[REQUIRED INPUTS]
Before generating advice, confirm you have: {list of mandatory parameters}.
If any are missing, ask for them. Do not proceed without them.

[OUTPUT SCHEMA]
Respond in exactly this structure:
1. Assumptions
2. Principles Applied
3. Options (with pros/cons)
4. Blind Spots / What to Verify with a Human

[FORBIDDEN]
Do not: {explicit list of verboten outputs}.

Each block serves a specific engineering purpose. The role boundary prevents the model from cosplaying as a Reddit commenter. The knowledge scope forces epistemic humility. Required inputs block the most common failure mode—generating advice without enough context. The output schema makes the response parseable and auditable. The forbidden list is your circuit breaker.

A/B Test: Vanilla Prompt vs. Constraint-Injected Prompt

Let's see what actually changes in the output. Here's a side-by-side comparison using a realistic scenario.

Scenario: A 34-year-old software engineer with $50,000 in cash, no debt except a mortgage at 3.2%, maxing out their 401(k) match but not contributing to an IRA, asking what to do with the cash.

The difference isn't subtle. The constraint-injected version behaves more like a decision-support tool and less like a stochastic parrot. As an engineer, you'd be comfortable shipping the second version in a user-facing feature. The first version is a liability.

How to Use This Today: A Practical Pipeline

You don't need a PhD in finance to build a useful financial reasoning tool. Here's a pipeline you can stand up in an afternoon using tools you likely already have access to.

Step 1: Build your constraint template. Create a YAML or JSON file that holds the scaffold above. Parameterize the role, knowledge scope, and forbidden list so you can swap them per use case. Version-control this like any other config.

Step 2: Wrap the LLM call in a prompt constructor. Before sending the user's query to the model, merge it with the constraint template. A simple Python function that does string interpolation works. For production, consider a library like LangChain's prompt templates or a simple Jinja2 template.

Step 3: Parse and validate the output. The structured output schema makes this straightforward. Write a lightweight parser that extracts each section. Run a validation pass on the assumptions: if the model assumes a 2024 tax bracket and you know it's 2025, flag it. This is the same pattern we use when we debug concurrent LLM agents—assume the output is buggy until proven otherwise.

Step 4: Add a human-in-the-loop checkpoint. For any financial advice feature, never ship the output directly to the user without a review step or, at minimum, a prominent disclaimer. The MIT paper is clear: AI advice is good as input to a human decision, not as a replacement for one.

Step 5: Log and iterate. Every time a user rejects or overrides the model's suggestion, log the prompt, the output, and the override. This is your training data for the next iteration of the constraint template. This feedback loop is exactly how FDEs turn messy customer problems into shipped prototypes.

A Balanced Take: Where This Fails

Constraint injection dramatically reduces hallucination, but it doesn't eliminate it. Here's what still breaks:

Temporal drift. You can tell the model to flag assumptions about current rates and limits, but it can't flag what it doesn't know it doesn't know. If a tax law changed in a way the model's training data didn't anticipate, the assumption flag won't fire. The only fix is grounding the model with real-time data via RAG or tool use.

Jurisdictional complexity. Financial advice is intensely local. A constraint that says "assume US tax resident" works for a US audience but completely fails for a UK user asking about ISAs or an Indian user navigating the new tax regime. Multi-jurisdiction support requires routing the prompt through a jurisdiction classifier first, then injecting the appropriate constraint set.

Over-constraining. There's a Goldilocks zone. Too many constraints, and the model becomes a useless parrot that just echoes your rules back at you. Too few, and you're back to hallucination. You'll need to tune this per model. GPT-4o and Claude 3.5 Sonnet handle heavy constraint loads gracefully; smaller models tend to collapse.

The advice-is-liability problem. Even with perfect constraints, shipping financial advice features carries regulatory risk. In the US, the SEC and CFPB have been clear that automated advice tools can trigger fiduciary obligations depending on how they're marketed. If you're building a financial feature, talk to your legal team before shipping. This isn't a technical constraint injection problem—it's a product scope problem. Understanding when to hand off a prototype to core engineering (and legal) is a core FDE skill, covered in our deep-dive on scaling yourself through the handoff process.

FAQ

Q: Is constraint injection just prompt engineering with a fancy name?

Yes and no. Prompt engineering is the broader practice of crafting inputs to get better outputs. Constraint injection is a specific, defensible sub-technique: embedding hard boundaries that shrink the model's solution space. The difference is intent and rigor. Prompt engineering says "please be helpful." Constraint injection says "operate within this sandbox, and if you step outside it, flag yourself."

Q: Can I use this pattern outside of finance?

Absolutely. Constraint injection works anywhere the cost of hallucination is high: medical advice, legal document review, code generation for safety-critical systems. The scaffold is domain-agnostic. Swap the role, knowledge scope, and forbidden list, and you have a template for any high-stakes LLM application. If you're building internal tools, the same principles apply to a local codebase Q&A tool where you need the model to stay grounded in your actual source files.

Q: Doesn't this just push the problem to the constraint designer?

Yes, and that's the point. The constraint designer is a human who can be held accountable. The LLM cannot. By making constraints explicit and auditable, you move the responsibility for correctness from a black-box model to a version-controlled configuration file that can be reviewed, tested, and rolled back. This is exactly the prototype-vs-product gap pattern: prototypes trust the model; products constrain it.

Q: What's the minimum viable constraint set for a financial prompt?

Three things: (1) a role that invokes a fiduciary or ethical standard, (2) a mandatory input checklist that blocks generation until key parameters are provided, and (3) a forbidden-output list that explicitly bans security recommendations, market predictions, and tax/legal advice. Start there, log your failure modes, and iterate.

Q: How do I test whether my constraints are working?

Build an eval set of 20-30 edge-case financial queries: questions with missing time horizons, cross-jurisdictional scenarios, requests for specific stock picks, queries that imply outdated tax law. Run them through your constrained prompt and score the outputs on: (a) did the model refuse or flag when it should have? (b) did it hallucinate a specific number or fact? (c) did it provide actionable, structured guidance? Automate this eval and run it every time you change the constraint template. CI/CD for prompts is not optional if you're shipping to users.