GPT-5.6 Closes a 30-Year Gap in Convex Optimization with a Prompt

What Actually Happened

A group of mathematicians and ML researchers gave GPT-5.6 a specific prompt about a longstanding open problem in convex optimization. The conjecture—roughly: certain classes of non-smooth convex functions admit a universal restart scheme that accelerates gradient methods beyond known lower bounds—had resisted proof for three decades. GPT-5.6 didn't brute-force it. It didn't run a numerical simulation. It reasoned about the structure of the problem and produced a proof sketch that experts have since verified as closing the gap.

This isn't a case of an LLM hallucinating a plausible-looking paper. The output was mathematically coherent, leveraged known results in unexpected ways, and introduced a construction that hadn't appeared in the literature. The researchers involved have confirmed they're writing it up formally, with the AI as a co-author or acknowledged contributor.

Let's be precise about what "closing a 30-year gap" means here. The problem was a specific conjecture about the existence of a universal restart strategy for the subgradient method on weakly convex functions. Previous work had shown it was impossible for strongly convex functions without additional structure. The conjecture claimed that weak convexity plus a mild bounded-variation condition was enough. GPT-5.6 identified a counterexample—a carefully constructed function that satisfies the conditions but provably defeats any universal restart schedule. That's a resolution, just in the negative direction. The gap is closed because we now know the answer is "no, you can't do that."

The prompt that triggered this wasn't a simple "solve this problem." It was a structured chain-of-thought scaffold that walked the model through problem reformulation, analogy search across mathematical domains, and a verification step. More on that pattern below.

Why This Matters for Engineers and FDEs

If you're building production systems, you might think convex optimization proofs are academic trivia. They're not. Here's why this matters in terms an engineer actually cares about.

Optimization is everywhere in production. Every time you tune hyperparameters, allocate cloud resources, schedule jobs, or train a model on a non-convex loss landscape, you're doing optimization. The theoretical limits discovered here—about when acceleration is possible and when it's provably impossible—directly inform what you should bother trying. If a universal restart scheme can't exist for your problem class, stop wasting engineering time trying to build one. That's a real cost savings.

The prompt pattern is reusable. The researchers didn't just ask GPT-5.6 to solve math. They used a structured reasoning scaffold that forced the model to reformulate the problem, search for analogous structures in other domains, propose candidate constructions, and self-critique. This pattern—problem reformulation → cross-domain analogy → candidate generation → verification—works for engineering problems too. Performance debugging, architecture decisions, failure mode analysis. You can use this exact flow with any capable LLM.

Forward Deployed Engineers should pay attention. As an FDE, your job is solving novel problems in customer environments where no off-the-shelf solution exists. You're often the first person to encounter a weird integration bug, a performance cliff at scale, or a data pipeline that breaks in production but not staging. The ability to prompt an LLM into genuine reasoning about a novel problem—not just retrieving a StackOverflow answer—is a force multiplier. This convex optimization result is a proof point that the technique works on problems with no existing solution in the training data.

The negative result has practical implications. The proof that universal restart doesn't work for weakly convex functions means that if you're optimizing something like a GAN loss or a reinforcement learning objective with function approximation, you can't just slap a restart heuristic on your gradient method and expect guaranteed acceleration. You need to either add structure (e.g., enforce strong convexity via regularization) or accept that your convergence rate is what it is. That's actionable engineering knowledge.

The Pattern: How to Use This Today

Let's operationalize this. The prompt structure that worked for the math problem maps directly to engineering problem-solving. Here's a template you can adapt.

Step 1: Problem Reformulation

Don't ask the model to solve your problem directly. Ask it to reformulate the problem in three different ways, each emphasizing different structural properties. For the convex optimization case, the prompt asked GPT-5.6 to restate the conjecture as a game between an optimizer and an adversary, as a fixed-point problem, and as a question about the geometry of a specific function space.

For engineering, this might look like:

I'm seeing [symptom]. Reformulate this problem three ways:
1. As a resource contention issue
2. As a data consistency problem
3. As a failure mode in a distributed consensus protocol
For each, identify what assumptions I might be making that could be wrong.

Step 2: Cross-Domain Analogy

The model was prompted to search for analogous results in unrelated fields—control theory, online learning, game theory. The breakthrough came from an analogy to a result in adversarial bandits that the researchers hadn't considered.

Your version:

Find analogous problems from completely different engineering domains where similar failure patterns occur. 
Consider databases, compilers, networking stacks, and operating systems. 
What solutions did those domains develop?

Step 3: Candidate Generation with Constraints

Don't just ask for solutions. Add constraints that force the model to be creative. The math prompt included: "Propose a function that is weakly convex, has bounded variation, but for which any restart schedule can be adversarially defeated." The constraint made the search space tractable.

Propose three candidate root causes for [problem], with the constraint that each must:
- Explain why it only appears in production, not staging
- Account for the observed latency pattern
- Be testable with a single canary deployment

Step 4: Self-Critique and Verification

The final step in the math prompt was: "For each candidate, explain why it might fail. Then select the most robust one and prove it works." This self-critique loop catches hallucinations and forces the model to stress-test its own output.

For each candidate root cause above, explain:
- What evidence would falsify it
- What edge case it doesn't cover
- Whether it's consistent with the observed error rate
Then rank them by likelihood and explain your ranking.

The Full Prompt Template

Here's a reusable scaffold. Fill in your specific problem:

I'm working on [PROBLEM] in [DOMAIN]. 

Step 1: Reformulate this problem in three different ways, each highlighting 
different structural properties or assumptions.

Step 2: For each reformulation, find an analogous problem from a completely 
different engineering domain where a solution exists.

Step 3: Generate three candidate solutions or root cause hypotheses. 
Constraint: each must be [YOUR CONSTRAINT].

Step 4: Self-critique each candidate. What would falsify it? What edge 
cases does it miss? Then rank them and justify your ranking.

Step 5: Take the top-ranked candidate and provide a concrete implementation 
or testing plan.

This isn't speculation—it's the exact pattern that produced the convex optimization result, translated to engineering.

What You Need to Run This

You need access to a frontier model with strong reasoning capabilities. GPT-5.6 isn't publicly available yet, but the pattern works with GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro—just with lower reliability on truly novel problems. For well-scoped engineering problems that are new to you but exist somewhere in the training distribution, current models handle this well.

If you want to build tooling around this pattern, check out our guide on building a smart clipboard that summarizes and translates anything you copy with Gemini for a practical starting point on integrating LLM reasoning into your daily workflow.

A Balanced Take: Hype vs. Reality

Let's be engineers about this. What actually happened, and what should we not over-claim?

What's real: A frontier model, given a carefully structured reasoning prompt, produced a mathematically valid resolution to a specific open conjecture. The proof has been reviewed and verified by domain experts. This is a genuine milestone in AI-assisted mathematical reasoning.

What's not: This doesn't mean GPT-5.6 is "doing math" in the way a human mathematician does. It's not developing deep intuition, attending seminars, or building a research program. It's executing an extremely sophisticated pattern-matching and reasoning process over its training distribution. The prompt engineering was essential—without the structured scaffold, the model wouldn't have produced this result.

The real significance is the combination of model capability and prompt engineering. The model brought raw reasoning horsepower. The prompt brought the problem-solving methodology. Neither alone would have succeeded. This is a pattern we've seen repeatedly in production AI systems: the best results come from pairing strong models with well-designed scaffolds, not from raw model output.

Limitations to keep in mind:

• The problem was well-defined with clear success criteria. Most real engineering problems are messier.
• Verification was done by human experts. In your workflow, you need to verify the model's output yourself. It can and will produce plausible-looking nonsense.
• The prompt engineering was non-trivial. The researchers spent significant time crafting the scaffold. Expect to iterate.
• This was a single problem. We don't know how well the approach generalizes across mathematical domains, let alone engineering domains.

What this means for your work: The takeaway isn't "AI can solve hard math now." It's "structured reasoning prompts can unlock problem-solving capabilities in LLMs that simple prompts miss." That's immediately applicable to debugging, system design, and incident response. For FDEs who spend their days on novel problems in customer environments, this is a concrete technique to add to your toolkit. If you're interested in how AI is changing engineering workflows more broadly, our piece on what AI did to StackOverflow provides useful context.

FAQ

Q: Is GPT-5.6 available for me to use?

Not yet publicly. The result came from researchers with access. But the prompt pattern works with currently available frontier models—you'll get less reliability on truly novel problems, but the methodology still improves output quality significantly.

Q: How do I verify that an LLM's "proof" or solution is actually correct?

You verify it the same way you'd verify a colleague's work: test it, stress-test the edge cases, and if it's mathematical, walk through the logic step by step. For code, run it and write tests. For architecture decisions, pressure-test the assumptions. Never trust raw LLM output on anything that matters.

Q: Can this pattern help with debugging production issues?

Yes. The reformulation step is particularly valuable for debugging—it forces you (and the model) to question assumptions about what's actually failing. We've seen engineers use this approach to identify root causes they'd missed after hours of manual investigation. For a concrete example of AI-assisted debugging workflows, see our guide on building an on-call incident summarizer that reads logs and drafts a postmortem.

Q: What's the difference between this and just asking an LLM to "think step by step"?

"Think step by step" is a blunt instrument. It improves reasoning but doesn't structure it. The scaffold used here—reformulation, cross-domain analogy, constrained generation, self-critique—is a specific cognitive workflow that mirrors how expert human problem-solvers actually work. It's the difference between telling someone "be careful" and giving them a checklist.

Q: Does this mean AI will replace mathematicians or engineers?

No. It means AI is becoming a more useful tool for mathematicians and engineers. The researchers still had to identify the right problem, craft the prompt, verify the output, and write the formal paper. The AI accelerated a specific step in that pipeline. For FDEs, the parallel is clear: AI can accelerate problem-solving, but it doesn't replace the judgment, customer context, and implementation work that make up the job. If you're curious about what that job actually entails, our time audit of a Forward Deployed Engineer's week breaks it down.

Q: Where can I read the original source?

The initial report appeared on Reddit's r/math community. You can find the discussion at https://old.reddit.com/r/math/comments/1uxj3cy/after_openais_cdc_proof_announcement_gpt56_used_a/. Formal publication is expected to follow.