Claude Finds Cryptographic Flaws: How LLMs Can Augment Security Audits

The Experiment: What Anthropic Actually Did

Anthropic’s research team set out to answer a sharp question: can a large language model spot cryptographic flaws that haven’t been explicitly described in its training data? Not just regurgitate known CVEs or flag md5() calls, but genuinely reason about novel weaknesses in protocol designs.

The short answer is yes—with the right scaffolding.

The team built an evaluation harness around Claude that fed it cryptographic protocol descriptions and asked it to identify vulnerabilities. These weren’t textbook examples. They included subtle flaws in zero-knowledge proof constructions, commitment schemes, and multi-party computation primitives. In several cases, Claude surfaced weaknesses that matched or anticipated findings from human cryptographers. The kicker: some of these flaws were genuinely novel, not simply memorized from public vulnerability databases.

Here’s the architecture of what they built:

The critical insight: Claude wasn’t operating in isolation. It was paired with formal verification tools—constraint solvers and counterexample generators—that could take Claude’s natural-language reasoning and ground it in mathematical proof. The LLM acted as an intelligent proposer of hypotheses, and the formal tools acted as the verifier. This is the hybrid approach that makes the results credible, not some magical AGI moment.

Cryptographic Reasoning vs. Pattern Matching

Engineers who’ve used LLMs for code review know the baseline: they’re decent at spotting strcpy into fixed buffers or missing authentication checks. That’s pattern matching against a massive corpus of insecure code examples. Cryptographic reasoning is a different beast entirely.

A cryptographic flaw rarely looks like a syntax error. It looks like a protocol step that’s mathematically sound in isolation but breaks under composition. It looks like an assumption about the adversary model that doesn’t hold when two primitives interact. These are logical gaps, not code smells.

What Anthropic demonstrated is that Claude can operate at this logical level. When given a protocol description—say, a commitment scheme where the verifier sends a challenge after seeing the commitment—Claude could reason about whether the order of operations leaks information or enables a selective-failure attack. This isn’t "the code uses SHA-1." It’s "the protocol allows a malicious prover to equivocate because the binding property fails under these specific conditions."

This matters because traditional static analysis tools are blind to these flaws. A linter won’t catch a subtle missing zero-knowledge property. A SAST tool won’t flag that your Fiat-Shamir transform is missing the statement in the hash. Claude, prompted correctly, can reason about these properties in plain English—and then formal tools can check its work.

The Audit Workflow: A New Loop for Engineers

The practical workflow that emerges from this research isn’t "replace your security auditor with an LLM." It’s a three-phase loop that any engineering team can instrument today:

Phase 1: Specification Extraction. Feed the LLM your protocol documentation, code comments, and implementation. Ask it to extract the formal security properties the protocol claims to provide: confidentiality, integrity, authentication, non-repudiation, forward secrecy, etc. This step alone catches documentation gaps where the team never explicitly stated what they’re trying to achieve.

Phase 2: Adversarial Reasoning. For each security property, prompt the LLM to act as an adversary. "Given this protocol description, propose an attack that violates the binding property. Consider composition with other protocol steps. Think step by step about what information each party has at each stage." The LLM generates candidate attack traces.

Phase 3: Formal Verification. Take the candidate attacks and encode them into a constraint solver or a tool like Tamarin Prover. If the solver finds a satisfying assignment, you have a machine-checkable counterexample. If it doesn’t, the LLM’s hypothesis was either wrong or too imprecise to formalize—both useful signals.

This loop is powerful because it addresses the fundamental bottleneck in security audits: human cryptographers are scarce and expensive. An LLM can generate dozens of attack hypotheses per minute. Even if 90% are wrong, the 10% that survive formal verification are gold—and they cost pennies to generate.

Why This Matters for Forward Deployed Engineers

If you’re an FDE working on customer integrations, you’re regularly touching authentication flows, API key management, webhook signature verification, and custom encryption wrappers. You’re not designing novel zero-knowledge proofs, but you’re absolutely making cryptographic design decisions—often under pressure, in a customer’s environment, with limited review.

This research matters to you for two reasons.

First, it’s a force multiplier for the security review you should be doing but probably aren’t. When a customer asks you to implement a custom token exchange between their legacy system and your platform, you can run the three-phase loop above on your design doc before writing a single line of code. It takes 30 minutes and might surface a flaw that would otherwise become a post-mortem.

Second, it changes how you communicate risk to customers. Instead of saying "this feels insecure," you can show a machine-checked counterexample generated from your LLM-assisted audit. That’s the kind of engineering rigor that builds trust with technical stakeholders, which we’ve explored in depth in our piece on building trust with non-technical stakeholders as an FDE under pressure.

This is exactly the kind of high-leverage skill that separates senior FDEs from junior ones. Not just prompting an LLM, but knowing when and how to pair it with formal tools to produce verifiable results. We covered the broader skill set in the highest-leverage skills for an FDE in the AI era beyond prompting.

How to Replicate This on Your Own Codebase

You don’t need Anthropic’s research infrastructure to get value from this approach. Here’s a concrete setup you can run this afternoon:

Step 1: Choose your target. Pick a small, self-contained cryptographic component. A JWT validation function. A webhook signature verifier. An API key derivation scheme. Not your entire auth stack—something you can fully describe in a page of text.

Step 2: Write the specification prompt. Feed Claude (or your preferred frontier model) something like:

You are a cryptographic protocol auditor. I will provide a protocol description. 
Your task is to:
1. Extract the explicit and implicit security properties this protocol claims to provide.
2. For each property, propose an attack that would violate it. Think adversarially.
3. If you find no attack, state why the property holds based on standard cryptographic assumptions.

Protocol description:
[Your protocol here, including message formats, order of operations, 
and assumptions about the adversary model]

Step 3: Review the output critically. The LLM will generate attacks. Some will be nonsense—attacks that assume the adversary can break AES, or that ignore an obvious mitigation you just forgot to write down. Filter those out. Focus on attacks that make you uncomfortable because they’re plausible given the protocol as described.

Step 4: Formalize the best candidate. Take the most plausible attack and encode it as a property violation. If you’re working with a protocol that has message sequences, Tamarin Prover is the standard tool. If you’re checking a smaller property like "this encryption is IND-CPA," you can use a simpler approach: write a Python script that simulates the attack and see if it actually works.

Step 5: Close the loop. If the attack works, fix the protocol. If it doesn’t, update your specification to include the mitigation that blocked it. Either way, you’ve improved your security posture with evidence, not intuition.

For teams that want to operationalize this, the pattern looks a lot like what we described in our Codex Security piece on OpenAI’s static analysis rules for AI-generated code audits. The same principle applies: LLMs propose, formal tools verify, humans judge.

The Sober Reality: Limitations and Noise

Let’s be clear about what this isn’t. It’s not a replacement for professional security audits. It’s not a guarantee that your protocol is secure. And it’s absolutely not a license to skip threat modeling because "Claude said it’s fine."

The false positive rate is high. For every genuine vulnerability Claude surfaces, it’ll generate ten attacks that don’t actually work—either because they violate a cryptographic assumption you didn’t state, or because they rely on an adversary capability that’s outside your threat model, or because they’re just hallucinated nonsense that sounds plausible.

There’s also a dangerous failure mode: the LLM might miss a real vulnerability entirely. Cryptographic flaws are precisely the kind of edge case where a model’s training distribution might not help. If your protocol has a novel weakness that doesn’t resemble anything in the training data, Claude might confidently assert it’s secure. You won’t know what you don’t know.

The research paper is honest about this. The success cases are impressive, but they’re cherry-picked. The team had human cryptographers validating Claude’s output and discarding the noise. In production, you’re the human cryptographer—and if you’re not one, you’re operating with a significant blind spot.

This is why the hybrid approach matters so much. The LLM is a hypothesis generator, not an oracle. The formal verification step is what separates signal from noise. Skip that step, and you’re just doing security theater with an expensive autocomplete.

For a deeper look at how AI-generated code stacks up against formally verified specifications—and the sometimes embarrassing gap between the two—check out our analysis of verified 3D CSG in 93 lines of spec vs. 1000 lines of AI-generated code. The lesson applies here too: formal verification catches what LLMs miss.

FAQ: Capabilities, Safety, and Practical Use

Q: Does this mean Claude is "good at cryptography" in general?

No. Claude can reason about protocol logic when given a clean, well-structured description. It can’t break real-world cryptographic primitives, and it will happily generate plausible-sounding but mathematically wrong cryptographic constructions if you ask it to design one from scratch. Use it as an auditor, not a designer.

Q: Should I run this on my production auth stack?

Yes, with the right expectations. It’s a cheap, fast way to generate hypotheses about potential weaknesses. Treat every finding as a lead to investigate, not a confirmed vulnerability. If something looks plausible, escalate it to a proper security review or formal verification.

Q: What if the LLM finds a vulnerability and I don’t understand it?

That’s a signal that you need to level up your cryptographic understanding, not that you should blindly apply a fix. A vulnerability you don’t understand is a vulnerability you can’t verify is actually fixed. This is where the FDE skill of knowing your limits and pulling in experts becomes critical.

Q: Can I use this approach for non-cryptographic security flaws?

Absolutely. The same three-phase loop—specification extraction, adversarial reasoning, formal verification—works for authorization logic, API access control, race conditions in distributed systems, and any other domain where you can state a security property and reason about violations. The cryptographic case is just the hardest and most interesting instance.

Q: Is Anthropic’s research publicly reproducible?

Anthropic has published the methodology and results. You can read the full paper at anthropic.com/research/discovering-cryptographic-weaknesses. The specific evaluation harness isn’t open-sourced, but the approach is described in enough detail to replicate with off-the-shelf tools and a frontier LLM.

Q: How does this change security engineering in the next 2-3 years?

It accelerates the trend toward continuous, automated security review. We’re moving from "audit once a year" to "every PR gets an LLM-powered adversarial review." The engineers who learn to run this loop effectively—and who understand its limitations—will ship more secure code, faster. The ones who ignore it or trust it blindly will create incidents. Same as every other tool.