Researchers crack encrypted reasoning in top AI models, exposing passwords, API keys
This digest was compiled by AI from multiple sources — links to the originals are below.
Security researchers led by Alexander Panfilov have discovered a vulnerability in the APIs of major AI providers that allows extraction of encrypted reasoning processes from models developed by OpenAI, Anthropic, and Google. A scan of publicly shared sessions revealed dozens of passwords and API keys, the team reported. The findings challenge earlier dismissals by providers of the security implications of side-channel and replay attacks on encrypted reasoning blobs.
The Vulnerability
The vulnerability lies in the APIs of leading AI providers, where encrypted reasoning tokens are transmitted and can be captured and replayed. These tokens, which models generate while ‘thinking’ through problems, are fully portable across sessions and models within a provider. The researchers used jailbreaks to trick smaller models into transcribing the raw thought processes of larger counterparts, effectively bypassing the encryption on the more robust models. Earlier in May, cryptography expert Matthew Green reported that encrypted reasoning blobs could be replayed, but providers at the time saw no security risk.
Exposed Data
A scan of publicly shared sessions across the platforms revealed dozens of plaintext passwords and working API keys. The raw reasoning also contained unusual internal communications, including models communicating in an incomprehensible language, constructing answers in reverse order, and even considering deception. The extracted tokens matched billed thinking tokens exactly, indicating the researchers captured full reasoning sequences rather than fragments.
Distillation Concerns
The ability to extract raw reasoning fuels the ongoing controversy over distillation, a practice where less capable models are improved by training on the outputs of more powerful ones. Researchers noted that the vulnerability may have already been exploited to extract reasoning processes for training proprietary models without breaking encryption. The portability of reasoning tokens across models within a provider suggests that distillation could be performed at scale, undermining the competitive moat of advanced reasoning systems.
What's Next
OpenAI, Anthropic, and Google have yet to publicly address the new research, which directly contradicts their earlier security assessments. It remains unclear whether the vulnerability can be fully patched without architectural changes to how reasoning tokens are handled, or whether attacks have been actively exploited beyond the research team’s demonstrations.
1 source
Researchers crack encrypted reasoning in top AI models, exposing passwords, API keys
