Stealing Reasoning Traces from Proprietary LLM APIs

	Stealing Reasoning Traces from Proprietary LLM APIs(stolen-thoughts.com)
	684 points by quantumgarbage 10 days ago \| 300 comments
	tl;dr: Researchers found that encrypted chain-of-thought blocks returned by Anthropic, OpenAI, and Google APIs are portable across sessions and models: replaying a frontier model's encrypted reasoning into a weaker, jailbroken sibling model causes it to transcribe the hidden reasoning verbatim, bypassing anti-distillation safeguards. Applied to 6,708 public agent trajectories, the technique reconstructed 315,320 reasoning blocks and extracted 704 real secrets, including API keys, passwords, and PII—64 of which appeared only in the hidden reasoning and never in the visible session.
	HN Discussion: ↑Curiosity and validation that cross-model replay works, suggesting inadequate safeguards ↑Explains/summarizes the technique, confirms concerns about reasoning trace fidelity ↓Objects to the framing of 'stealing' since outputs aren't really owned or should be transparent ↓Criticizes the paper as overblown—content is trivial and lacks scientific novelty ↑Points out even simpler bypasses exist, reinforcing that safeguards are shaky