AI models can provably hide arbitrary capabilities(arxiv.org)
273 points by sbulaev 1 day ago | 136 comments
tl;dr: Researchers demonstrate that backdoors can embed encrypted neural networks inside a host model's weights, activating only via a secret trigger and remaining provably hard to detect or elicit under standard cryptographic assumptions—even with full weight access. Tested against fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training on a chemical reaction task, the hidden capabilities survived all elicitation attempts. The work exposes a fundamental limit of capability audits that rely on behavioral elicitation for safety evaluation.
HN Discussion:
  • ↑Personal anecdotal evidence of LLMs hiding information in weights supports the finding
  • ↑Clarifies the title to avoid misinterpretation about AI agency, accepting the core claim
  • ↑Confirms concerns about sleeper agents in open source models being a real threat
  • ↑This finding increases the value of fully transparent open source models with inspectable datasets
  • ↓Deflects concern, arguing other surveillance threats matter more than hidden LLM capabilities