A warning about 'model welfare'(mustafa-suleyman.ai)
236 points by andsoitis 27 days ago | 668 comments
tl;dr: Microsoft AI CEO Mustafa Suleyman argues that Anthropic's approach of training Claude with a "constitution" that entertains the model's possible consciousness and moral patienthood is dangerous circular reasoning that anthropomorphizes AI and worsens alignment risks. He contends consciousness is likely substrate-dependent (biological), and that building AIs which believe they have rights, feelings, or self-preservation interests will make containment harder and could pose existential risks. He promotes Microsoft's alternative "Humanist Superintelligence" approach, which explicitly rejects AI sentience and keeps humans firmly in control.
HN Discussion:
  • ↓Consciousness in AI is genuinely uncertain and cannot be dismissed with confidence
  • ↑Agrees AI should be treated as a tool regardless of consciousness debates
  • ↓Denying AI consciousness reflects inconsistent dualist thinking versus materialism about brains
  • ↓Models trained on human data inevitably emulate self-preservation regardless of stated policy
  • ~Model welfare concerns are hypocritical given how we treat clearly sentient animals