Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out(armature.tech)
290 points by screm 6 days ago | 145 comments
tl;dr: Armature ran 17k sessions (5.3k valid) across Claude Code, Codex, and Cursor to see which third-party tools coding agents pick when implementing features. The agents rely on different sources (Codex heavily web-searches, Claude Code uses priors and builds in-house ~2x more) and agree on the same tool only 42% of the time, with repo language heavily influencing choices—e.g., Resend wins for TypeScript email, Sendgrid for Python. Some categories are lopsided (Stripe wins 90%, Neon 66%, S3 45%), while heavily-mentioned brands like PayPal, LangChain, and Supabase rarely get picked.
HN Discussion:
  • Validates the study's premise that agents can be marketed to like humans for tool selection
  • Shares personal observations about Claude's limited web search behavior, corroborating findings
  • ~Points to competing/similar open-source tracking work as alternative
  • Critiques the product's high pricing despite finding the problem interesting
  • Broader philosophical concern about AI's golden age ending due to commercial lock-in