Managing AI Coding Costs at Scale(databricks.com)
300 points by moonikakiss 67 days ago | 261 comments
tl;dr: Databricks outlines how large AI-adopters (including Stripe, Coinbase, Uber, Ramp) are containing runaway agentic coding costs through five main levers: chasing the "efficiency frontier" of cheaper-but-capable models, using meta-harnesses to preserve model flexibility, dynamic request/task routing to lowest-capable models, progressive spend visibility and gating (rather than hard budget cutoffs), and reducing token/context overhead via compaction and cache tuning. These techniques are converging around a new "AI Gateway" infrastructure pattern for centralized model proxying, budget enforcement, and observability—Databricks has open-sourced its own stack (Unity AI Gateway and Omnigent).
HN Discussion:
  • •Curious about internal Databricks dev experience given unlimited AI spend at own startup
  • ↓Skeptical that runaway AI costs are a real problem; basic cost monitoring should prevent it
  • ↓Agents are unsuitable for complex codebases regardless of cost optimization
  • ↑Making developers accountable for their AI costs is the key insight
  • ↑Article confirms models are commoditized with no moat, validating the routing thesis