Breaking01 / AI
Databricks launches index comparing AI coding costs
Databricks reported different costs per task for models with equal success rates in its own coding evaluation.
Databricks announced the launch of its AI Index on 17 November, comparing enterprise AI use, coding performance and costs.
In the company's own evaluation, GPT-6 Luna and Claude Opus 5.5 both achieved an 89.1% task success rate. Luna's average cost was $0.05 per task. Opus 5.5 averaged $1.62 per task.
The methodology describes real development tasks drawn from Databricks' private codebase. Scoring uses hidden tests rather than an AI model's judgement. Cost estimates use providers' publicly listed prices.
Databricks said token prices alone were insufficient to assess coding costs. Repeated steps, long outputs and failed attempts add to the cost of reaching an answer. It urged businesses to evaluate models and the tools running them against their own workloads.
The company said it had considered switching internally from Opus 4.8 to Opus 5. It replayed records of its own workloads to compare cost and performance. It kept the existing model after deciding the upgrade did not justify the additional cost.
Databricks did not disclose details of the private code or hidden tests in its announcement. It said it would update the index weekly as it evaluated new models.