iCreatAI News
Skip to content
Trending storyWatching

Airbnb runs coding agents with cost per task

1 reports1 sources2 days agoUpdated

Understand this story

AI overview

Airbnb shared how it operates its coding agent AirChat on Amazon Bedrock: it routes model requests through a self-built LLM gateway and uses "cost per task" as the basis for model selection. Cost per task refers to the list-price cost of all tokens an agent consumes to complete one task. Airbnb evaluates each model's score and cost on its own tasks simultaneously, then combines CloudWatch metrics on latency, throttling, and token usage to calibrate the default model and the reasoning-effort cap.

AI overview based on source reports · Updated 2 days ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 19
  1. AWS Machine Learning Blog
    How Airbnb uses cost per task to operate coding agents on Amazon Bedrock

    Airbnb runs its AirChat coding agents through an LLM gateway on Amazon Bedrock and evaluates each model on its own engineering tasks, recording a score and a cost per task for every model and reasoning effort to set defaults and effort ceilings. After rollout, CloudWatch runtime metrics and gateway usage records are compared with eval results; one model that scored well in evals drove several times higher weekly spend per user in production, and moving its heavy users to a newer model cut that group's spend on its main model to about a seventh. The post also covers latency and throttling diagnosis, quota reservation and output-token burndown, and attributing spend by team, application, or feature.

Activity for this story

Current comparable activity
3
Peak comparable activity
10Nov 19 07:00
Change in the past 24 hours
-50%

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.