LLM model routing: Uber, AT&T, and Ramp cut AI costs 30–56%

Ramp opened its internal model router to the public in August, after three years in production. The numbers behind it are some of the most credible routing data anyone has published: 100+ AI features, 2.75 trillion tokens a month through one endpoint. It cut Ramp's own LLM costs by about 30%.

You’ll see cost savings of 85% thrown around when companies talk about LLM model routing. Here’s where that number comes from and a closer look at how Ramp, Uber, and AT&T are all cutting AI costs by routing the right workflows to the right models.

How LLM model routing saves you money

A router sits between your application and your model providers. It reads each request, estimates how hard it is, and sends it to the cheapest model that can clear your quality bar.

The number that made the technique famous comes from RouteLLM, out of UC Berkeley and LMSYS: roughly 85% cost reduction on MT-Bench, ~45% on MMLU, and ~35% on GSM8K, while retaining about 95% of GPT-4's performance. Those are results for routing between GPT-4 and Mixtral-8x7B on single-turn benchmark questions, and they don’t transfer directly to your workflows and model choices.

Amazon Bedrock Intelligent Prompt Routing headlines up to 30%. Databricks' AI Gateway Smart Router reports more than 30%. FrugalGPT's cascade approach reported 50–98% depending on the dataset.

What Uber, AT&T, and Ramp saved in production

The Pragmatic Engineer pulled the thread together: Uber, Pinterest, Stripe, Coinbase, Ramp, and AT&T have all cut AI bills by moving simpler workloads off frontier models, and the savings cluster around 50%.

Uber blew through its 2026 AI budget in four months, and its COO told Fortune in May the spend was getting hard to justify. By August, Uber had cut cost per AI request by 34% and cost per session by 52%, and total spend has been flat since April while weekly active users grew 7x. The levers: weekly benchmarking of frontier and open-weight models on real work, cheaper default models for subagents, medium reasoning effort by default, and aggressive prompt caching.

AT&T cut the cost of coding and other advanced AI tasks by 56% while measuring a 2% drop in output quality. The mechanism was LiteLLM routing: frontier models still handle code generation, and open models like Nemotron, Llama, and Gemma handle summaries and lighter work.

Databricks, which interviewed engineers at Stripe, Coinbase, Uber, and Ramp, put it plainly: "The single greatest cost lever is moving coding spend to more efficient models as they are released."

Every one of these companies still sends its hardest work to a frontier model. They’ve just figured out how to route defined tasks to cheaper models whenever possible.

Why model routing can break on agentic workloads

Single-turn benchmarks don’t capture the full cost of agentic workflows. Almost nothing an engineering team runs looks like that. AgentRouter (ICML 2026) applied RouteLLM and FrugalGPT step by step across agent workflows and got 31% and 44% cost reductions. A router that understands the whole trajectory hit 72% with under 3% degradation in task completion.

Most of the gap between 85% and 30% comes down to prompt caching and switching models at the wrong time.

Prompt caches are model-specific. A cached prefix on one model is worthless on another. When a router switches models in the middle of an agent session, the accumulated context gets written to cache again from scratch, turning a read priced at 10% of base into a write priced at 125% of base, on the full context.

Cursor's team ran into this while tuning its agent harness: a switch means a cache miss and a slower, more expensive first turn. Summarizing the conversation to hand off between models lost details on complex tasks. Their recommendation is to stay on one model for the session and use subagents, which start with fresh context, when a subtask can run on a cheaper model.

Route once per workflow and pin the model

The fix is model affinity. Route once, at the start of a defined unit of work, then pin that model for every call inside it. When the unit of work ends, the pin releases and the next one gets routed fresh.

Easy tasks still land on cheap models and hard tasks still land on expensive ones. The savings move up a level instead of getting eaten by cache resets.

Uber's subagent default is this pattern in production: the main session stays on one model, bounded subtasks get a cheaper one, and no cache gets invalidated mid-work.

Two routing systems now do it explicitly. vLLM's Session-Aware Agentic Routing adds session memory, hard locks around tool loops, and cache-aware switch pricing, and cut model switches by 79% across 21,600 deterministic turns.

Factory Router routes per coding-agent session, starting at the cheapest model class likely to handle the work and escalating only if the session starts failing. It holds 99% of Claude Opus 4.7's pass rate on Terminal-Bench 2 at 20% lower cost per session, and Factory reports 43% aggregate savings in production against the same workload priced at top-tier rates.

Both depend on the same prerequisite. You can't pin a model to a stage until you've defined the stages – which steps are classification, which are retrieval, which are generation, and which need frontier reasoning.

Where model routing stops saving money

Map the stages, pin the model, and there's still an output quality ceiling. Factory published its full cost-quality curve, including the part where it breaks.

Turn their router hard toward cheap models and on Terminal-Bench 2 you pay 44% less and finish 19% fewer tasks. On Legacy-Bench, the harder suite, you pay 70% less and finish half as many. Whether that tradeoff saves money depends on the cost of retries, escalations, and unfinished work. 

Cheap routing looks like a win on your billing dashboard within hours. The cost lands days later in support tickets, and never on a cost report. That's why routing needs an eval gate: run 50–500 real cases before any routing change ships, and block anything that drops quality below your bar.

How to measure routing savings on your own LLM workloads

No published percentage is credible for your stack and workloads. The only number that matters comes from your own production systems, and getting it takes about a month to get an idea.

• Tag every call by team, workflow, model, and prompt version. Start with visibility before you try to control anything.
• Measure cost per successful outcome, not cost per token. Retries and escalations belong in the numerator.
• Pick one high-volume, well-understood workflow and baseline it against your current API spend.
• Define the quality bar before you route, then gate every routing change against it.
• Route at the workflow boundary and pin the model through the session.

That experiment tells you more than any vendor benchmark, because it's running your prompts against your models.

Model routing works. It just doesn't work in production the way the benchmarks describe. The teams getting real savings mapped their workflows well enough to know which stage deserves a frontier model. The router can only be as smart as your map of the work.

Codingscape builds this in production – model serving, eval gates, and open and closed models routed by task. We won't quote you a savings percentage before we've measured your workloads, because nobody can. If your token bill is growing faster than outcomes you can measure, let’s talk about LLM model routing.