Why We Route Planning to Haiku and Execution to Sonnet — The Token Routing Layer That Cut Our Bill 70%

TL;DR: 40% of agent tokens are planning (file selection, refactor strategy, test approach) — routing these to Haiku/4o-mini at $0.25-0.60/M vs. Opus/Sonnet at $15-30/M saves 60-70% with no quality loss. Full comparison →

The Context

Three engineers, ~200 agent runs/month across refactors, test generation, and feature scaffolding. Before routing: blended cost ~$14/M tokens. After: ~$4.2/M. The router is a 50-line YAML config + a thin middleware that classifies each agent turn as “planning” or “execution” and selects the model. LangSmith enforces per-seat daily caps in USD (not tokens) so model switches don’t silently blow the budget.

What We Tested

Tool	Use Case	Verdict	Why
Custom YAML router + LangSmith	Planning → Haiku/4o-mini, Execution → Sonnet/Opus	✅	70% savings, model-agnostic, budget-aware in USD
LiteLLM router	Multi-model gateway with fallbacks	⚠️	Good routing, weak budget enforcement, no agent-turn classification
OpenRouter	Model marketplace with routing rules	⚠️	Broad model access, but adds latency and another vendor
Cursor (native)	No routing — single model per run	❌	All tokens at premium rate
LangChain RunnableRouter	Code-level routing in LangGraph	⚠️	Works but couples routing logic to graph definition

The Pivot Point

Analyzing a month of agent traces: 1.3M planning tokens at $30/M (Opus) = $39,000. Same planning at $0.25/M (Haiku) = $325. The execution tokens (1.9M at $15/M Sonnet) stayed at $28,500. The Planning-to-Execution ratio was ~40:60 — consistent across refactors, test-gen, and scaffolding. The waste was structural, not incidental.

What We Use Now

token_router.yaml:

classifier:
  planning_keywords: ["plan", "strategy", "approach", "select", "identify", "analyze", "design"]
  execution_keywords: ["write", "edit", "refactor", "implement", "generate", "fix", "create"]
models:
  planning: "claude-3-haiku-20240307"      # $0.25/M in, $1.25/M out
  execution: "claude-3-5-sonnet-20241022"   # $3/M in, $15/M out
  fallback: "gpt-4o-mini"                   # $0.15/M in, $0.60/M out
budgets_usd:
  daily_per_seat: 15.00
  task_max: 3.00
  alert_at: 0.8

Middleware intercepts each agent turn, classifies via keyword + heuristic (first 3 turns = planning), routes accordingly. LangSmith webhook pauses at 80% of daily USD cap.

When You’d Choose Differently

If your agents do mostly execution (single-file edits, bug fixes), the routing overhead isn’t worth it — just use Sonnet directly. If planning requires 200K+ context (entire monorepo + docs), Haiku’s window may truncate; consider 4o-mini (128K) or accept Sonnet for planning. And if you’re on a single-model contract (e.g., enterprise Anthropic only), the router can’t help.

Tool Crucible Rating

Dimension	Rating
Overall	★★★★★
Ease of Setup	★★★★☆
Value (Cost Savings)	★★★★★
Support/Documentation	★★★☆☆ (Custom)

---

This is part of our Token Routing Strategies evaluation series. See full comparison: Token Routing Strategies 2026