Blended across the normalized routing mix.
Free LLM inference cost calculator
Compare inference architectures before you benchmark them.
Project the cost, latency, and quality of three model configurations, including routing, retries, provider prompt caching, and full-response caching. Then export the assumptions for a representative local test.
04 / Project the frontier
Plan a benchmark from measured workload inputs.
Replace the illustrative inputs with a measured workload profile and provider prices. The planner identifies configurations that appear dominated and makes the assumptions portable.
- Three configurations
- Two cache types
- Browser-only calculations
Projection workspace
Illustrative defaults. Replace prices and performance assumptions with your own evidence.
Shared workload
Traffic + routing mix
Configuration A
Tokens per model call
Provider price per 1M tokens
Full-response cache
A hit serves a stored answer and avoids the model call entirely.
Configuration B
Tokens per model call
Provider price per 1M tokens
Full-response cache
A hit serves a stored answer and avoids the model call entirely.
Configuration C
Tokens per model call
Provider price per 1M tokens
Full-response cache
A hit serves a stored answer and avoids the model call entirely.
Projection assumptions
The planner routes each request once and normalizes traffic shares to 100%. A response-cache hit bypasses the model path. Retry assumptions apply to cache misses: 25% means 1.25 model attempts per miss; 150% means 2.5. The planner adds retry latency in sequence. Input includes prompt-cached tokens. Enter all billable output, including reasoning tokens that the provider bills as output. Cost covers provider usage in USD and excludes implementation, infrastructure, and labor.
Projected blended portfolio
Use this projection to plan the benchmark.
$11,724projected provider cost / month
Derived from entered path, cache, and retry assumptions.
You supply the score; this page calculates its weighted average.
After response-cache hits and retry assumptions.
Positive means lower cost versus the same mix with no caching.
Projected configuration map
Cost × latency, with quality shown by point size
Lower-left is less expensive and faster. A ring marks a non-dominated configuration.
| Configuration | Status | Normalized traffic | Cost / 1K | Monthly cost | Average latency | Quality |
|---|---|---|---|---|---|---|
| Frontier model | Frontier | 40% | $23.40 | $9,360 | 988 ms | 92 / 100 |
| Efficient model | Frontier | 45% | $4.5873 | $2,064 | 509 ms | 86 / 100 |
| Specialized endpoint | Frontier | 15% | $1.9958 | $299 | 297 ms | 81 / 100 |
Projection boundary
You supply the prices and quality scores. The planner calculates weighted average latency. Use a representative local benchmark to test rate limits, tail latency, failures, cache eligibility, stale responses, and workload-specific quality.
Method version: 2026-09-05.
Inputs in the projection
Compare cost, speed, and quality together.
Routing, cache eligibility, retries, response length, latency, and workload quality can outweigh a lower token price.
Configuration frontier
Compare cost per thousand requests, projected average latency, and a user-supplied quality score without collapsing them into one arbitrary rating.
Two kinds of caching
Provider prompt caching discounts eligible input tokens. A full-response cache avoids the model call. The planner models them separately.
Reusable benchmark inputs
Download the workload and configurations as JSON, then use the same assumptions in a local benchmark.
Run a benchmark before choosing a configuration
A local benchmark with representative prompts, real endpoints, and your evaluation rubric measures workload quality, tail latency, reliability, cache eligibility, retrieval relevance, and failure behavior. Use the planner to choose the configurations for that benchmark.