Key findings

• Benchmark inference prices fell dramatically, expanding the feasible automation set.

• List price per token does not equal cost per completed workflow.

• Agent loops, output tokens, search, and human review can dominate economics.

• Model routing, caching, batching, prompt control, and deterministic code reduce cost.

• ROI should use accepted outcomes and include correction and maintenance.

Why inference became economically important

Inference runs a trained model on new input. It is the variable compute behind classification, extraction, drafting, search, and agent decisions.

Stanford's 2025 AI Index reported that the cost of querying a model at a GPT-3.5-equivalent MMLU score fell from $20 per million tokens in November 2022 to $0.07 by October 2024, more than a 280-fold reduction. That comparison holds capability at a particular benchmark level; it does not mean every model or task became 280 times cheaper.

Lower-cost models expand feasible work, but larger contexts, tools, and multi-step requests can raise total consumption.

Read vendor prices correctly

Most APIs charge separately for input and output tokens. Output commonly costs more because generation is computationally intensive. Some models also bill reasoning or thinking tokens as output. Long prompts, retrieved documents, conversation history, and tool results can repeatedly enter the context. A workflow with a short visible answer may therefore process much more text than the user sees.

Current vendor pages illustrate the range. Google's July 2026 Gemini pricing lists low-cost text models at fractions of a dollar per million input tokens and stronger models at materially higher rates, with output priced above input. Google also states that managed agent usage includes underlying input, output, intermediate reasoning tokens, and applicable tool fees. Prices change, so a business case should link to the live page and record its date.

Caching and batch processing can reduce cost when repeated context or immediate latency is unnecessary. OpenAI states that its Batch API is priced at a 50% discount to synchronous APIs.

Calculate cost per accepted task

Use a task-level equation: model input cost plus model output cost plus tool fees plus retrieval and storage plus orchestration compute plus monitoring plus human review plus expected correction cost. Divide the total by accepted outcomes, not attempts. If an automation runs 10,000 tasks but only 7,000 are accepted without rework, the economic denominator is 7,000.

For a simple illustrative model, assume each task uses 8,000 input tokens and 1,000 output tokens. Multiply those quantities by the chosen vendor's current per-million rates, then add search, document parsing, or other tool fees. This is an assumption for modeling, not a statement about typical usage. Measure actual tokens in logs because context size can vary widely by case.

Now add human economics. If reviewers inspect every output for 45 seconds, multiply review hours by fully loaded hourly cost. If 5% of tasks need five minutes of correction, include that expected cost. Add a monthly share of integration and maintenance. The model call may be cheap while review or error handling determines whether the workflow pays back.

Compare with the right baseline

The baseline is the current process at the same boundary. Include employee time, contractor cost, software, queue delay, rework, error impact, and management overhead. Do not compare an automated draft with the total cost of a finished human task unless the automation also performs the remaining steps. Conversely, include avoided backlog or faster response when it has a measurable business value.

Separate cash savings from capacity. A workflow that saves ten hours does not create cash if payroll stays unchanged. It creates capacity that can increase output, reduce overtime, avoid hiring, improve service, or support growth. Label the value according to the actual plan. This prevents a time-savings estimate from being presented as guaranteed profit.

Use conservative utilization. Not every eligible task will be automated, and adoption may ramp slowly. Model low, base, and high volumes, but do not assume the same quality at every scale. Edge cases often become a larger share after easy cases are automated. Review cost per accepted task monthly and by workflow segment.

Why agents can reverse simple cost assumptions

An agent may plan, call a CRM, retrieve a file, search the web, draft an action, evaluate its own output, retry, and ask another model for help. Every step can add tokens or a tool charge. Loops can grow when instructions are unclear or systems return errors. A seemingly small task can become expensive and slow without budgets and stop conditions.

Set maximum steps, token budgets, tool-call limits, and wall-clock time. Distinguish recoverable errors from conditions that should escalate. Cache stable policies, summarize long histories, and pass only relevant fields. Use deterministic code for arithmetic, validation, routing rules, and data transformations. The model should handle ambiguity, not work that ordinary software can perform exactly.

Track retries and abandoned runs as first-class costs. An agent that succeeds on the third attempt may appear successful in a completion dashboard while consuming triple the expected resources. Report median and high-percentile task cost, because averages can hide a small group of runaway cases.

A practical cost-control architecture

Route by difficulty. Use rules or a small model for straightforward classification, a mid-tier model for standard drafting, and a stronger model only for complex exceptions. Evaluate the router because a cheap incorrect decision can send hard cases to a weak model. Use confidence and policy checks to escalate rather than forcing every task through an autonomous path.

Reduce repeated context. Store structured facts instead of resending entire conversations, retrieve only relevant document chunks, and cache stable instructions where the provider supports it. Cap output length and request structured data when prose is unnecessary. Batch non-urgent enrichment, summaries, and back-office processing. Each technique should be tested for quality, not applied solely to lower the bill.

Create operational alerts for daily spend, cost per task, retries, tool-call volume, and unusual context length. Tag costs by customer, workflow, and version. When a prompt, model, or knowledge source changes, compare a controlled sample before full rollout. Cost optimization without versioning can silently trade money for error.

Build a 2026 automation business case

Start with observed monthly volume and a representative sample of cases. Run the proposed workflow in shadow mode and record actual token use, latency, accepted outputs, reviewer time, and failure categories. Price the workload with current vendor rates and include a sensitivity range for price, usage, and quality. Do not extrapolate from a single demo.

Calculate gross benefit, total operating cost, implementation cost, net benefit, payback, and risk-adjusted value. Keep observed inputs separate from assumptions. For example, monthly ticket volume and reviewer time may be observed; future adoption rate and avoided hiring are assumptions. Show a downside in which utilization is lower and review is higher than planned.

Approve the workflow when the base case is attractive, the downside is survivable, and non-financial risks are controlled. Revisit economics after launch because prices, models, prompts, and case mix change. Inference cost is a moving input; disciplined measurement is the durable capability.

• Measure actual input, output, reasoning, tool, and retry usage.

• Use accepted outcomes as the denominator.

• Include review, correction, integration, and maintenance.

• Separate capacity value from cash savings.

• Reprice and re-evaluate after model or workflow changes.

Sources and methodology

This article synthesizes the primary sources below as of the publication date. Forecasts and recommendations are directional scenarios, not guarantees; they should be tested against your workflow, data, risk tolerance, and current vendor documentation.

Stanford HAI: AI Index 2025: State of AI in 10 Charts (accessed 2026-07-30)

Google: Gemini Developer API Pricing (accessed 2026-07-30)

Anthropic: Claude API Pricing (accessed 2026-07-30)

OpenAI: OpenAI Batch API FAQ (accessed 2026-07-30)

Stanford HAI: 2026 AI Index Economy Chapter (accessed 2026-07-30)