Agent Harness Token Costs: What You Actually Pay For

Learn more：https://treerouter.com

An agent can produce a short final answer after making a long series of model requests.

That is why the length of the answer on screen is a poor guide to the cost of an agent task. Before responding, the system may read files, inspect tool output, generate a plan, revise code, run tests, and retry a failed step.

An agent harness does not inherently consume model tokens just because its local control code is running. Tokens are consumed when the workflow sends requests to a model. The harness determines how often those requests occur and what information they include.

To understand agent harness token costs, measure the complete task.

Why one task can generate many billable requests

A simple chat exchange might involve one request and one response. An agent task often contains a sequence.

For example, an agent asked to fix a failing test might:

1. Read the task and inspect the repository.
2. Request relevant files.
3. Analyze a failure.
4. Propose and apply a change.
5. Read the test output.
6. Revise the change.
7. Produce a final report.

Not every step requires a separate model call, and different harnesses organize the work differently. However, each additional model call can introduce more input and output usage.

The final report may contain only a few hundred words while the task’s cumulative usage is much larger.

Repeated context is an important cost driver

A later request may include instructions, conversation history, selected files, tool definitions, and previous tool results.

The API may therefore process some information more than once across a task.

Consider a simplified, hypothetical sequence:

These numbers are illustrative, not measurements from a particular model or platform.

They show why the context size of the final request is not the same as cumulative input usage. The final request contains 15,000 input tokens, but the task has processed 40,000 across all four requests.

Caching may change the billing treatment of eligible input. It does not mean you can ignore repeated context when analyzing the workflow.

Calculate request cost before calculating task cost

For a simple token-priced request without caching or additional charges:

Request cost =
(input tokens × input price per million ÷ 1,000,000)
+
(output tokens × output price per million ÷ 1,000,000)

Suppose the hypothetical task above uses a model priced at $1 per million input tokens and $5 per million output tokens.

Its model-token cost would be:

Input:  40,000 × $1 ÷ 1,000,000 = $0.04
Output:  4,000 × $5 ÷ 1,000,000 = $0.02
Total:                              $0.06

These are assumed prices for explaining the calculation. They are not a current provider quotation.

For an actual task, sum all requests and apply the provider’s billing categories. Cached input, reasoning usage, model-hosted tools, and other charges may require additional treatment. Avoid counting a usage category twice when the provider already includes it in another total.

Do tools consume tokens?

A local tool and a model request are different operations.

Running a shell command does not automatically create an additional model-token charge. But if the tool’s output is sent back to the model, that output becomes part of the next request’s input.

Some tools or execution environments also have their own charges. Those belong in the task’s total cost even when they are not billed as tokens.

This distinction explains why verbose tools can be expensive indirectly. A command that prints thousands of irrelevant lines can inflate the next prompt without helping the agent make a better decision.

Return the information needed for the next decision. For a failed test, that might mean the failing assertion and relevant stack trace rather than the entire build log.

Retries can outweigh a lower token price

Comparing models only by their published token rates misses the cost of unsuccessful attempts.

Use a batch of representative tasks and calculate:

Cost per successful task =
total cost of all attempts ÷ number of successful tasks

For example, imagine two hypothetical configurations tested on the same 100 tasks.

Configuration B costs more across the batch but less per successful outcome.

This does not prove that a more expensive model is always better. It shows why completion rate, retries, and task difficulty belong in the same evaluation as price.

Define success before testing. A plausible explanation is not equivalent to a verified code change, a correct extraction, or a completed workflow.

Record usage at the request level

For each model request, record a task identifier, model name, usage information, elapsed time, and outcome. Associate retries with the original task instead of treating them as unrelated activity.

OpenAI’s documentation distinguishes usage fields by API: Chat Completions commonly reports prompt_tokens and completion_tokens, while Responses uses input_tokens and output_tokens. Follow the schema of the API you actually use. OpenAI Help Center

Streaming needs extra care. OpenAI documents that an interrupted stream may prevent the final usage chunk from arriving. Missing client-side usage should therefore not automatically be interpreted as zero cost. Reconcile incomplete records with available provider usage data. OpenAI API Reference

Reduce waste without removing necessary context

Start with repeated work that does not improve the outcome.

Keep tool results focused. Avoid repeatedly loading unchanged files. Set sensible limits on retries and require the agent to stop or escalate when it encounters the same failure repeatedly.

For longer tasks, preserve a compact progress record containing completed work, unresolved issues, and the next action. Make sure summaries retain details needed to continue accurately.

Model selection can also be task-specific. A simple classification step may have different requirements from debugging a complex interaction. Test each assignment rather than assuming one model should handle every stage.

When using TreeRouter or another API gateway, check whether its usage reporting can be reconciled with your application’s task records. A model-level bill is useful, but a task-level cost record explains which workflows are consuming the budget.

The metric to watch is not simply tokens per answer. It is the cost of completing a useful task, including failed attempts and the work needed to recover from them.