Should You Use Claude Tool Search for a Large Catalog?

Decision: keep a small, task-specific tool catalog loaded directly. Evaluate native Claude tool search when a broad catalog is genuinely needed and most definitions are irrelevant to each request. Approve the change only after testing discovery quality and confirming support for the exact model, account, API protocol, and route.

This memo addresses a concrete integration decision: should an application send tools for immediate model visibility, or make some discoverable through defer_loading? It is not a general agent architecture blueprint. The examples describe an illustrative internal support assistant; they are not measurements from a deployed customer system.

For a Treerouter evaluation, native tool search support is unverified. Ask whether the selected account and route preserve the native search tool, deferred definitions, and continuation blocks before designing a pilot around them. The Claude gateway evaluation guide helps turn that question into an explicit compatibility record.

Context: the catalog grew beyond the task

The illustrative assistant started with two read-only operations: look up an invoice and inspect a vendor contract. Its owners then added tools for project records, inventory, shipping, and support tickets. Each team supplied detailed descriptions, so the request gradually accumulated a directory of capabilities even though any individual question usually concerned one business area.

The decision is not settled by the number of tools alone. A few enormous schemas can be awkward, while many concise and unambiguous definitions may be manageable. Inventory the serialized definitions, the distribution of actual tool use, and common ambiguities between operations. Identify whether the pressure comes from definitions, returned data, or growing history before changing the loading policy.

Anthropic's tool-context guidance distinguishes these sources of pressure. Tool search addresses definitions loaded initially; it adds a discovery step. It does not remove old tool results from a long conversation. This memo therefore leaves oversized results and history retention to separate decisions.

The existing Claude token-counting guide is useful for a separate budgeting investigation. Do not use a reduction in your local JSON byte count as proof of reduced model context, or treat deferred loading as a reason to stop measuring actual request behavior.

Options considered

Decision dimensionSmall catalog, directly loadedSearchable catalog with deferred tools
Best initial fitNarrow workflow with predictable operationsBroad workflow with a long tail of relevant operations
Discovery dependencyModel sees supplied definitions immediatelyModel must discover deferred capabilities before using them
Description responsibilityClarify which visible tool to chooseClarify both discoverability and correct use
Failure investigationInspect selection and executionInspect search, expansion, selection, and execution
Application permissionsFilter and authorize operationsFilter and authorize operations before and after discovery
Rollback approachKeep the validated task-specific setReturn to a validated smaller set for affected workflows

These are design tradeoffs, not benchmark claims. A small catalog can be the correct production choice indefinitely. Search is attractive when the application needs breadth, but it creates another place where a useful operation may fail to become available to the model.

There is also a middle option: keep routinely needed tools visible while deferring the long tail. Choose that split from workload evidence. If almost every support question requires an invoice lookup, deferring it may add discovery without creating a meaningful benefit for that workflow. Conversely, an obscure inventory audit operation need not occupy the initial model view for every invoice question.

The API contract behind the decision

The native tool-search guide documents regex and BM25 search variants. Deferred tools still require complete definitions in the request's tools array. Deferral changes when definitions enter model context, not whether the API receives them. Search returns references that the API expands into definitions, and the search tool itself must remain immediately available.

That distinction rules out a tempting implementation shortcut: replacing every deferred tool with only a name and fetching its schema later in application code. The native contract described here still sends complete definitions. A home-built registry retrieval system would be a separate design with different responsibilities.

The tool reference lists the versioned native search tool types and their server execution role. Pin the version your integration has actually validated. Avoid treating every field accepted by a generic SDK wrapper as proof that the selected route implements the corresponding native behavior.

Tool search also does not move your custom business operations onto Anthropic's servers. The tool execution overview distinguishes model requests from application execution. After a deferred client tool is discovered and called, your executor still owns authorization, input validation, the operation itself, and result delivery.

Proposed catalog policy

Give every tool a stable name and an owner. Its description should identify the business object, the action, and a meaningful limitation. “Look up an invoice by invoice ID; does not search purchase orders” tells a different story from “retrieve finance data.” A searchable catalog needs terminology that users actually employ, while preserving the distinctions that prevent selecting a nearby but unsuitable operation.

The official definition guide describes name, description, and input_schema. Treat those fields as a maintained interface. When a product team renames a business concept, review discoverability along with schema compatibility. An unchanged JSON shape can still become harder to find when the vocabulary in user requests changes.

Create an owner-approved set of representative questions for each deferred capability. Include ordinary phrasing, internal abbreviations, and a neighboring request that should select a different tool. Add negative cases where no authorized capability can fulfill the request. The objective is to evaluate both useful discovery and restraint, rather than rewarding any search result as progress.

The earlier Claude Code context article provides background on tool loading inside a coding workflow. This decision concerns your own Messages API catalog. Its implementation details should come from the current native API documentation and your application contract, not from assumptions about another product's internals.

Offline request construction before an online pilot

This fixture builds a native-style request body from complete authorized definitions. It keeps one frequently used tool visible and defers another. The model is supplied as a variable; the synthetic model value is used only by assertions. No HTTP request is sent, and no route support is implied.

from copy import deepcopy

SEARCH = {
    "type": "tool_search_tool_bm25_20251119",
    "name": "tool_search_tool_bm25",
}


def build_request(model_id, catalog, allowed_names, visible_names):
    if not model_id:
        raise ValueError("Provide a model identifier verified for your route")
    names = [tool["name"] for tool in catalog]
    if len(names) != len(set(names)) or SEARCH["name"] in names:
        raise ValueError("Tool names must be unique, including search")
    allowed = set(allowed_names)
    visible = set(visible_names)
    if not allowed <= set(names) or not visible <= allowed:
        raise ValueError("Unknown or unauthorized tool selection")
    tools = [deepcopy(SEARCH)]
    for original in catalog:
        if original["name"] not in allowed:
            continue
        tool = deepcopy(original)
        if not tool.get("description") or "input_schema" not in tool:
            raise ValueError("Complete definitions are required")
        tool["defer_loading"] = tool["name"] not in visible
        if tool["defer_loading"] and "cache_control" in tool:
            raise ValueError("Deferred tools cannot carry cache_control")
        tools.append(tool)
    return {
        "model": model_id,
        "max_tokens": 1024,
        "tools": tools,
        "messages": [{"role": "user", "content":
                      "Find the invoice record for DEMO-1."}],
    }


catalog = [
    {"name": "read_invoice", "description": "Read an invoice by invoice ID.",
     "input_schema": {"type": "object", "properties": {
         "invoice_id": {"type": "string"}}, "required": ["invoice_id"]}},
    {"name": "read_vendor_contract", "description": "Read vendor contract terms by vendor ID.",
     "input_schema": {"type": "object", "properties": {
         "vendor_id": {"type": "string"}}, "required": ["vendor_id"]}},
]
before = deepcopy(catalog)
request = build_request("offline-model-fixture", catalog,
                        {"read_invoice", "read_vendor_contract"}, {"read_invoice"})
assert catalog == before
assert request["tools"][0] == SEARCH
assert request["tools"][1]["defer_loading"] is False
assert request["tools"][2]["defer_loading"] is True
assert "input_schema" in request["tools"][2]
restricted = build_request("offline-model-fixture", catalog,
                           {"read_invoice"}, {"read_invoice"})
assert "read_vendor_contract" not in {t["name"] for t in restricted["tools"]}
try:
    build_request("offline-model-fixture", catalog, {"read_invoice"},
                  {"read_vendor_contract"})
except ValueError:
    pass
else:
    raise AssertionError("Unauthorized visible tool was accepted")
print("offline catalog fixture passed")

This is a request-construction fixture, not a search emulator or complete API client. It does not test model compatibility, retrieval relevance, reference expansion, result blocks, or the continuation loop. Its basic schema check only verifies presence. Production code needs proper schema validation and a dedicated path for specialized toolsets rather than treating every tool as this simple custom-tool shape.

The cache_control rejection reflects a constraint recorded in the tool reference. It is included to prevent an incompatible definition from entering this request, not to introduce a caching strategy. Keep a separate record of the catalog revision and permission policy used to construct each pilot case so failures can be reproduced.

An explicit decision threshold

Approve deferral for a workflow only when its task set needs access to a broader catalog than the directly loaded baseline can reasonably expose, and the pilot demonstrates acceptable discovery. Write down what acceptable means for that application. A developer support assistant may tolerate another discovery attempt; a tightly timed interactive lookup may have much less room for it. There is no universal catalog-size cutoff in this memo.

Assess failures by stage. A search that returns no useful capability is a discovery problem. A discovered tool that receives the wrong arguments is a selection or input problem. A correct call rejected by the executor may indicate permissions or business validation. Combining these outcomes into one success percentage hides the action the owning team should take.

Assign the remedies accordingly. Catalog owners improve names and descriptions. The application team corrects filtering and execution behavior. The integration owner investigates missing or transformed native blocks. This division helps prevent a search relevance issue from becoming an unproductive endpoint migration, or an adapter defect from becoming repeated prompt edits.

A defensible decision can also be to postpone native search. If the current workload uses a compact set of tools and the larger catalog is only a future possibility, keep the proven configuration. Store the broader definitions in your application registry and revisit the decision when real tasks require them. A feature becomes useful when it solves a demonstrated integration problem.

Authorization remains an application decision

Searchable does not mean executable. Filter the catalog to the tools available to the current user before building the request, then check authorization again when dispatching an operation. A stale conversation or an application bug should not turn the model's recollection of a tool name into permission to run it.

Keep secrets out of descriptions. A tool definition should explain a capability without embedding access tokens or private configuration. Narrow the returned business data to what the task needs. These controls belong to the application's execution boundary and remain necessary regardless of whether the model found a tool through search or saw it immediately.

For catalog changes during an active conversation, use a deliberate migration policy. Removing or renaming an operation can leave older history referring to a definition that no longer exists. Prefer stable catalog revisions for a pilot. If access is revoked, deny execution and handle the pending conversation explicitly; do not preserve access merely to avoid an integration error.

Evidence required to accept the change

Anthropic's advanced-tool-use article explains the motivation for discovering capabilities on demand. Treat that as a design rationale, not a transferable performance result for your application. Decide success criteria before running the comparison.

Run the same representative tasks through the directly loaded baseline and the searchable candidate. Record whether the necessary capability was discovered, whether the intended operation was selected, whether arguments were appropriate, and whether the task completed correctly. Track discovery attempts and total elapsed time separately. A shorter initial context is not a useful outcome if the application repeatedly misses the tool required to finish.

Use the approach in model API acceptance testing to keep fixtures and configuration stable. The agent harness compatibility guide is also relevant when an existing adapter normalizes responses. Confirm that it preserves the native information needed for the complete discovery and execution loop.

Retain the small-catalog configuration as a practical rollback for workflows it can fulfill. If the searchable path cannot discover an essential operation, avoid silently expanding the tool set with unrelated capabilities. Record the failure, route the task into a known supported workflow, and revise the catalog or loading policy with evidence.

For Treerouter, first obtain confirmation of native tool-search support on the intended account and route. If confirmed, register and begin with a read-only catalog and a single discover-and-call task. Only expand after the complete round trip is observable. The buying decision then rests on a demonstrated API capability that your application needs.

Learn more: Treerouter.