If Gemini keeps requesting a function after your application has already returned its result, inspect the calling mode on the next model request. In GenerateContent, ANY constrains the model to produce function calls. Leaving that setting enabled while asking for a final answer can keep the workflow in its tool-selection phase. A prompt saying “you have enough information now” does not change the request configuration.
The practical fix is to decide when another lookup is allowed, then configure each phase accordingly. This walkthrough uses a read-only order-status assistant: obtain an authorized order record once, then answer from that record or explain what is missing. Its limits are application policy, not a claim about every agent task. The example is deliberately small enough to test without a model account or customer database.
Choose the protocol before choosing the mode
All configuration below targets the native Gemini GenerateContent REST protocol. Its request uses contents, tools, and toolConfig; the function mode lives at toolConfig.functionCallingConfig.mode. Google’s current GenerateContent method reference documents that contract. The code returns configuration fragments, not complete API requests.
This distinction matters because the current function-calling guide includes Interactions examples with a different request contract. Do not transplant those fields into GenerateContent or assume an OpenAI-compatible gateway accepts the native configuration unchanged. The Gemini gateway endpoint guide helps identify which interface an application actually sends. Confirm the selected model and route support your required behavior before using these settings through a relay.
Google’s GenAI mode reference distinguishes the three choices relevant here:
| GenerateContent mode | What the model may produce | Place in this workflow |
|---|---|---|
ANY | Function calls rather than a natural-language answer | Obtain the required order record |
AUTO | A function call or natural-language response | Optional lookup in a broader assistant |
NONE | No function calls | Write the answer after lookup |
AUTO permits the model to stop calling functions; it does not promise that it will stop after one call. NONE prevents function selection; it does not certify the resulting answer. These are different controls from your database permissions, execution budget, and rules for accepting a customer-facing statement.
Check the serialized second request, not just the configuration object your code intended to build. A shared client default or a merge that retains the first phase’s settings can make two apparently different code paths send the same mode. The useful diagnostic is a sanitized record of the mode actually submitted, paired with the application phase that selected it.
Decision 1: is this lookup ready to run?
Suppose a signed-in customer asks, “Has my order shipped, and when should it arrive?” The interface already knows which order the customer selected. Before asking the model to choose a function, the application checks that selection and the customer’s access. A reference typed into chat must receive the same checks. The model does not get to establish ownership by producing plausible arguments.
If no order is selected, ask the customer to select one. If access is denied, stop the lookup. Neither situation becomes more solvable by forcing a function call. Supplying ANY while withholding a required business identifier creates pressure to produce an unusable request; the execution layer must still reject it.
For this product, admission means an authorized order reference is available and no lookup attempt has been consumed. Only then does the application request the first model turn with ANY and an allowlist containing lookup_order. Restricting selection to that name is useful, but it grants no extra database privileges. The tool still runs under the backend’s established identity and tenant boundary.
If the interface already determines the exact lookup, calling the service directly is also reasonable. Function calling is useful when the assistant needs to interpret a request or choose among supported operations. It should not be inserted merely to make an otherwise deterministic screen look agentic. The bounded design remains useful whichever component chooses the lookup.
Decision 2: should the proposed function actually execute?
Google’s function-calling guide explicitly separates the model’s proposed function from execution by your application. Treat the response as a request for an operation. Validate its name, argument structure, selected order, and current permissions before touching the order service. A successful model response is not an execution receipt.
The narrow workflow accepts exactly one complete lookup_order proposal with exactly the expected order reference. An unexpected function, a different reference, extra authority-bearing arguments, or multiple proposals ends this attempt without execution. Do not run the first proposal and inspect the rest afterward. Admission should consider the completed response as a whole.
Reserve the execution attempt before invoking the service so a timeout does not leave the budget looking unused. In a production worker system, that reservation needs durable state and an atomic update; a local counter alone cannot coordinate duplicate deliveries. Read-only operations still deserve this boundary because repeated requests consume capacity and can expose additional records if scope validation is weak.
This decision is separate from repairing malformed conversation history. If the continuation is rejected rather than repeatedly requesting a valid function, the GLM tool-history investigation illustrates that different failure category. Its protocol-specific repair should not be copied into this Gemini workflow. Establish which boundary failed before changing the calling mode.
Decision 3: answer, qualify, or ask for clarification
After the lookup, classify the service result in application code. A complete record can support both shipment status and the recorded delivery estimate. A partial record may support shipment status while leaving the delivery date unknown. Missing information is an outcome to communicate, not automatic permission to repeat the same request.
| Application evidence | Next step | Customer-facing boundary |
|---|---|---|
| Authorized record with delivery estimate | Answer with NONE | Describe the date as an estimate |
| Authorized record without delivery estimate | Answer with NONE | State that no estimate is available |
| Missing order selection | Clarify without a lookup | Ask which order the customer means |
| Record unavailable or service failure | Stop this attempt | Explain that status could not be verified |
| Access denied | Stop this attempt | Follow the account’s access-recovery flow |
For an answerable result, make the second generation request with NONE. Supply the validated service facts and explicit instructions to acknowledge unavailable fields. Keep the required GenerateContent continuation intact when returning the function result; changing the mode is not a substitute for a valid conversation. The application should preserve the provider’s returned content rather than reconstructing only a convenient text summary.
For example, a successful trace might contain one proposed lookup, one admitted service call, a record showing shipment without an estimated arrival, and one answer acknowledging both facts. The absence of an estimate does not make this task unsuccessful. The customer received everything the service could establish. Repeating the same lookup immediately would need a separate reason, such as a documented freshness requirement, rather than a general desire for a more satisfying answer.
The answer still needs a product-level acceptance rule. If the record says “shipped” but has no delivery estimate, “Your order will arrive tomorrow” is unsupported even though it contains no function call. For a small status panel, rendering trusted fields with a template may be preferable to generating prose. If generated wording is used, reject unsupported commitments and provide a deterministic fallback.
Google’s structured-output guidance separates formatting an answer from requesting an action and warns that application validation remains necessary. A schema can help a UI consume an answer, but it cannot make an absent delivery date true. Do not add a second tool request just to repair the presentation of facts already obtained.
Put the phase boundary in code
The following policy function uses application-owned state. complete and partial describe validated service results; they are not statuses the model assigns to itself. The action and reason keys are local routing decisions, not Gemini request fields. Only the nested toolConfig belongs in a GenerateContent request, alongside its other required content and declarations.
from dataclasses import dataclass
@dataclass
class State:
order_ref: str | None
authorized: bool
model_turns: int = 0
tool_attempts: int = 0
outcome: str = "pending"
def next_step(s: State) -> dict:
if not s.authorized:
return {"action": "stop", "reason": "access_denied"}
if not s.order_ref:
return {"action": "clarify", "reason": "order_reference_needed"}
if s.model_turns >= 2:
return {"action": "stop", "reason": "model_budget_exhausted"}
if s.outcome in {"complete", "partial"}:
if s.model_turns != 1 or s.tool_attempts != 1:
return {"action": "stop", "reason": "invalid_state"}
return {
"action": "answer",
"toolConfig": {"functionCallingConfig": {"mode": "NONE"}},
}
if s.outcome != "pending" or s.model_turns or s.tool_attempts:
return {"action": "stop", "reason": "no_further_lookup"}
return {
"action": "request_lookup",
"toolConfig": {"functionCallingConfig": {
"mode": "ANY", "allowedFunctionNames": ["lookup_order"]
}},
}
# Synthetic application state; no provider request or real order is used.
ready = State(order_ref="fixture-order", authorized=True)
assert next_step(ready)["toolConfig"]["functionCallingConfig"]["mode"] == "ANY"
assert next_step(State(None, True))["action"] == "clarify"
assert next_step(State("fixture-order", False))["action"] == "stop"
for outcome in ("complete", "partial"):
state = State("fixture-order", True, 1, 1, outcome)
assert next_step(state)["toolConfig"]["functionCallingConfig"]["mode"] == "NONE"
assert next_step(State("fixture-order", True, 1, 1, "error"))["action"] == "stop"
assert next_step(State("fixture-order", True, 2, 1, "complete"))["action"] == "stop"
print("PASS: 7 policy fixtures")The dispatcher must increment the model-turn counter before dispatch, enforce the execution admission check, and record the tool outcome before requesting a new decision. It must also handle stop and clarify locally. Repeatedly calling this pure policy function without updating state would not enforce anything. Persist these transitions with the task, including when a worker restarts.
The inline assertions exercise seven synthetic application states without making network requests. They cover initial lookup selection, missing input, denied access, complete and partial records, a failed lookup, and an exhausted model budget. A passing run demonstrates the policy’s chosen transitions. It does not demonstrate tool execution, model compliance, or gateway compatibility; those require the dispatcher and route checks described here.
Save the complete block as a Python file and run it with Python 3.10 or later; the expected output is PASS: 7 policy fixtures. A later customer request to refresh the order should create a new, authorized task under your product’s refresh policy. It should not reset the exhausted counters inside the existing task, because that would erase the very boundary the example is intended to enforce.
Keep retry decisions outside the function loop
This design permits at most two logical model turns and one order-service attempt. Those limits alone do not bound transport retries hidden inside an SDK. Give HTTP retries their own attempt limit and deadline, and account for them in the same task record. Configure the client deliberately rather than assuming each function invocation represents exactly one network request.
Google’s troubleshooting guide recommends bounded retry strategies for appropriate transient failures. A repeated, valid function proposal is a different event from a retryable transport error. If the answer phase unexpectedly contains another proposal, reject execution and investigate the actual request configuration or route behavior. Do not silently restart the lookup phase.
An empty or blocked model response also needs its own handling. The Gemini safety-response guide addresses that classification. Neither a block nor an unexplained empty result proves that fresh order data is needed. Preserve the verified facts and offer the appropriate fallback instead of making the customer wait through repeated lookups.
Verify a complete task before increasing scope
Use a small acceptance set containing the same cases as the offline policy, then run it through the actual selected model and route. Observe the outbound mode on both turns, the proposed function, the execution count, the service result category, and the final customer message. Include a partial record specifically to check that the assistant admits what it does not know.
Record the stop reason as well as the final answer. A task that ends because its budget was exhausted differs from a task that answered correctly. The agent-harness cost guide explains why cumulative task usage matters; use your observed request records without inventing savings from the offline fixtures.
When moving this workflow between models, the harness integration guide provides broader compatibility checks. Keep this acceptance criterion narrow: one authorized lookup, an evidence-based answer or explicit limitation, and no execution after the answer phase begins. Expand the tool budget only when a new business requirement explains why another operation is necessary.
FAQ
Should I replace every ANY setting with AUTO?
No. Use ANY where a ready, authorized phase specifically requires a function proposal. Use AUTO where either a direct answer or a lookup is acceptable. For this workflow’s final writing phase, NONE expresses the intended boundary more clearly. Each choice still needs an application execution limit.
Does NONE guarantee a correct final answer?
No. It prevents function calls; it does not establish factual accuracy or completeness. Validate the answer against the retrieved record, preserve uncertainty, and use a deterministic response when the necessary facts are unavailable. A missing delivery estimate should remain missing rather than becoming a confident prediction.
Can a gateway fix the loop automatically?
A route may expose relevant settings, but its exact support must be verified. The application owns the phase transition and execution policy. Before a TreeRouter pilot, confirm the selected Gemini route, model, mode handling, and function-result behavior. Then register through the platform and run the small acceptance set before moving production traffic.
References
- Google: GenerateContent method and request configuration
- Google: Function calling guide
- Google GenAI: FunctionCallingConfigMode
- Google: Structured outputs and application validation
- Google: Troubleshooting and bounded retries
- Gemini gateway endpoint selection
- GLM proxy tool-history errors
- Gemini safety blocks and empty responses
- Agent harness token costs
- Connecting an agent harness to other models






