Uncertainty in an agent workflow is not a single confidence score. It appears across a sequence of plans, tool calls, observations, retries, and state-changing actions. Reindeer treats uncertainty as observable behavior in a control loop: the system identifies an unresolved question, determines whether it matters to the next action, and chooses whether to proceed, verify, request human input, defer, or stop.
Where agent uncertainty arises
An LLM response can be uncertain because the model lacks knowledge or because the prompt supports several plausible answers. An agent can encounter those conditions and additional uncertainty introduced by its tools and environment.
- Input and context uncertainty: Required information is missing, ambiguous, stale, or inconsistent across documents.
- Evidence uncertainty: Retrieved sources disagree, or the available evidence does not support a required conclusion.
- Tool uncertainty: A tool fails, returns an incomplete result, or does not confirm whether an action succeeded.
- Policy uncertainty: The agent's instructions do not cover the case, or two instructions imply conflicting actions.
- State uncertainty: The workflow state changes during the run, invalidating an earlier assumption or observation.
These forms of uncertainty can accumulate across a trajectory even when the final response sounds confident. For that reason, a final confidence statement is not enough to decide whether an agent should act.
How an agent can reduce uncertainty
An agent can gather information before escalating. Depending on the workflow and its permissions, it can inspect additional records, retrieve another source, repeat a read-only check, compare conflicting values, or validate the current state through a tool.
The agent should request human input when the ambiguity is material to the outcome and remains unresolved after those checks. If the missing information does not affect the permitted action, the agent can proceed while recording the assumption. If the action could create an incorrect or irreversible result, the agent should defer or stop.
Clarification and authorization are separate controls. A clear request does not authorize an agent to exceed its permissions, and an authorized action can still require clarification when critical information is missing.
Signals used to assess uncertainty
Reindeer evaluates several types of signal because no single metric identifies every uncertain case.
- Deterministic checks identify missing required fields, conflicting values, tool failures, schema violations, and other conditions with explicit rules.
- Repeated-run disagreement reveals decisions that change when the same task is run more than once.
- Trajectory divergence compares differences in plans, tool calls, observations, and end states rather than comparing only final outputs.
- External critic evaluation reviews the trajectory or a trajectory prefix for missing context, unsupported conclusions, and unresolved contradictions.
- Agent-reported confidence records the agent's own assessment, but it is treated as one signal rather than an authoritative score.
Repeated-run noise is useful but incomplete. High disagreement can expose a fragile decision, while low disagreement can hide an agent that reaches the same incorrect result consistently.
In controlled Reindeer experiments, agent-reported confidence provided little separation between well-specified and misspecified cases. An external judge over trajectory prefixes identified context gaps more reliably, but it only flagged some forms of misspecification when its evaluation criteria explicitly required that check. This result supports using task-specific metrics instead of a universal uncertainty score.
Turning uncertainty into workflow behavior
Uncertainty signals become useful when they change what the system does during a run. A workflow can use them to select one of four responses:
- Proceed when the evidence supports the action and no material ambiguity remains.
- Verify when another inspection, retrieval, or read-only tool call can resolve the uncertainty.
- Request human input when a person must supply missing information or choose between materially different outcomes.
- Defer or stop when the uncertainty cannot be resolved within the workflow or the next action would create unacceptable risk.
For human-in-the-loop review, the case should include the relevant inputs, retrieved evidence, tool observations, unresolved question, and proposed next action. This context lets the reviewer answer the specific question without reconstructing the entire run.
Evaluating uncertainty handling
An uncertainty policy must balance two failure modes. Under-asking allows the agent to act when essential information is missing. Over-asking transfers routine decisions to people and reduces the value of automation.
Evaluation should therefore measure more than the number of questions asked. Useful checks include whether the agent inspected available evidence first, whether the unresolved ambiguity could change the outcome, whether the question was specific enough to answer, and whether the agent used the answer correctly. Production review can then compare the agent's internal signals with external critic results and the final workflow outcome.
Comments
0 comments
Article is closed for comments.