Runtime Findings
Runtime findings turn configured, repeatable checks over agent-execution evidence into reviewable signals. They show a measured condition; they do not claim to prove its cause.
What a finding means
A runtime finding is a structured signal that a configured condition has been met. For example, it can show that a tool is failing much more often in recent executions than it did in its established baseline. It records the rule, numbers, time window, and evidence used, so a person can decide what to investigate.
A finding is not a diagnosis
A latency finding may come from a provider outage, a new tool version, traffic shape, or another cause. Use it as a well-explained starting point, not as an automatic conclusion.Detector Families
The platform includes deterministic detector families for common operating conditions. Enable and tune only the ones that make sense for a particular agent and its responsibilities.
| Detector | What it looks for | Example review question |
|---|---|---|
| Tool failure | A tool's failure rate has increased. | Did the policy-lookup service or its integration change? |
| Agent execution failure | More agent runs are ending unsuccessfully. | Which stage is preventing completion? |
| Latency regression | Recent work is measurably slower than its baseline. | Is the delay at a model, tool, or runtime boundary? |
| Evaluation failure | A recorded evaluation is failing more often. | Did an evaluation rule, input distribution, or agent change? |
| Policy denial | Governance decisions are denying work more often. | Is a policy change or a new request pattern responsible? |
| Repeated error | The same error pattern keeps returning. | Should the runtime handle or route this error differently? |
How evidence is compared
Most detectors compare a baseline window (what normal has looked like over a longer period) with a recent window (what has just happened). They also require a minimum amount of evidence, so a single unlucky request does not become a misleading alert.
longer baseline of completed executions → recent observation window → threshold comparison → finding or no finding
The comparison can use an absolute change, a relative change, or both. Severity bands classify how far the observed value crosses the configured threshold. A detector only considers evidence available inside its window; late events remain recorded, but may not change a window that has already been finalized.
Detect, Reconcile, Review
| Action | Use it when | Result |
|---|---|---|
| Detect | New evidence arrived, or after a deliberate runtime change. | Runs enabled checks and creates or refreshes operational findings. |
| Reconcile | A suspected issue has been fixed. | Looks for the configured number of consecutive healthy windows before resolving a finding. |
| Review | A person needs to decide the response. | Use the recorded scope, numbers, and evidence to investigate outside the control plane. |
A healthy single execution is useful, but it is not enough evidence of recovery by itself. Causal Audit findings are different: they describe a completed historical audit and are not automatically reconciled by later traffic.
Settings and Scope
Each detector family can be configured with a minimum observation count, baseline period, recent observation period, thresholds, severity bands, allowed event lateness, and recovery rule. Start with conservative thresholds and enough ordinary traffic to establish a useful baseline. Tune one agent or project before widening a setting to the organization.
When more than one setting applies, the most specific governed scope wins; environment overrides, when configured by an operator, take priority and are not ordinary Studio edits. Record why a threshold changed and revisit it after a deployment or traffic-shape change.
Response Playbook
- Read the finding's detector, baseline, recent window, threshold, and severity.
- Check the affected agent version, tool, policy, and deployment changes.
- Investigate in the owning runtime or service; the control plane does not repair it.
- After remediation, run detection and reconciliation according to your operating process.
- Adjust a setting only when evidence supports a durable change in expected behavior.
See Agents Runtime Operations for access, worker, and troubleshooting guidance. To investigate the effect of an approved tool result on one completed run, use Causal Audit.