Agent Tool Evidence to Causal Audit

Architecture

The runtime owns payloads; Core owns the governed record.

Prerequisites

Prepare boundaries.

• Python 3.12 or later, Git, and Docker Desktop.
• A running AIGP stack with the API, replay worker, and a project-scoped operator identity.
• A local OAuth configuration for normal Synthetic Runtime evidence ingestion.
• A separate shared secret for the protected runtime replay endpoint.

git clone https://github.com/Bhanuj-AI/synthetic-agent-runtime.git
cd synthetic-agent-runtime
python3.12 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -e '.[dev]'

# Use one locally generated secret in both processes. Do not commit it.
export REPLAY_SHARED_SECRET='replace-with-a-local-secret'

# Terminal 1 — Synthetic Agent Runtime
export SYNTHETIC_RUNTIME_REPLAY_AUTH_TOKEN="$REPLAY_SHARED_SECRET"

# Terminal 2 — AI Governance replay worker
export AI_GOVERNANCE_SYNTHETIC_RUNTIME_REPLAY_AUTH_TOKEN="$REPLAY_SHARED_SECRET"
export AI_GOVERNANCE_SYNTHETIC_RUNTIME_REPLAY_ENDPOINT='http://host.docker.internal:8090/replay'

Run Flow

Produce Observed Source Execution.

Start the Synthetic Agent Runtime, then execute the built-in fraud-causal-guide scenario. It sends bounded operational evidence to Core while retaining its tool inputs and outputs inside the runtime.

.venv/bin/python -m synthetic_agent_runtime serve \
  --host 127.0.0.1 \
  --port 8090

.venv/bin/python -m synthetic_agent_runtime run \
  --scenario fraud-causal-guide \
  --seed 42 \
  --duration 1

Synthetic Agent Runtime run: <run-id>
External execution: sar-<run-prefix>-0000000
Core execution: <platform-execution-id>
Source decision: REVIEW
Source outcome score: 0.90

Inspect fan-in

Confirm dependency graph before intervenention.

Open the source execution in Agents Runtime. The two input calls share one tool_call_group_id and neither depends on the other. The risk score depends on both. A later claim-history observation depends on the risk score, demonstrating that a counterfactual execution can diverge from the source's downstream topology.

Configure policy and audit

Authorize one narrow transformation.

Start in Studio: open Agents Runtime, select the Causal Audit tab, then open Intervention Policies. Create an active policy before you create an audit. The policy is the approval boundary: it says exactly which observed tool evidence a replay worker may change and how.

1. 1Click Create Policy. Under Select Observed Tool Evidence, choose the completed fraud source execution that the runtime printed.
2. 2Select transaction_risk.score as the auditable tool result. Let Studio populate the tool name, evidence schema ID, and schema version from that observed descriptor; do not type a near-match.
3. 3Set the structured JSON intervention provider and choose PERTURB. Enter the bounded control shown below, then validate and activate version 1 of the policy.
4. 4Return to Causal Audit → Audits, create an audit for the same observed execution, and select this active policy. Submit it and keep the source execution ID handy for the Replay Management step.

Policy target
Tool name: transaction_risk.score
Evidence schema: synthetic-fraud-risk
Schema version: v1

Counterfactual controls
Provider: structured-json
Provider version: v1
Strategy: PERTURB
Field path: /confidence
Operation: NUMERIC_DELTA
Delta: -0.50
Permitted range: 0.00 to 1.00
Samples: 1

Intervention: original evidence → controlled PERTURB → replay result.

Core now freezes the source descriptor and submits the replay job. The worker authenticates to the external runtime with its bearer token. The runtime validates the execution ID, tool-call ID, source evidence digest, policy and intervention identity, approved transformation, and frozen snapshot before it applies the perturbation.

Boundary and Provenance

A digest is not a private payload.

source_evidence_digest protects the bounded original evidence reference. intervention_provenance_digest identifies the governed transformation. After replay, the runtime computes counterfactual_evidence_digest for what it actually produced. None is a reason to disclose private evidence bytes to Core.

Find the replay and result

Close the loop in Controlled Replay and Causal Audit.

Do not jump from policy creation to the result. First use the left navigation to open Replay. In Replay Management, paste the Core execution ID printed after the source run into Source execution ID. Open the controlled causal replay whose source is agent-runtime-source:<your-core-execution-id>, then select Details.

1. 1On the replay overview, confirm EXECUTION COMPLETED and Controlled Causal Replay. This excludes ordinary historical or demo replay rows.
2. 2Open the Audit tab. The trail should show source evidence frozen, the replay job submitted, and the replay execution succeeded.
3. 3Use View evidence influence from the replay row, or return to Agents Runtime → Causal Audit → Audits and select the audit for this execution. Click Explain to open the Evidence influence detail.

Counterfactual decision: ALLOW
Counterfactual outcome score: 0.25
Counterfactual evidence digest: sha256:<runtime-computed-digest>
Causal classification: EVIDENCE_ALIGNED
Evidence influence: +0.650

Proven result: REVIEW 0.90 → ALLOW 0.25 · influence 0.65 · EVIDENCE_ALIGNED.