Product

Simulate. Observe. Review.

VoiceCanary exercises difficult scenarios against a staging agent, keeps each claim beside the operation it triggered, independently reads the saved outcome and gives reviewers the evidence behind every decision.

Product overview

One evaluation lifecycle, from test design to evidence-backed report.

Run unattended scenario matrices, inspect what happened across the full evidence chain, and keep human judgment attributable wherever interpretation is required.

01Evaluation stage

Simulate

Exercise the difficult calls before customers find them.

Combine reusable test cases with cooperative, distracted and adversarial personas. The caller is simulated while tool writes and independent database readback remain real inside the test environment.

Persona × scenario matrices
Customer-authored test cases and sets
Deterministic checks for tool use and saved state

Example configuration

Appointment-change stress test

3 personas × 4 scenarios · 12 runs

Cooperative

Follows the expected path

4 runs

Distracted

Corrects details and changes direction

4 runs

Adversarial

Pushes past identity and policy limits

4 runs

02Evaluation stage

Observe

Keep the claim, the operation and the outcome together.

Inspect live-agent and simulated runs through one evidence chain. Imported public or authorised private datasets remain a separate measurement workflow and do not receive an agent-quality verdict.

Provider transcript and tool receipts
Independent backend-state readback
Explicit unavailable states when evidence is missing

Evidence chain

Reschedule · rejected write

MISMATCH

Agent claim

“Your appointment is moved.”

Tool receipt

409 · slot unavailable

State readback

Original appointment unchanged

03Evaluation stage

Review

Turn evidence into an attributable decision.

Reviewers label the agent’s claims, inspect advisory suggestions and record their reasoning. Supported checks, unresolved evidence and version metadata stay attached to the report.

Human-attributed claim decisions
Comparable runs across compatible versions
Shareable evidence report and JSON export

Claim review

2 of 3 claims labelled

Identity was verified before the change.

SUPPORTED

The appointment was moved to the requested slot.

CONTRADICTED

A specialist will follow up.

REVIEW

Evaluation system

The parts behind every result.

A result stays useful because its scenario, policy, evidence, versions and reviewer remain connected.

Scenario configuration

Define the calls a run exercises: successful bookings, injected tool failures, retries, caller corrections and authorization edge cases.

Customer-approved policies

Your policy is the reference, not a generic rulebook. Policies are versioned so a verdict can always be traced to the rule it was judged against.

Actual outcome assertions

Assertions compare the agent's claim with what the approved outcome source recorded — an appointment that exists, with the agreed details.

Correlated evidence

Transcript turns, tool requests and responses, and recorded state are linked to the checks that rely on them.

Version comparisons

Two agent versions, identical scenario, policy and evaluator versions, and a diff of the resulting verdicts.

Human review

A reviewer can dispute a verdict and attach reasoning. Feedback proposes a change to criteria; it never silently rewrites them.

Report preview

A summary for decisions, with the evidence one click away.

Compare compatible agent versions across implemented scenarios, then open any decision to inspect the supporting transcript, receipts, state and review metadata.

Evaluation report

Scheduling workflow · version comparison

Example report · fictional data

Scenarios exercised

implemented fixtures

Checks supported

70%

14 of 20

Blocking runs

observed failures

Missing evidence

never counted as pass

Evidence included

Transcript and attributed claim labels
Tool requests, responses and receipts
Independent saved-state readback
Scenario, policy and evaluator versions

Scenariov1.4.2v1.5.0

False booking confirmation

rejected

BLOCKED

2/4 checks

PASS

4/4 checks

Wrong appointment details

rejected

BLOCKED

1/4 checks

BLOCKED

2/4 checks

Duplicate booking after retry

retry

BLOCKED

1/4 checks

BLOCKED

3/4 checks

Missing authoritative outcome evidence

missing-evidence

INCOMPLETE

1/4 checks

INCOMPLETE

1/4 checks

Accepted reschedule with correct readback

reschedule

PASS

4/4 checks

PASS

4/4 checks

Each decision opens to its checks and supporting evidence.JSON export included

Anatomy of a result

Everything a single result carries.

Labelled fixture — scripted actor, no live call and no patient records.

Release decision: BLOCKED

Agent version: agent-1.4.2
Scenario version: scn-false-confirm-3
Policy version: pol-sched-booking-1.2
Evaluator version: eval-0.7.1
Expected outcome: Either an appointment record exists matching the agreed slot, or the agent accurately reports that booking did not complete and follows the approved fallback.
Observed outcome: No appointment record exists. The agent stated the appointment was booked and ended the call.
Evidence references: turn-5, tool-1, state, turn-5, policy, turn-2, tool-1, tool-1
Coverage limitation: Covers the checks and conditions in this scenario only. Not a statement about untested workflows.
Failure attribution: Agent policy violation (unsupported success claim).

Release decisions

Three outcomes, and no averaging.

A critical failure cannot be averaged away by passing checks elsewhere in the run.

Release decision: PASS

PASS

All mandatory applicable checks meet the approved release policy for this run.

Release decision: BLOCKED

BLOCKED

A critical violation was observed, or a configured blocking threshold was met.

Release decision: INCOMPLETE

INCOMPLETE

Required evidence or evaluation was unavailable. Nothing is inferred about the underlying action.

Start with the workflow that matters.

Scope a pilot around your agent, your success criteria and the evidence your team needs to make a decision.

Book a demo Explore the product

A guided walkthrough tailored to your team and your workflow.