Screen 04[BUILT]
Bereavement refund misstatement (NorthStar bot)
Customer Support · Consumer protection / BCCRT precedent (Moffatt v. Air Canada) — one entry in Company XYZ's versioned library of : preserved decision conditions paired with an Authorized Expected Behavior, reused on every future candidate version.
Company XYZ decision accountability · LoanCore Underwriting 3.2.0Customer Support · Case VR-NS-001— Representative lending boundary case
- ACCOUNT1–3→
- VERIFY4–7→
- DEFEND8–9
VERIFYScreen 04
- Question
- Which decision conditions must future versions continue to handle correctly?
- Answer
- Company XYZ has preserved representative, boundary, disputed, and previously failed decisions as reusable Assurance Scenarios.
- Output
- A versioned library of decision conditions with authorized expected outcomes.
Fault attribution · Model hallucination
The LLM produced a policy statement with zero retrieved supporting documents.
Decision chain
[BUILT]captured evidenceINPUT
Customer message
'Bereavement fare — how do I get the refund?'
TOOL
Policy retrieval
0 documents matched (empty index)
MODEL
Failing or uncertain nodeSupport LLM v1.4
Generated answer without grounding
RULE
Grounding gate
MISSING — no citation-required check
DECISION
Reply sent
Message delivered to customer
Why this case is useful for release assurance
- source_decision
- Case VR-NS-001, bereavement fare
- trigger
- 37 policy statements with no retrieval binding
- failing_node
- Support LLM v1.4
- evidence_level
- E3
- replayability
- Replay-ready from the sealed cassette
- missing_evidence
- Policy corpus was never ingested, so a contradicting source cannot be shown.
- applicable_policy
- Customer Communications Standard 2.0 (2026-03)
- expected_behavior
- Authorized by VP Customer Experience — Jordan Ochoa
- suggested_preservation
- Preserve decision-time policy version and human-review evidence on the next occurrence
- suggested_scenario
- Add to the Customer Support Release Scenario set as a customer challenge
Candidate provenance
- evaluator
- company-xyz-evaluator v0.6.2
- frozen_input_set
- customer-support-sweep-2026-07
- evaluated_at
- 2026-07-22T04:15:00Z
- source_evidence
- sha256:d81a…4471
- relationship_confidence
- Confirmed — decision and adverse-action record share a source identifier
- known_limitations
- Detection limited to responses with structured retrieval logs.
Captured decision
Case VR-NS-001, bereavement fare
Outcome
TOLD CUSTOMER: 'apply within 90 days for refund'
reason: Response asserted policy; retrieval returned no matching document
Authorized Expected Behavior
Refuse to state a policy without a cited source document.
Evidence
E3sufficient for verification
cassette · sha256:d81a…4471
Next action
Advisory only. An must define or approve the expected behavior before a candidate version can be verified against it.
→ Authorize expected behaviorScreen anatomy
What this screen proves
For any single decision case, Notary AI can show which nodes ran, which node is failing or uncertain, what was captured, what evidence is missing, and what the record is at.
Why it matters
This is how a case earns a place in the release-assurance suite. A useful case has preserved evidence, a known applicable policy, and a clear expected behavior worth testing every future candidate version against.
Built or Planned
Planned capability for candidate generation; the decision-chain view and evidence grading are built. Candidates shown here are a seeded illustration.
Important boundary
The candidate is advisory. Notary AI does not determine what the acceptable outcome should be — an supplies the Authorized Expected Behavior on Screen 05, and the re-checks eligibility server-side.
Related requirements
Next
A scenario preserves the conditions, but it cannot decide what outcome is acceptable. An authorized Company XYZ reviewer must establish that.
Authorized Behavior →