The paper
Evaluator Assurance: Two Case Studies of Formal-Checker Integration.
The question grew out of my audit background and Receiver Reliance research: what evidence justifies accepting a result? In an audit, a control’s existence and evidence that it operated are different matters. For an AI evaluator, I want to know what a formal check covers, what evidence it produces, and whether acceptance actually depends on that evidence.
What the work contains
The completed local case study traces public evaluator artifacts from assurance claims through checks and evidence to accepted outcomes. Its materials include an evidence-trace schema, a bounded audit instrument, a registry of public evaluator paths and controlled comparisons.
The analysis separates source inspection from observed execution and records where evidence is absent or insufficient. It distinguishes a local observation from a concern that still needs verification, and retains counterevidence when it narrows a claim.
What it can support
This is a bounded case study. It does not establish how common a problem is across AI evaluations, demonstrate that a published final score changed, or show downstream harm. A finding about one path also does not establish that a whole system is unsafe or that its authors acted dishonestly.
The purpose is to make assurance claims easier to examine: identify the exact claim, the evidence it depends on and the decision that evidence supports. Broader safety, reliability and governance effects would need their own evidence.
My contribution and verification limits
I set the research direction and motivating questions. Codex agents performed substantial study design, implementation, evidence analysis, internal review and writing. I do not claim independent Lean or Dafny proficiency, or completed personal verification of the technical results.
The manuscript has been revised following internal AI review. That review is not independent human validation. Human technical verification and external review remain outstanding; the manuscript and its research package remain private.
How this connects to Receiver Reliance
Receiver Reliance asks when one agent should rely on another’s work. Its published synthetic study found fewer improper handoffs under structural checks, but also fewer completed seeded tasks; wrong answers with coherent paperwork still passed.
The evaluator study follows the same question upstream: what makes an evaluation result reliable enough for another person or system to use? This is the direction I want to develop through evaluation research, assurance and work connecting technical evidence to governance decisions.