Evaluation pipeline integrity — fixed-scope audit
Read your evaluation export the way an auditor would
Most model evaluation pipelines are trusted because they produce rows. Rows are not evidence. This audit examines one sanitized evaluation export and reports what its rows can and cannot establish about identity, duplication, fixture leakage, pair completeness, timing, unknown states, and reproducibility.
10,417 sealed answer rows looked like a substantial pilot. Only 951 answer identities sat behind them, 2,800 rows were deterministic fixtures, and 0 of 1,400 comparison rows contained a real model response on both sides.
Current production listing decision: UNLIST. newest seal older than declared cadence, synthetic evidence present, no real evidence.
A benchmark that cannot prove both sides of a comparison is not evidence.
Offer status: withheld
This evidence shelf is not currently an offer. We do not list, quote, or accept requests for the audit while its supporting comparison record is stale, synthetic, missing a real pair, future-dated, or unreadable.
The newest verified seal is 2026-09-07 (38 UTC calendar days old as of 2026-10-15). The daily source contains 2,800 fixture answer rows, 1,400 fixture comparison rows, and 0 real comparison pairs.
What our own pilot export turned out to contain
We built an internal evaluation pipeline and then audited it. The volume and date spread looked healthy from the outside, but most stored rows were repeated fixtures and no comparison had genuine model output on both sides. The pilot figures are evidence about evaluation-pipeline integrity—not model quality.
| Evidence check | What volume suggested | What the rows supported | Audit area |
|---|---|---|---|
| Answer identity | 10,417 rows | 951 distinct answer IDs | Duplicate resealing |
| Fixture boundary | 2,800 fixture rows | 7,617 real-answer row | Fixture leakage |
| Comparison completeness | 1,400 comparison rows | 0 real model pairs | Pair completeness |
| Time coverage | 28 snapshot dates | 34 collection runs | Timestamp spread |
What these numbers are not. This is not a model leaderboard, and this page does not rank vendors or models. We publish no model responses—ours or yours. These counts say nothing about whether any model under test was good. They show only what this evaluation record could support.
Pilot provenance and privacy boundary
The first-party pilot spans 2026-07-10 through 2026-09-07 across 28 snapshot dates and 34 collection runs. Before rendering, the reader reproduced 11,893 row, blob, and fetch seals plus every collection root. The newest batch seal is 4dc6ffbfa8229d626a88eaac1a6792ce49866a1a2dd9e2e1e1824bc63b42e9fc.
Source URL: https://ustechautomations.com/permits/model-evidence-audit#pilot-evidence
Evidence effective date: 2026-09-07
Newest collection time: 2026-09-07T14:48:51+00:00
The reader queries aggregate counts only. Prompts, answers, raw JSON, usage payloads, model endpoints, credentials, and customer data are unreachable from the template. A missing or unverifiable source makes the page unavailable rather than producing zero.