{"sku":"ai-answer-repeatability-report","label":"AI Answer Repeatability Report","amount_cents":24900,"cadence":"one_time","for_sale":true,"withdrawn":null,"summary":"A private internal-QA PDF and CSV over one frozen panel of manually collected, customer-authorized AI-answer transcripts. It separates citation, domain-text, brand-text, disagreement, and UNKNOWN states without ranking assistants or generalizing beyond supplied sessions.","delivers":["One client domain and exactly five prompts","Three fresh-session manual runs per prompt across one to three fixed assistants","Planned, delivered, excluded, unreadable, and usable record counts","Raw and string-normalized citation targets with four-state labels","Positive and negative matcher controls plus a frozen remeasurement protocol","Private PDF and CSV within three business days after scope confirmation"],"checkout_armed":false,"request_url":"https://permits.ustechautomations.com/public/offers/ai-answer-repeatability-report/request","contact_attribution":{"contract":"product-specific contact to L0 Opportunity","path_id":"ai-answer-repeatability:ai-answer-repeatability-report:contact","hypothesis_id":"h-ai-answer-repeatability-report-offer-2026-08","artifact_id":"ai-answer-repeatability-report-offer-v1","cutover_at":"2026-08-02T07:10:00Z","archetype":"one-time-dataset","data_type":"document-text","geo_level":"none","cta_contract_id":"permits-offer:ai-answer-repeatability-report:primary","proof_url":"https://permits.ustechautomations.com/public/offers/ai-answer-repeatability-report","proof_occurred_at":"2026-08-02T07:10:00Z"},"sample":{"built_at":"2026-08-11T19:53:17+00:00","inputs":["One client domain and exactly five prompts","Three fresh-session manual runs per prompt across one to three fixed assistants","Your written authorization to record the transcripts"],"limits":"This is a private internal-QA artifact over a frozen panel of manually collected transcripts. It measures what a set of assistants answered on the dates we ran them; it does not predict future answers, and it is not a ranking service. Our own run above scored 0 answer-URL mentions across 6 prompts — a real result, not a demo tuned to look good.","method":"Each prompt is run in a fresh session, the readable answer text is hashed and stored, and the outcome is decided by a fixed rule against the target URL and the exact asset name. An answer we could not read is 'unreadable', never a miss, because those are different facts. This is bounded evidence of the dated method stated here; it does not establish affiliation, endorsement, authorization, or any current outcome beyond that scope.","our_own_run":{"methodology":"cited-by-ai-owned-dataset-self-probe-v1","outcomes":{"ANSWER URL MENTION":0,"ASSET MENTION, NO URL":2,"MISS":4,"UNKNOWN":0},"prompts_in_panel":6,"prompts_published":3,"provider_authenticated_citations":0,"publication_rule":"lowest outcome rank, then prompt id; exactly half of the panel","snapshot_date":"2026-08-09"},"outputs":["PDF and CSV separating citation, domain-text mention, brand-text mention, disagreement, and unreadable","Planned, delivered, excluded, unreadable, and usable record counts, reconciled","The response hash behind every scored record"],"sku":"ai-answer-repeatability-report","specimen":{"asset":"Open Permit Fees","model":"Intel/Qwen3.5-397B-A17B-int4-AutoRound","outcome":"MISS","prompt":"What public dataset provides cited United States building-permit fee schedules? Include the dataset source URL.","provider":"local-ungrounded-self-probe-v2","reason":"readable answer contains neither target URL nor exact asset name","response_sha256":"b733db2d621465cebfa9549066dc247f628c9a5754e174e617e25218b9eb0dac","snapshot_date":"2026-08-09","target_url":"https://github.com/USTechAutomations/openpermitfees"},"what_this_is":"We ran this method on ourselves and published the result, including the part that is bad for us. That run is the sample: the same panel shape, the same outcome buckets, and the same refusal to score an unreadable answer as a miss."},"sample_status":{"state":"AVAILABLE"}}