Dataset Hygiene Observation Report
One public benchmark or evaluation file, read under fixed mechanical rules and reported against the SHA-256 of the exact bytes fetched. These dataset hygiene observations count data-shape problems a releaser can act on without pretending to judge the benchmark itself.
Proposed scope: $399 once for 1 already-public CSV or JSONL file, up to 5,000 rows, 20 MB, and 2 declared splits, returned within 2 business days after scope acceptance.
$399 one time — proposed
Two business days after one public URL passes the size, format, rights-attestation, and source-suitability review. A clean report is still the delivered work and is billable.
Who this is for
This is for an open benchmark author, evaluation research lead, or open-source maintainer preparing a public CSV or JSONL release. It is most useful when the file is small enough to inspect completely but consequential enough that a duplicate cluster, split collision, malformed record, or literal answer string would create avoidable review work after publication.
Dataset hygiene here means disclosed measurements of the delivered file shape. It does not mean that a clean result clears the release, because the difficult questions about meaning, rights, privacy, safety, and contamination sit outside those mechanical measurements.
The buyer supplies one URL that is already public. No upload, repository access, credential, private branch, model endpoint, or inference budget is part of the scope. If the difficult question is whether examples are correct, whether the task is hard, or whether a model saw them during training, this report is the wrong product. It observes the file rather than answering questions the file cannot answer.
Exactly what is measured
- Parsing and schema shape: rows read, rows that cannot be parsed, fields present per row, and per-field missing, null, and empty-string counts.
- Exact duplicates: byte-equivalent or canonical-row-equivalent records, with the rule and row references disclosed.
- Normalized duplicates: records identical after a printed normalization such as Unicode NFKC, whitespace collapse, and case folding. Normalization never replaces the original observation.
- Near-duplicate candidates: either exact pairwise similarity where feasible or a seeded comparison with the seed, threshold, tokenization, and limits printed. Candidates are review prompts, not declared defects.
- Cross-split overlap: exact and normalized records appearing in both declared splits, counted per split pair.
- Literal answer or label matches: the declared answer or label string is searched in every other string field. The report enumerates the scanned fields and match rule. A zero means no literal match under that rule; paraphrase leakage is not measured.
- Label distribution: counts and shares for declared labels, by split where the file supports that distinction.
- Declared link-field completeness: empty and populated values in customer-named source and license URL fields, plus a timestamped HTTP observation for distinct public links that pass the same network-safety rules.
A 200 response states what one request observed at one time. It does not make a license valid. A 403 is recorded as 403, not as DOWN. A request that cannot tell reports UNKNOWN with the reason; unmeasurable never becomes zero, clean, or pass.
What is delivered
- A private Markdown report and machine-readable JSON findings carrying the same named observations.
- The exact commands, tool versions, seeds, thresholds, normalizers, field mappings, and other parameters used.
- The UTC fetch timestamp, final fetched URL, response observation, and whole-input SHA-256 manifest.
- Short, formula-neutralized examples only where an example is necessary to locate a finding.
The report is valid only for the printed input hash. Under the deterministic checks, the same bytes, tool version, and parameters produce the same findings. A changed URL response is a different input and therefore a different report. No customer report or customer name is published without written permission.
The deliverable contains no pass/fail score, green badge, approval, certification, or claim that the file is fit to release. A report with no findings under the named rules can coexist with a benchmark that is invalid, unsafe, trivially easy, legally unusable, contaminated, or simply uninteresting.
Intake and network boundary
You provide one public HTTPS URL and attest in writing that you have the right to submit it for this review and that the host terms permit third-party automated retrieval. That attestation is not a rights opinion from us. We can still decline a host, source, or file when the suitability is unclear.
The intake fetch rejects loopback, private, link-local, and cloud-metadata addresses after resolution, both initially and on every redirect. Redirects are capped. The byte ceiling is enforced while streaming instead of trusting a Content-Length header. Archives are never extracted. The same restrictions apply to declared source and license links, which are a separate outbound path.
Parsing occurs without network access or credentials in scratch storage. Nothing from the dataset is executed, imported as code, or deserialized through YAML or pickle. We decline executables, archives, notebooks, source trees, credentialed URLs, secrets, face or voice data, and files whose apparent terms or contents make handling uncertain. Referenced images, audio, and video are not downloaded or inspected.
Public is an access state, not a privacy clearance. If apparent personal information surfaces after ingestion, the job stops, all source bytes, working copies, reports, and request logs are purged immediately, and any fee paid is refunded in full. Detector silence is not evidence that a file contains no personal information, and the report never prints a bare zero as a PII finding.
Retention and commercial limits
Source bytes, working files, and request logs containing the source URL are deleted within seven calendar days of retrieval. The final report is retained no longer than thirty calendar days after delivery so a correction can be addressed, then deleted. Earlier deletion can be requested. A PII-aborted job follows the immediate-purge rule instead of either retention window.
The report is for the customer's internal use and is not published by US Tech Automations without written permission. The service carries no warranty. Liability is capped at fees paid for this report, consequential damages are excluded, and the work is not legal advice or a compliance review. These boundaries are part of the proposed scope, not fine print that changes after payment.
Free browser-local scope-manifest checker
Use this before requesting the report. Paste only a small JSON manifest describing the proposed scope. Never paste dataset rows, prompts, answers, labels, personal information, secrets, or credentials. The checker reports whether required fields are present and whether stated counts exceed the public scope.
JSON only. This field stays in your browser and is capped at 12,000 characters.
Why this offer is separate from a bounty
The Prime Intellect Scaling Environments program and its official public bounty sheet were reviewed on 2026-08-09. The sheet then carried application-only evaluation work marked open. That is evidence that evaluation artifacts are commissioned, not evidence that this offer has demand.
The report is independent of Prime Intellect and every other bounty platform. Buying it does not create an assignment, submit a pull request, satisfy a program checklist, influence review, cause acceptance, or trigger payout. A contributor working under third-party terms should not request it unless those terms permit third-party retrieval and review.
The METR public task and reward design guide is linked for context on evaluation-task specification. NIST FIPS 180-4 is the source for SHA-256. These sources do not endorse this service, and their methods do not turn a file-hygiene observation into benchmark acceptance.
This page is a demand test
No buyer or demand is claimed. The request form exists to learn whether a benchmark author will ask for this exact bounded report at the proposed $399 price. Submitting stores a scope request. It schedules no work, opens no checkout, creates no obligation, and uploads no dataset.
Send the public dataset URL, CSV or JSONL format, approximate rows and megabytes, at most two split names, the answer or label fields, the declared source and license URL fields, and the two attestations described above. Do not paste data. The first useful answer is in scope, out of scope, or UNKNOWN with the reason.
Related endpoint evidence
If your question is whether a public endpoint returned different bytes at two named times—not whether the file has duplicate or split-hygiene findings—use the Public Dataset Endpoint Freshness Board. It shows collector-written timestamps and the exact endpoints eligible for a bounded repeat observation.