Can AI Read Your Software Vendor's Website in September?

Can AI read your software vendor’s website in September 2026? Of the 30 named products asked, 26 returned a readable robots.txt and 5 named any AI company at all. Of those 26, 1 blocked any AI company. Of those 26, 21 named none.

This is the dated September 1, 2026 edition of the census, not the series home. The evergreen question-and-method page is 30 software vendors, one AI-crawler question. A sealed snapshot is a frozen reading of each vendor’s own published robots.txt on one day, scored only for 11 crawler names from 6 AI companies.

26 of 30 named products returned a readable robots.txt.

Thirty named vertical business-software products, five in each of six industries (Insurance agency software; Accounting and tax software; Healthcare practice software; Legal practice software; Property and construction software; Field service software), asked one question each: what does the robots.txt you publish say about 11 named AI crawlers from 6 AI companies.

The cohort is US Tech Automations’ own named list, written out in the builder, and is not a ranking, a sample, or anything derived from another publisher’s data.

Thirty named companies are not an industry. Nothing here is representative of insurance software, accounting software, or the web, and no figure may be presented as a rate for anything wider than these thirty. A vendor that never names an AI company has NOT allowed it — unmentioned is its own answer and must never be reported as allowed.

A vendor whose address returned an HTML page instead of a plain-text file has not answered at all; it is unknown, never open. robots.txt is a request about crawling, not a licence and not permission to use content, and a vendor blocking a crawler is not evidence about its API, its integrations, or its product. This edition is one reading on one day; a change between editions is a change in what the vendor published, not proof of intent.

5 of 26 answering vendors named any AI company.

Key Takeaways

• 30 asked, 26 answered, 4 unreadable on September 1, 2026.

• 5 of 26 named any AI company; 1 of 26 blocked any.

• 21 of 26 named none — 80.8% of the 26, cohort 30.

• Field service software: 60% of those 5 name an AI company.

• Legal practice software: 0 of 5 named any AI company.

• Accounting and tax software holds the 1 blocker row.

Where Each Industry Row Lands

The comparison is the story of this edition. Six shelves, five named products each, same question, same day. Every share is over the vendors that answered on that row, and the cohort remains 30. Unmentioned is still not allowed.

Two shelves actually name crawlers. Field service software: 3 of 5 name an AI company, 0 of 5 block any, a 60% naming share of those five. Accounting and tax software: 2 of 5 name any, 1 of 5 blocks any, a 40% naming share of those five. Walk those products on the field service software page and the accounting and tax software page.

Four shelves name no AI company among the files that answered. Legal practice software is complete silence with a complete file set: 5 of 5 answered, 0 named any. Insurance agency software and healthcare practice software each have 4 of 5 readable and 0 naming. Property and construction software has the thinnest file set: 3 of 5 answered, 2 unreadable, 0 naming.

60% of the five field service software products name an AI company; 0 of those 5 block any.

0 of 5 legal practice software products named any of the 11 crawler names.

A 0% naming share is not an industry that “allows everything.” It is five (or fewer readable) files that never used the 11 tokens. Putting that next to field service software’s 60% of five is the September 1, 2026 picture. It is not a trend.

Crawler by Crawler on This Reading Day

The distinctive crawler fact on this day is not a wave of blocks. It is three tokens with a blocked count of 1 — Google-Extended, CCBot, and Applebot-Extended — against a wall of unmentioned counts in the 21–24 range, always over the 26 that answered, cohort of 30.

OAI-SearchBot and PerplexityBot share an allowed count of 5. Perplexity-User is allowed in 2 files and unmentioned in 24. GPTBot, ChatGPT-User, and ClaudeBot each sit at blocked 0, allowed 4, unmentioned 22. Claude-User and Claude-SearchBot sit at blocked 0, allowed 3, unmentioned 23.

robots.txt is an honor system: a Disallow: / is a request, not an enforcement. This table does not say whether any crawler complied, whether a page was cited, or whether a ranking moved.

Unreadable Hosts, Named

The 4 unreadable products are unknown, never open, never closed. They are outside the 26-vendor shares. Name them and what came back.

A 200 with an HTML body is not a robots.txt. A 403 is not a GPTBot rule. llms.txt is not a stand-in for the missing file. Property and construction software is the shelf with 2 of these 4 unknowns, which is why its 3 readable silences cannot be inflated into a construction-software rate.

Tokens the Companies No Longer List

On this edition, a legacy Anthropic token recorded as anthropic-ai is blocked in 1 readable file. Claude-Web is blocked in 0 readable files. Those strings were not on the 11-name ask. They are leftover names still sitting on a published file.

A leftover token is a change in what was published only when a later edition is sealed against this one. This page does not make that comparison. One reading, one day.

Reading Method for This Edition

Read from each vendor’s own published robots.txt on September 1, 2026 as USTA-DataResearch. The public description of that research lane is on the research index. Figures here are copied from the sealed snapshot; nothing is estimated, modeled, or extrapolated.

Thirty named vertical business-software products, five in each of six industries (Insurance agency software; Accounting and tax software; Healthcare practice software; Legal practice software; Property and construction software; Field service software), asked one question each: what does the robots.txt you publish say about 11 named AI crawlers from 6 AI companies. The cohort is US Tech Automations’ own named list, written out in the builder, and is not a ranking, a sample, or anything derived from another publisher’s data.

Thirty named companies are not an industry. Nothing here is representative of insurance software, accounting software, or the web, and no figure may be presented as a rate for anything wider than these thirty. A vendor that never names an AI company has NOT allowed it — unmentioned is its own answer and must never be reported as allowed.

A vendor whose address returned an HTML page instead of a plain-text file has not answered at all; it is unknown, never open. robots.txt is a request about crawling, not a licence and not permission to use content, and a vendor blocking a crawler is not evidence about its API, its integrations, or its product. This edition is one reading on one day; a change between editions is a change in what the vendor published, not proof of intent.

The Productivity sites crawler audit is a different named cohort. Do not blend it with these 30 products.

1. Ask the same 30 hosts. Five products in each of 6 industries, listed in the builder, not drawn from another publisher.

2. Score three answers. Blocked, allowed, or unmentioned for each of 11 tokens. HTML or a challenge is unreadable, not a hidden allow.

3. Publish the dated freeze. Keep the 26-of-30 denominator on every share. Seal the industry rows as they stood on September 1, 2026.

80.8% of 26 answering vendors named no AI company, cohort 30.

Frequently Asked Questions

Q: Why does unmentioned dominate this September 1, 2026 reading?
A: 21 of 26 answering vendors, in a cohort of 30, never named one of the 11 crawler names. That is silence, not permission. Only 5 of 26 named any AI company. Most files simply do not discuss these tokens.

Q: Does a Disallow line actually stop a crawler?
A: robots.txt is a request. This edition records what the vendor published, not whether a crawler obeyed, not whether a page was used for training, and not whether an assistant cited the vendor. A block is not enforcement.

Q: Why are the denominators 26 and 30 on the same page?
A: 30 is the cohort asked. 26 is the set that returned a readable robots.txt. Every share, including 80.8%, is over the 26 that answered. The 4 unreadable hosts are named and left unknown.

Q: Is 60% of field service software an industry rate?
A: No. 60% is 3 of 5 named field service software products. Five products are not an industry. The same-day legal practice software row is 0 of 5. Both facts are about those named products only.

Q: What did the 4 unreadable hosts return?
A: NowCerts answered 200 as text/html. athenahealth, Yardi, and Buildertrend each answered 403 as text/html. None of those is a plain-text robots.txt, so none is scored as blocked or allowed.

Put AI-Access Data to Work

An insurance-agency operations lead running HawkSoft, EZLynx, or Applied Systems (Applied Epic) is looking at 4 readable files that name 0 AI companies, plus NowCerts unknown. The recurring job is a weekly re-read of those five addresses, with an alert the moment a file that currently leaves 11 names unmentioned adds a named group.

A construction-firm systems owner watching Yardi, Procore, or Buildertrend has a thinner file set: 3 of 5 readable, 2 unknown. The trigger is any of those 3 silences gaining a Disallow: /, or either unknown host publishing a parseable file for the first time.

A competitive-intelligence analyst covering vertical SaaS should watch the two shelves that already name crawlers — field service software at 60% of five, accounting and tax software at 40% of five — because that is where published policy already exists to drift.

US Tech Automations automates those re-reads, hashes each file, and routes a change into an agentic workflow. Start from the company homepage if you need the public door.

Source: US Tech Automations Research — computed from the sealed vendor AI-access snapshot, September 1, 2026.