Do Fishkeeping Sites Block AI Crawlers? 1 of 9 Do

Fishkeeping is one of the most open verticals in the entire snapshot. Of the 10 Fishkeeping sites we checked on 14 June 2026, 9 returned a parseable robots.txt, and exactly 1 of those 9 disallows a named AI crawler — an 11.1% block rate.

That single holdout is the whole story. Where many information-rich categories gate at least a quarter of their policied sites, the aquarium hobby leaves nearly every door open. A lone blocker against eight allowers is a vertical that has, almost unanimously, decided AI readers are welcome.

1 of 9 Fishkeeping sites blocks at least one AI crawler.

This report reads only public robots.txt files, sealed under snapshot sha 92ed5cd2858657d9. Every count below comes straight from those files; nothing is estimated, modeled, or extrapolated from traffic or rankings.

Who Gates the Crawlers Here

The one Fishkeeping site posting an AI-crawler disallow rule is fishlore.com, a long-running community forum. Forums are a recognizable shape among blockers across the wider snapshot: user-generated discussion threads are exactly the kind of dense, conversational text that training crawlers prize, and a community publisher may feel its members' posts are not theirs to hand over.

Everyone else stays open. aquariumcoop.com, aquariadise.com, theaquariumguide.com, buildyouraquarium.com, aquaticarts.com, liveaquaria.com, practicalfishkeeping.co.uk, and fishkeepingworld.com all returned a robots.txt that leaves the named AI crawlers a clear path. This mix of retailers, care-guide publishers, and enthusiast blogs all share one incentive: be the answer when someone asks how to cycle a tank.

Of the 9 Fishkeeping sites with a published policy, only fishlore.com disallows a named AI crawler.

One more site, aqadvisor.com, returned no parseable robots.txt at all. That is neither an allow nor a block — there was simply no published rule to read. We count it in the checked total but not in the block-rate denominator, which is why the rate measures against 9, not 10.

What an 11.1% Block Rate Signals

An 11.1% block rate is about as permissive as a category with a real publisher base gets. The number says one of nine policied Fishkeeping sites wrote a disallow rule for a named AI operator; the other eight did not.

The distribution matters more than the average. A single forum blocking while eight commercial and editorial sites stay open tells you the economic logic of the hobby. The sites here mostly sell livestock and gear or earn from guide traffic — both depend on being found, including inside AI answers. Closing the door would cost them the discovery they rely on.

Fishkeeping sites post an 11.1% AI-crawler block rate.

For anyone building AI-access tooling, Fishkeeping is the near-clean baseline against which gated verticals stand out. When even one site here writes a disallow rule, it is worth a second look.

The shape of the one block is the tell. fishlore.com is a forum, and forums recur among blockers across the wider snapshot because their value is the accumulated discussion of their members — content a community steward may feel is not theirs to surrender to a training crawler.

The commercial sites around it face the opposite calculus: a care guide or a product page earns nothing locked away, so leaving it open is the rational default. Read that way, the lone block is not a contradiction of the open trend but an illustration of who, specifically, has a reason to gate.

Where Fishkeeping Sits in the Corpus

Fishkeeping shares its 11.1% rate with a tight band of lightly gated categories. The focused window below centers on Fishkeeping and its nearest neighbors in the ranking, using verbatim sealed counts and naming each category rather than ranking it.

Fishkeeping sits shoulder to shoulder with Coffee, Skateboarding, and Cybersecurity — all single-blocker categories. Just below, Productivity, Marketing, and Hunting drop to one blocker out of ten policied sites. This is the calm end of the ranking, where the vertical's commercial incentive to be found outweighs any instinct to gate. The skateboarding report traces the same exact 11.1% rate from a different hobby's angle.

The extremes show how far that is from the top. A handful of categories block most of their sites, while several block none.

Fishkeeping sits much closer to the zero-block floor than to Gaming's 88.9% ceiling — a permissive vertical by any reading.

What separates the calm band from the gated top is largely the content's economic role. Gaming and News properties often run subscription or ad models where their archives are the asset, and they treat AI ingestion as leakage. Fishkeeping's policied sites run on the opposite logic: their guides and shop pages are advertisements for a purchase or a community, and being quoted is the win. The single forum in the blocker column is the exception that proves the rule, because a forum's archive behaves more like the gated-category asset than like a product page.

The neighbors reinforce this. Coffee, Cybersecurity, and Insurance all post the same one-blocker-of-nine result, and each is a vertical where most sites are either selling something or building brand authority through openly readable content. A category lands in this band when its participants would rather be cited than walled off.

Which Bots Get Disallowed Most

The category rate tells you how many sites gate; the bot leaderboard tells you which crawlers they name. These figures are corpus-wide across all 743 sites with a published policy, listed by bot name rather than rank.

CCBot, Common Crawl's fetcher, leads at 169 sites and a 22.7% share, with ClaudeBot and GPTBot close behind. These are the named tokens a site would add to its robots.txt to turn away the largest training crawlers — and the exact tokens fishlore.com would have used to land in the blocker column.

Across all 743 sites, CCBot is the single most-disallowed crawler, named by 169 sites.

The corpus-wide rate frames the contrast. While 31.1% of all policied sites block at least one crawler, Fishkeeping's 11.1% is roughly a third of that.

The leaderboard also hints at what a future Fishkeeping block would most likely target. The crawlers at the top — CCBot, ClaudeBot, GPTBot, Bytespider — are the named tokens site owners reach for first when they decide to gate, because they are the most visible large-scale fetchers.

If a second Fishkeeping site ever joins fishlore.com in the blocker column, the odds are it disallows one of these before any niche operator. That is useful for a monitoring system: the tokens worth watching are knowable in advance, so an alert can be scoped tightly rather than firing on every minor robots.txt edit.

Corpus-wide, 231 of 743 sites block at least one AI crawler.

Reading the Sealed Numbers

The method is intentionally simple. We fetched each Fishkeeping domain's public robots.txt, parsed it for disallow rules naming known AI crawlers, and recorded the outcome as a verbatim count. A site is a blocker only if its file names at least one AI operator in a disallow rule — which is how fishlore.com, and no one else, landed in that column.

We read these figures straight from the files. The 11.1% rate is a literal one-of-nine count, not a sample or a projection. When a site returned no parseable robots.txt — as aqadvisor.com did — we record that plainly and leave it out of the rate rather than guessing at a hidden policy.

The full file set was content-hashed and sealed under sha 92ed5cd2858657d9 on 14 June 2026, so the counts are reproducible by anyone re-fetching the same files. Because robots.txt changes whenever an owner edits it, only a sealed point-in-time read can state a block rate honestly; this one describes 14 June and no other day.

Fishkeeping is one named slice of a wider snapshot spanning 883 sites and 88 categories, 743 of which returned a parseable policy. Its near-open posture is a real reading of nine sites, not an inference about the hobby at large.

Key Takeaways

Fishkeeping is a near-open vertical with a single holdout. Its 11.1% rate sits well below the 31.1% corpus average, and the one forum that gates stands out precisely because everything around it stays open.

• 1 of 9 Fishkeeping sites with a policy blocks at least one AI crawler.

• fishlore.com is the only named blocker — a community forum.

• Eight sites, from aquariumcoop.com to fishkeepingworld.com, leave AI crawlers a clear path.

• aqadvisor.com returned no parseable robots.txt.

• Corpus-wide, 231 of 743 sites — 31.1% — block at least one AI crawler.

In a vertical this open, the meaningful event is the next disallow rule. That is what a monitoring workflow is built to catch.

Frequently Asked Questions

Q: Which Fishkeeping site blocks AI crawlers?

A: Just one — fishlore.com, a community forum, is the only Fishkeeping site of the 9 with a policy that disallows a named AI crawler. The other eight, including aquariumcoop.com and liveaquaria.com, leave them a clear path.

Q: Why is Fishkeeping so permissive at 11.1%?

A: Most Fishkeeping sites sell livestock and gear or earn from care-guide traffic, so visibility — including inside AI answers — is their business. Only the forum, fishlore.com, treats its members' threads as content worth gating, which is why the rate stops at 11.1%.

Q: Is fishlore.com's block actually enforced?

A: No. A robots.txt disallow is an honor-system request, not a technical barrier. Compliant crawlers respect fishlore.com's rule; nothing in the file stops a non-compliant one. The 11.1% rate measures stated intent, not enforced access.

Q: Why is the rate measured against 9 sites, not 10?

A: One of the 10 Fishkeeping sites we checked, aqadvisor.com, returned no parseable robots.txt, so there was no policy to classify. The 11.1% block rate counts only the 9 sites that published a readable file.

Put AI-Access Data to Work

A point-in-time count becomes useful when it turns into a standing watch for change.

An aquarium-ecommerce catalog manager at a retailer like aquariumcoop.com or liveaquaria.com should track whether peer shops shift from open to gated. Re-crawl the nine policied Fishkeeping domains weekly and alert the moment a current allower adds a disallow line — if competitors start gating their care guides, that reshapes who AI answers cite when a shopper asks for tank-stocking advice, and a catalog still indexed gains an edge.

A community-platform operator in the same mold as fishlore.com should monitor whether its lone-blocker stance spreads to other forums, and whether the named-bot list it disallows stays current as new crawlers appear. The trigger is any new AI token in a peer forum's robots.txt; the cadence is weekly.

A retrieval-pipeline engineer ingesting hobby content should re-check these policies before every crawl cycle, honoring the current disallow set rather than a cached copy. US Tech Automations runs this scheduled robots.txt and llms.txt monitoring, raising change alerts and tracking AI-access drift on a dashboard.

For contrast, see how chess sites gate at a higher rate and how drone sites stay fully open. To automate this watch for your own list, explore agentic workflows.

Source: US Tech Automations Research — Closing Web edition; figures are verbatim counts from public robots.txt files sealed June 14, 2026 (snapshot sha 92ed5cd2858657d9).