Do Board Game Sites Block AI Crawlers? 2 of 8 Do

The board-game web is mostly open to AI. Of the 10 Board Game sites we checked, 8 returned a parseable robots.txt, and only 2 of those disallow even one AI crawler. The most interesting detail is not who blocks — it is that two prominent publishers serve no robots.txt at all.

2 of 8 Board Game sites block at least one AI crawler.

A robots.txt file is the plain-text rulebook a site publishes telling automated crawlers which paths they may fetch. We read those files directly — nothing is estimated, modeled, or extrapolated. With a low block rate and a pair of policy-silent publishers, this category sits among the more permissive slices of the snapshot.

The distinctive read here is not the block rate at all — it is the silence. In most categories every site we checked had published some robots.txt, so the only question was whether it gated AI. Board Games breaks that pattern: two of its most recognizable publishers, the companies behind some of the best-known modern tabletop lines, serve no machine-readable policy whatsoever.

That absence is easy to skim past in a headline number, but it changes how the category should be read. A vertical can look open simply because much of it has not yet decided to engage with the question.

Who Gates the Crawlers Here

Only two sites carry an AI-crawler disallow rule: boardgamegeek.com, the genre's dominant database and forum, and dicebreaker.com, an editorial site. Both sit on large user-generated or original-content archives, which is the usual reason a site decides to gate. boardgamegeek.com in particular hosts years of community discussion, ratings, and game metadata that exist nowhere else in the same form, so a disallow rule there protects a genuinely unique corpus. dicebreaker.com's reason is more conventional — it is a publication guarding original reviews and guides.

Six sites allow every crawler we tested: shutupandsitdown.com, tabletopgaming.co.uk, wizards.com, cmon.com, meeplemountain.com, and boardgamequest.com. That open group spans reviewers, a major publisher, and community sites — so a permissive policy is the default posture across most of this vertical.

It is worth noting that boardgamegeek.com, the one database giant that does block, is the closest thing this category has to a content monopoly: its forums, ratings, and game pages are the reference layer for the entire hobby. A site sitting on that much irreplaceable user-generated data has the strongest incentive to gate, which is why its decision stands out against six open peers. For a vertical where the dominant site reaches the opposite conclusion, the woodworking report shows publishers split right down the middle.

Two prominent publishers, asmodee.com and fantasyflightgames.com, served no robots.txt at all.

Board Game sites post a 25% AI-crawler block rate.

What This Block Rate Actually Means

Two of the 10 sites we checked — asmodee.com and fantasyflightgames.com — returned no robots.txt file, so they sit outside the policy count entirely. A missing file is not a block and not a deliberate allow; it simply means the site published no machine-readable instruction, and compliant crawlers treat unlisted paths as fetchable by default.

That detail reframes the headline. The category is not just lightly gated; a meaningful share of its publishers have not engaged with the robots.txt convention at all. For anyone reading AI-access posture in this space, "no policy" is its own signal — and a quieter one than an explicit allow.

Reading the category well means holding three states apart. Two sites block, six sites publish a file that allows everything, and two sites publish nothing. The blockers have made a deliberate choice; the allowers have made an explicit open declaration; the policy-silent pair have simply left the default in place, which compliant crawlers treat as permission.

Lumping the last two groups together would overstate how intentional the openness is. Asmodee and Fantasy Flight are large publishers, and their silence may reflect priorities elsewhere rather than a considered AI stance — which is exactly the kind of gap that can close the day someone on their web team revisits it.

That makes the board-game category a clean teaching case for why a single block-rate number understates the picture. The 25% counts only the sites that published a policy; it says nothing about the two that did not. A reader who wants the full posture of the vertical has to count the silence as well as the rules.

Where Board Games Sit in the Corpus

Across the snapshot, 196 of 614 sites with a published policy block at least one AI crawler — a 31.9% corpus rate. Board Games' 25% lands below that line, clustering with the craft-and-hobby verticals rather than the heavily gated publisher categories.

The neighbors confirm the placement. Crafts, InteriorDesign, and Space share the same 25% mark, while Pets sits a step above and HR, Skiing, and Podcasts a step below. These are hobby, lifestyle, and service verticals — none of them media-heavy — and Board Games belongs comfortably among them. That ordinariness is itself the read: this is a stable, low-conflict category around AI access, not one in the middle of a gating wave.

For scale, the corpus runs from a heavily gated top to a fully open floor.

Which Bots Are Blocked Most

When a board-game site does write a disallow rule, it tends to name the same crawlers that dominate the corpus-wide picture. The bot leaderboard across all 614 sites shows CCBot in front, with the major model-builder agents close behind.

CCBot leads because Common Crawl's archive feeds many downstream datasets, making it the highest-leverage single bot to disallow. ClaudeBot and GPTBot follow, so the two board-game blockers are almost certainly naming this same front tier rather than fringe agents. The skiing breakdown covers another low-block hobby slice for comparison, and the podcast report reads the same leaderboard against an even more open category.

Across all 614 sites, CCBot is the single most-blocked bot at 145.

How the Snapshot Was Sealed

Our research team fetched each site's robots.txt at one point in time, parsed the user-agent and disallow directives, and recorded which AI crawlers were named. Every figure follows the honesty rule: nothing is estimated, modeled, or extrapolated. A site counts as a blocker only when its own file disallows a known AI user-agent.

The corpus spans 725 sites checked, 614 with a parseable robots.txt, across 72 categories. Separately, 141 sites publish an llms.txt file — 23% of those with robots — a newer convention for declaring AI-access intent. The snapshot is content-addressed under sha 77d0521dc8809a6c, so every count here can be reproduced exactly.

Corpus-wide, 196 of 614 sites block at least one AI crawler.

Because robots.txt is editable in seconds, this 25% is a single-day reading. The two policy-silent publishers could add a file tomorrow, and either current blocker could open — which is why the durable value is in watching the file change over time.

Frequently Asked Questions

Q: What does it mean that two board-game sites have no robots.txt?

A: asmodee.com and fantasyflightgames.com returned no file, so compliant crawlers treat their paths as fetchable by default. It is neither a block nor a deliberate allow — just an absence of any machine-readable instruction. We count them separately from the 8 sites that published a policy.

Q: Does a disallow rule actually stop a crawler?

A: No. robots.txt is an honor-system standard. Well-behaved crawlers respect it, but the file enforces nothing on its own. Hard enforcement requires server-side blocking. Our report measures stated policy, not compliance.

Q: Why is the board-game block rate below the corpus average?

A: At 25%, Board Games sits under the 31.9% corpus rate. Most of the vertical is reviewers, retailers, and community sites that benefit from broad discoverability, so few of them gate. Only the genre's largest database and one editorial site block. Counting the two policy-silent publishers, the real share of sites actively restricting AI is small either way.

Q: How would a publisher act on this data?

A: By monitoring its own and its peers' robots.txt on a schedule. A publisher can confirm its file still names the operators it intends, and watch whether competitors begin gating — an early signal of shifting industry norms around AI access.

Q: How is a sealed snapshot different from just checking the sites today?

A: A sealed snapshot is frozen and content-addressed under sha 77d0521dc8809a6c, so the exact counts can be reproduced. Checking today reads whatever the files say at that moment. The value of the snapshot is as a baseline: later reads compared against it reveal precisely which site changed its policy and when.

Q: Why publish a category report on something this fast-changing?

A: Because the baseline is what makes change legible. A point-in-time count of 2 of 8 is only mildly interesting on its own; it becomes valuable when a future read shows the number moving. The report exists to anchor that comparison, not to claim the figure is permanent.

Put AI-Access Data to Work

A board-game publisher community lead — the kind of role inside companies behind cmon.com or wizards.com — can run this as a standing watch: re-crawl boardgamegeek.com, dicebreaker.com, and the two policy-silent publishers weekly and get alerted the instant any of them adds an AI-crawler token to a disallow list, since a competitor closing its archive reshapes how AI assistants surface the category. A storefront merchandiser at a tabletop retailer can track whether rivals are gating product-content paths from AI shopping agents. A retrieval-systems engineer can watch the corpus bot leaderboard for threshold shifts in CCBot or GPTBot blocks.

Each is a recurring, automatable job: the snapshot count anchors it, and the value is detecting drift on a fixed cadence. US Tech Automations automates that monitoring with scheduled robots.txt and llms.txt crawls, change alerts, and an AI-access policy dashboard. See how the workflow runs.

Key Takeaways

Board Games is a permissive category: 2 of 8 policy-bearing sites block an AI crawler, a 25% rate below the 31.9% corpus line. The sharper finding is that two prominent publishers serve no robots.txt at all, so "no stated policy" is the category's quiet signal. As an editable single-day reading, the durable insight comes from watching it change — the recurring monitoring US Tech Automations runs.

Source: US Tech Automations Research — Closing Web edition; figures are verbatim counts from public robots.txt files sealed June 14, 2026 (snapshot sha 77d0521dc8809a6c).