Which sites block Perplexity-User?

Perplexity · declared purpose: on-demand fetch for a user's prompt · latest sealed date 2026-08-07.

On 2026-08-07, 1,594 of the 61,552 panel domains that served a parseable robots.txt name perplexity-user explicitly. 780 of them (1.3% of parsed) disallow it from the entire site, 360 restrict it to part of the site, and 454 name it and allow it. The remaining 59,958 parsed domains do not mention it at all — their wildcard group governs it.

What the denominator is. On 2026-08-07 we attempted /robots.txt on 100,000 panel domains. 24,130 could not be reached at all (DNS failure, TLS failure, timeout, refused connection) and 14,318 answered with something other than a 200. Both of those groups have no observed AI-crawler policy, and neither is counted as allowing anything — an unreachable site is UNKNOWN, not permissive. Every crawler figure below is out of the 61,552 domains that returned a robots.txt we could parse on that date.

Blocks outright — the file has a group for that exact product token containing Disallow: / with no overriding allow. Partly restricts — a group exists with at least one non-empty Disallow, but not the whole site. Names & allows — a group exists with no effective Disallow. Everything else is not mentioned: the wildcard * group governs it, and wildcard effects are counted separately rather than folded in, because a site that blocks everything has not made a decision about any AI crawler in particular. Matching is exact and case-insensitive per RFC 9309 — never substring.

Get told when Perplexity-User change their AI crawler rules

Sites publish today's robots.txt and overwrite it — no archive, no history, no notification. We keep a dated daily copy. Leave your email and we'll send you what moved for Perplexity-User: who started blocking, who stopped, and on which day.

Email me changes for Perplexity-User →

Work email only. No newsletter, no obligation.

Prefer to just buy it? See the sealed-archive monitoring offers, priced, with a dated sample.

What moved: Perplexity-User verdict changes

Counted domains whose robots.txt changed what Perplexity-User may do, between one sealed date and the previous one. Aggregate only — we do not publish which site.

Across the whole archive: 556 Perplexity-User verdict transitions, of which 84 moved TO a full block and 64 moved AWAY from one.

Sealed date	Verdict moved	Domains
2026-08-07	not mentioned → partly restricted	7
2026-08-07	not mentioned → named and allowed	5
2026-08-07	blocked outright → not mentioned	1
2026-08-06	not mentioned → named and allowed	6
2026-08-06	not mentioned → partly restricted	3
2026-08-06	named and allowed → not mentioned	2
2026-08-05	not mentioned → partly restricted	2
2026-08-05	not mentioned → named and allowed	2
2026-08-05	named and allowed → not mentioned	2
2026-08-05	named and allowed → partly restricted	2
2026-08-04	not mentioned → partly restricted	7
2026-08-04	not mentioned → named and allowed	5
2026-08-04	named and allowed → partly restricted	2
2026-08-04	named and allowed → not mentioned	1
2026-08-03	not mentioned → named and allowed	2
2026-08-03	not mentioned → partly restricted	1
2026-08-03	partly restricted → not mentioned	1
2026-08-03	named and allowed → not mentioned	1
2026-08-02	not mentioned → named and allowed	2
2026-08-02	not mentioned → partly restricted	1
2026-08-02	not mentioned → blocked outright	1
2026-08-02	named and allowed → not mentioned	1
2026-08-01	blocked outright → not mentioned	2
2026-08-01	not mentioned → partly restricted	1
2026-08-01	not mentioned → blocked outright	1
2026-08-01	not mentioned → named and allowed	1
2026-08-01	partly restricted → not mentioned	1
2026-08-01	named and allowed → not mentioned	1
2026-07-31	not mentioned → partly restricted	24
2026-07-31	not mentioned → named and allowed	3
2026-07-31	not mentioned → blocked outright	2
2026-07-31	blocked outright → not mentioned	1
2026-07-31	named and allowed → not mentioned	1
2026-07-30	not mentioned → named and allowed	5
2026-07-30	not mentioned → partly restricted	4
2026-07-30	not mentioned → blocked outright	2
2026-07-30	blocked outright → not mentioned	1
2026-07-30	blocked outright → partly restricted	1
2026-07-30	blocked outright → named and allowed	1
2026-07-30	named and allowed → partly restricted	1

Read this before quoting a trend. We hold 55 sealed daily snapshots between 2026-06-09 and 2026-08-07, out of 60 calendar days. 5 calendar day(s) inside that span have no collection run at all: 2026-06-11, 2026-06-12, 2026-06-19, 2026-07-04, 2026-07-15. Consecutive rows in every table below are consecutive SEALED dates, not consecutive days — where a gap falls, that row's diff covers more than 24 hours, and the span is printed with it. We do not interpolate across a day we did not collect.

Every sealed date

The denominator column is the number of panel domains that served a parseable robots.txt that day. It moves, so a raw count moves with it — compare the ratio, not the count.

Sealed date	Names it	Blocks outright	Partly restricts	Names & allows	Parsed that day
2026-08-07	1,594	780	360	454	61,552
2026-08-06	1,587	782	354	451	61,421
2026-08-05	1,594	795	354	445	61,527
2026-08-04	1,589	797	345	447	61,620
2026-08-03	1,584	801	339	444	61,533
2026-08-02	1,576	797	338	441	61,559
2026-08-01	1,572	797	336	439	61,326
2026-07-31	1,594	812	337	445	61,595
2026-07-30	1,557	808	312	437	61,444
2026-07-29	1,539	798	306	435	61,492
2026-07-28	1,536	800	301	435	61,562
2026-07-27	1,527	798	299	430	61,440
2026-07-26	1,530	800	300	430	61,573
2026-07-25	1,531	802	300	429	61,584
2026-07-24	1,520	799	297	424	61,576
2026-07-23	1,525	801	297	427	61,758
2026-07-22	1,522	803	293	426	61,744
2026-07-21	1,513	805	288	420	61,565
2026-07-20	1,501	800	284	417	61,665
2026-07-19	1,502	808	279	415	61,451
2026-07-18	1,503	807	281	415	61,782
2026-07-17	1,492	809	275	408	61,659
2026-07-16	1,479	806	271	402	61,526
2026-07-14	1,465	806	260	399	61,542
2026-07-13	1,461	805	261	395	61,448
2026-07-12	1,458	805	260	393	61,518
2026-07-11	1,456	804	258	394	61,526
2026-07-10	1,453	806	258	389	61,425
2026-07-09	1,450	806	257	387	61,495
2026-07-08	1,441	804	256	381	61,364
2026-07-07	1,437	804	253	380	61,491
2026-07-06	1,424	796	252	376	61,455
2026-07-05	1,424	797	251	376	61,562
2026-07-03	1,423	799	251	373	61,535
2026-07-02	1,418	798	247	373	61,553
2026-07-01	1,408	797	241	370	61,599
2026-06-30	1,405	797	240	368	61,590
2026-06-29	1,401	793	241	367	61,814
2026-06-28	1,393	791	240	362	61,835
2026-06-27	1,389	789	235	365	61,848
2026-06-26	1,390	791	237	362	61,851
2026-06-25	1,378	782	238	358	61,867
2026-06-24	1,379	790	234	355	61,660
2026-06-23	1,383	798	231	354	61,877
2026-06-22	1,380	795	231	354	61,911
2026-06-21	1,379	793	231	355	61,905
2026-06-20	1,369	789	229	351	61,754
2026-06-18	1,357	787	222	348	61,805
2026-06-17	1,365	793	225	347	61,952
2026-06-16	1,349	791	217	341	61,645
2026-06-15	1,352	793	220	339	61,715
2026-06-14	1,357	794	221	342	61,697
2026-06-13	1,352	792	220	340	61,648
2026-06-10	1,342	789	215	338	61,489
2026-06-09	1,332	787	213	332	61,599

Why you will not find a robots.txt file here. These pages render mechanical verdicts and aggregate diffs, never a fetched body. Site operators leave contact addresses in robots.txt comments — on the 2026-08-06 snapshot 2,200 of the distinct bodies we fetched contain an email-shaped string — and republishing bodies would republish those. Nothing on this surface identifies an individual site, either: every figure is a count.

Compiled from each site's own published /robots.txt, /ai.txt, /llms.txt and /.well-known/tdmrep.json, fetched daily over a frozen 100,000-domain research panel (source list: Tranco; no Tranco rank is republished here). Verdicts are mechanical readings of what those files say under RFC 9309 grouping with exact product-token matching — they are not legal advice, not a statement about any site's intentions, and not a claim about what any crawler actually did. A robots.txt is a request, not an access control. Figures are provided for informational purposes only and carry no warranty of accuracy or completeness.

AI crawler policy home · What changed · How this is collected · US Tech Automations.