Claude-User
The fetcher Claude uses when a user asks about your specific page or product. Blocking it means Claude cannot open your site on a user's behalf.
| Operator | Anthropic |
|---|---|
| What it does | Fetches when a user asks |
| Obeys robots.txt | Yes, per its operator's docs |
| Governs | Claude |
| Has a crawler | Yes |
| Last verified | 13 August 2026 |
What blocking it costs you
What to know
This runs when a person asks Claude about you by name or by URL. It is one request for one page, triggered by a human, not a crawl of your site.
Anthropic documents that this fetcher honours robots.txt — which is worth noting, because the equivalent fetchers at some other operators do not.
The robots.txt rules
To block Claude-User:
User-agent: Claude-User
Disallow: /To allow it explicitly — worth doing when your file also contains a blanket User-agent: * disallow, since the most specific matching group wins and a token with no group of its own falls back to the wildcard:
User-agent: Claude-User
Allow: /If neither rule is present, Claude-User is allowed. That is the default, and it is why an accidental block is nearly always something that was added rather than something that was forgotten — usually a snippet copied from a blog post that listed more tokens than the author intended to block.
Is it blocked on your site?
Your robots.txt is only the half you can read. A CDN or bot-management rule can turn Claude-User away while your file still says it is welcome, and that mismatch is invisible from the front of the site. We check both layers separately and tell you which one is doing what.
We request two things: your robots.txt and your homepage. No other pages, ever.
Official sources
- Anthropic’s crawler documentation
- Published IP ranges — the only way to verify a request claiming this user agent really came from Anthropic
Other Anthropic tokens
- ClaudeBot — Model training
- Claude-SearchBot — Search & citations