OAI-SearchBot: what it does, and what blocking it means
The crawler behind ChatGPT's search answers. Blocking it removes you from ChatGPT citations, and it is the most damaging accidental block we see.
| Operator | OpenAI |
|---|---|
| What it does | Search & citations |
| Obeys robots.txt | Yes, per its operator's docs |
| Governs | ChatGPT Search |
| Has a crawler | Yes |
| Last verified | 2 September 2026 |
| Next review | 2 December 2026. We re-check every entry against the operator’s documentation each quarter. |
What happens if you block OAI-SearchBot?
Blocking it affects
- ChatGPT search answers and citations
It does not affect
- OpenAI model training
Is OAI-SearchBot the same as GPTBot?
This is the token that decides whether ChatGPT can surface and cite you when someone asks a question you have already answered. It is separate from GPTBot, and the two are constantly confused.
The mistake has a shape. Someone copies a "block AI scrapers" snippet from a blog post, the snippet lists every OpenAI token its author could find, and the site quietly leaves ChatGPT's index. Nothing warns you. Blocking OAI-SearchBot while allowing GPTBot is almost always backwards from what was intended.
OpenAI notes that blocked sites may still appear as bare navigation links: a name with nothing behind it, which is not the same as being recommended.
How do you spot OAI-SearchBot in your logs?
To find OAI-SearchBot in your logs, match the User-Agent shown on this page. The robots.txt token is what the crawler reads, not what appears in the request.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbotAnyone can send that User-Agent, so treat it as a claim. OpenAI's published IP ranges are the stronger way to verify the request.
Worth knowing about OAI-SearchBot
When OAI-SearchBot fetches your robots.txt itself, it sends a slightly different User-Agent with a robots.txt; marker in it. A log filter keyed on the exact string above will miss those requests.
How do you block or allow OAI-SearchBot in robots.txt?
To block OAI-SearchBot:
User-agent: OAI-SearchBot
Disallow: /To allow it explicitly, use an Allow: / rule if your file has a blanket User-agent: * disallow:
User-agent: OAI-SearchBot
Allow: /With neither rule, OAI-SearchBot is allowed by default.
Not sure how OAI-SearchBot fits with the others? Build the whole file, or see which AI crawlers to allow.
How many sites block OAI-SearchBot?
In our scan of the Tranco top 1,000 (1 to 2 October 2026, 499 sites measured), 8.2% disallowed OAI-SearchBot in robots.txt, and 10.4% refused a request identifying as OAI-SearchBot. For 6.8%, the refusal came from a server whose robots.txt allowed it: a CDN or bot rule deciding something the site’s own file does not say.
Is your site blocking OAI-SearchBot?
A CDN or bot-management rule can still block OAI-SearchBot even when robots.txt says it is welcome. You cannot see that mismatch from the front of the site.
We request your robots.txt and homepage. We read your sitemap only when robots.txt hides pages from a crawler and we need to measure how much is affected. No other page.
Guides about OAI-SearchBot
- How to block AI crawlers and stay in AI search
The robots.txt that refuses training and keeps citations, built from the registry, plus what to do about the crawlers that do not read the file. - Why your site is not showing up in ChatGPT
The common failure modes, in order, with a check you can run for each one. - GPTBot vs OAI-SearchBot: which to block?
OpenAI's two crawlers do opposite jobs. One costs you nothing to block and the other takes you out of ChatGPT's answers. - Is Cloudflare blocking AI crawlers?
A correct robots.txt can still leave a site inaccessible. Here is what to look for when Cloudflare is in the way. - How to detect GPTBot, ClaudeBot and AI bots
A user agent is a claim, not an identity. How to find each AI crawler in your logs and check it against the operator's own IP list. - Do the top 1,000 sites block AI crawlers?
What the most visited sites say in robots.txt, what their servers actually do, and how often the two disagree.
Official sources
- OpenAI’s crawler documentation
- Published IP ranges is the only way to verify a request claiming this user agent really came from OpenAI
How is OAI-SearchBot different from OpenAI’s other crawlers?
| OAI-SearchBot | GPTBot | ChatGPT-User | OAI-AdsBot | |
|---|---|---|---|---|
| Its job | Search & citations | Model training | Fetches when a user asks | Reference |
| Blocking it affects | ChatGPT search answers and citations | OpenAI model training | ChatGPT opening your page when someone asks it to (a request, not a control) | Review of the landing pages you submit as ChatGPT ads |
| Blocking it leaves alone | OpenAI model training | ChatGPT search answers and citations | ChatGPT search results; OpenAI model training | ChatGPT search answers and citations; OpenAI model training |
| Obeys robots.txt | Yes, per its operator's docs | Yes, per its operator's docs | May bypass, per its operator's docs | Yes, per its operator's docs |
| Verifiable by IP | Yes, ranges are published | Yes, ranges are published | Yes, ranges are published | Yes, ranges are published |
| Last verified | 2 September 2026 | 2 September 2026 | 2 September 2026 | 2 September 2026 |
Which other crawlers do the same job as OAI-SearchBot?
- Claude-SearchBot: Search & citations, run by Anthropic
- PerplexityBot: Search & citations, run by Perplexity
- meta-webindexer: Search & citations, run by Meta
- MistralAI-Index: Search & citations, run by Mistral
- DuckAssistBot: Search & citations, run by DuckDuckGo