meta-externalads: what it does, and what blocking it means

Meta's crawler for advertising and business products. It reads the landing pages behind ads you are paying for, and blocking it costs you rather than Meta.

What happens if you block meta-externalads?

Should you block meta-externalads?

This is not a discoverability crawler and it has no bearing on whether Meta AI can cite you. It crawls for advertising and other business products, which in practice means the pages you point your own campaigns at.

It matters here because of what a blanket rule does. A "block every Meta bot" snippet catches this token along with the training one, and the site owner ends up degrading the performance of ads they are still buying, for no privacy or policy gain. OpenAI's OAI-AdsBot creates the identical trap one operator over, which is why both are in this registry even though neither belongs on a report card.

If you want out of Meta's training data, that is meta-externalagent. This token is a separate decision, and for most sites running Meta ads the answer is to leave it alone.

How do you spot meta-externalads in your logs?

To find meta-externalads in your logs, match the User-Agent shown on this page. The robots.txt token is what the crawler reads, not what appears in the request.

meta-externalads/1.1 (+https://developers.facebook.com/documentation/sharing/webmasters/web-crawlers)

Anyone can send that string, and Meta publishes no address ranges for meta-externalads, so there is no way to confirm from outside that a request carrying it is really theirs.

How do you block or allow meta-externalads in robots.txt?

To block meta-externalads:

User-agent: meta-externalads
Disallow: /

To allow it explicitly, use an Allow: / rule if your file has a blanket User-agent: * disallow:

User-agent: meta-externalads
Allow: /

With neither rule, meta-externalads is allowed by default.

Not sure how meta-externalads fits with the others? Build the whole file, or see which AI crawlers to allow.

Is your site blocking meta-externalads?

A CDN or bot-management rule can still block meta-externalads even when robots.txt says it is welcome. You cannot see that mismatch from the front of the site.

We request your robots.txt and homepage. We read your sitemap only when robots.txt hides pages from a crawler and we need to measure how much is affected. No other page.

Official sources

• Meta’s crawler documentation

How is meta-externalads different from Meta’s other crawlers?

← All AI crawlers