Find out when it breaks, not months later

A scan tells you what is true today. The problem is that it can change tomorrow: a release adds noindex, a CDN setting changes, or someone pastes the wrong robots.txt rule. Run one command in CI or on a schedule and you can catch that change.

npx deployradar https://example.com --watch .deployradar/example.json

The first run creates a baseline and exits 0. Later runs compare against it and replace it with the new baseline. Exit 1 means access got worse, 0 means it held or improved, and 2 means the scan was not reliable enough to judge, so nothing is stored.

MonBaseline First run. Every crawler gets through.exit 0 · baseline recorded
TueHeld Nothing changed.exit 0 · baseline replaced
WedRegression A deploy changed a CDN rule. OAI-SearchBot now gets a 403.exit 1 · baseline replaced, so it is reported once
ThuUnreliable robots.txt answered 503. Nothing can be judged.exit 2 · baseline kept: an outage is not stored
FriHeld Still a 403, compared with Wednesday. Already reported.exit 0 · baseline replaced
MonImproved The rule was fixed. OAI-SearchBot gets through.exit 0 · baseline replaced

An example week. The regression fails one build, not every build after it, and the outage on Thursday neither fails the build nor overwrites the last good baseline.

In GitHub Actions

name: AI access
on:
  schedule:
    - cron: "0 7 * * 1"   # Monday morning, before anyone reads it
  workflow_dispatch:

jobs:
  watch:
    runs-on: ubuntu-latest
    permissions:
      contents: write     # to commit the updated snapshot
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22

      # Exits 1 when AI access got worse, 0 when it improved or held,
      # and 2 when the scan was not reliable enough to conclude anything.
      - run: npx deployradar https://example.com --watch .deployradar/example.json

      # Only reached when nothing got worse, so the snapshot that gets
      # committed is always one that was compared against.
      - name: Keep the new baseline
        if: success()
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git add .deployradar
          git diff --quiet --cached || git commit -m "AI access baseline"
          git push

Or drop it into the deploy pipeline you already have, against whatever URL you just shipped:

- run: npx deployradar $DEPLOY_URL --watch .deployradar/prod.json

What fails the build

A crawler that was allowed is now disallowed in robots.txt	The commonest one, and almost never deliberate. It usually arrives as a snippet somebody copied to “block AI” that also listed the citation crawlers.
A crawler that was getting through is now turned away at the edge	Your CDN or bot-management rules, not your robots.txt. This is the layer a file cannot show you and the one that changes without a commit.
Content paths a crawler could reach are now excluded	Your homepage still answers, so nothing looks wrong. Every article underneath has left the index.
Server-rendering parity dropped	A framework upgrade or a component moved client-side, and the crawlers that do not run JavaScript now see less of the page than they did.

What deliberately does not

This list is the harder half, and it is why the check is worth having in a pipeline at all. A monitor earns its place by being right on the day it fires, and every false alarm spends some of that.

The scan could not read your robots.txt	A 503 for one minute turns every token's policy source to “none” at once. Diffed naively that is a site-wide event that did not happen, so nothing is concluded and the stored baseline is left alone, with exit code 2, neither pass nor fail.
We added a crawler to the registry	A new token appears in a scan having never been absent from your site. It is reported as ours, and it never fails a build.
Something improved	Reported, and exits 0. A monitor that fails on good news is a monitor people mute, and a muted monitor is worth nothing on the day it is right.
Parity moved a point or two	A rotating testimonial moves that number. Only a real drop counts.

Where the snapshot lives

The snapshot lives in your repository next to your code. It is a small JSON file with the policy, enforcement verdicts, excluded paths and scan coverage for the 26 tokens. It contains no page content or HTML samples.

Because the snapshot stays with you, we do not know that you ran the check, what it found or even that your site exists. It is the right option if you want the check without sending results to us.

Or let us watch it

Scan your site and there is a box at the end of the report asking for an address. Confirm it, and the same check runs here every week and emails you when an answer changes. No repository, no pipeline, nothing to install.

Monitoring is different: we keep a small snapshot so we can compare future scans. It is kept only while monitoring is active, and the scanner identifies itself as DeployRadarMonitor. You can stop it with one robots.txt rule, whether or not you are the person who signed up.

Without installing anything

The JSON API answers the same scan over HTTP, so a watcher in any language can poll it and keep its own history. Results are cached 24 hours per domain and every response says how old it is, so a daily check costs one scan.

← DeployRadar