Skip to the main content

AI access Coming soon

BotGate

Decide which crawlers and AI scrapers may read your site.

One email on the day it opens, and nothing else.

The problem

Anyone can block a crawler. Nobody on Wix can tell you it obeyed.

Wix serves your pages, so AI crawlers hit Wix's servers and those logs are not yours. Wix Analytics runs in the browser and bots do not run scripts, so a crawler never appears in it. You can write a rule into a text box and then you are done: there is nothing that tells you who read it and who ignored it.

Your robots.txt, before a rule and after one

The file Wix gives you

User-agent: *
Disallow: /checkout

No AI crawler is named.
Nothing records who came.

After BotGate

# Policy: block training, keep AI search

User-agent: GPTBot
Disallow: /

User-agent: PerplexityBot
Allow: /

One switch for each crawler, in words.

Writing the file is the easy half and the free half. The paid half is finding out whether the rule was obeyed.

What you get

Six things it does for you

A switch for every crawler

A structured editor over the raw file, so you never type a directive. Every AI crawler worth naming is in the list, grouped by what it is for, and new ones join your existing policy as they appear rather than when you notice them.

Four presets, one click each

Allow all. Block AI training. Block all AI and keep AI search. Block everything AI. The third is the one most owners want and the hardest to write by hand, because it means naming which bots are which.

See which bots actually came

A fetch count for every crawler, when each one first appeared, when it last came back and whether it is coming more often or less. Not a guess from your page analytics, which never saw a bot in its life.

Proof when one ignores you

A path that your robots.txt disallows to every crawler. Nothing a person reads links to it, so a fetch of it is a crawler that read your rules and went there anyway: a timestamp, an address and a name, from the file rather than a guess.

It checks the bot is who it says

A large share of traffic calling itself GPTBot is not GPTBot. Every hit is put through reverse DNS and checked against the IP ranges the vendor publishes, so a spoofed agent is not counted as the real one.

A guard before anything goes live

You see the exact file before it is published, and a change that would block Googlebot, hide your sitemap or shut out your own CSS is stopped and explained. One click puts back Wix's stock file if you want out.

What it looks like

The crawler list, and how a violation is proved

The crawlers BotGate ships knowing about
  • GPTBot

    OpenAI. Reads pages to train models on them.

    Training
  • ClaudeBot

    Anthropic. Same job, different company.

    Training
  • CCBot

    Common Crawl. Its archive feeds several other training sets.

    Training
  • Google-Extended

    Gemini. Separate from Googlebot, and blocking it does not affect your search ranking.

    Training
  • PerplexityBot

    Answers a question and can send a person to you.

    AI search
  • OAI-SearchBot

    The search side of OpenAI, not the training side.

    AI search
  • ChatGPT-User

    A person asked for your page while in a chat.

    AI search
  • Googlebot

    Ordinary search. Never blocked by a preset, and BotGate warns you before you block it by hand.

    Protected
  • Bingbot

    Ordinary search, on the same footing.

    Protected

A working list rather than a full one: the plan treats it as a standing commitment, because the names change every few months.

How a fetch becomes evidence
  1. Sitemap:

    An extra sitemap in your file, served by us. Bots that read your rules fetch it

  2. Disallow:

    A hidden path, disallowed to everyone. A bot that read your rules never touches it

  3. Reverse DNS

    The fetch is checked against the ranges the vendor publishes

  4. Evidence

    The time, the address and the path, for a bot told to stay out

Neither line changes what a visitor or Google sees, and taking them out stops the monitoring and nothing else.

In the free scan

3 of the 18 checks name this app

Run a scan and you will see how your own site does on exactly these, with the evidence beside each one.

  • robots.txt found Can they find your pages
  • AI crawler access Can they read your content
  • Nothing important blocked Do your images and pages hold up

Pricing

Plans

Starter

$0

No card

Starter writes the rules. Pro tells you whether anyone followed them.

Tell me when it opens

Most sites

Pro

$9.99

Per month, or $6.99 a month billed yearly

For a site whose content is worth knowing the fate of.

Tell me when it opens

Questions

Before you install

Does BotGate take over my robots.txt?

No. It writes the file through the API Wix provides for it, the same one the Wix SEO dashboard uses. Serving the file from our servers with a redirect would give us better logs, and we do not do it: Wix reserves certain paths, it fights the API instead of using it, and a misfire would break your search listing.

Can it stop a bot that ignores the rules?

No, and nothing can. robots.txt is a request with no enforcement behind it. What BotGate gives you is the evidence that a particular crawler was asked and refused, which is what you need before anyone will act on it.

Will it catch every bot?

No. The trap catches crawlers that follow links from your file. One that only fetches addresses it already holds in an index may never touch it. We would rather say that here than sell you complete coverage and be wrong.

If I block AI, do I lose search traffic?

Not from ordinary search. The crawler that ranks you in Google is a different agent from the one that trains Gemini, and the presets keep them apart. The choice worth thinking about is AI search: blocking it also blocks the answers that send people to you.

Can I change my mind?

In one click, either way. Every version of the file is kept with a diff, you can roll back to any of them, and if the file is changed outside BotGate by another app or another person, you are told.

What does the Starter plan actually do?

A robots.txt editor with syntax checking; Four one click policies; Preview the exact file before publishing; A warning if a change would block Google or your sitemap; Roll back one version; A plain English guide to every crawler; A weekly count of AI crawlers seen, without their names. Where Starter stops: Which crawlers those were; Compliance evidence; Impostor checking; Path rules. The second list is what Pro is for, because writing the rules is the easy half and knowing whether they were followed is the other.

Others in the same category

Tell me when it opens

BotGate is not in the Wix App Market yet. Leave your address and we will send one email on the day it is, and nothing else.

One email, for this app. We do not sell or share addresses.