Knowledge base

Bad and fake bots are blocked on every site

Bad bots and fake crawlers are blocked on every site and server by default, at the edge and on the server. Real search and AI crawlers always get through, and you choose per site whether AI crawlers may read it.

Zinn®, our Viking, reading his checklist beside an index of articles with one open.

Every site and every server we host blocks bad bots and fake crawlers by default. You do not need to switch anything on, and it costs nothing extra.

What is blocked?

  • Known bad bots: scrapers, content thieves, spam tools and vulnerability scanners that announce themselves by name.
  • Fake crawlers: programs that pretend to be a well-known crawler, such as Googlebot, Bingbot, Applebot, GPTBot or ClaudeBot, to get past your defences. A request that claims to be one of these but does not come from that company's own published network addresses is refused.

The check happens twice: on our edge network before a request reaches your server, and again on the web server itself, so a request that goes straight to your server is checked too.

What is never blocked?

Real, verified crawlers always get through. Googlebot, Bingbot, Applebot and the real AI crawlers (such as GPTBot and ClaudeBot) are recognised by where they come from, not just by the name they give, so your site stays in search results and in AI answers.

Can I block AI crawlers on my site?

Yes, and the choice is yours for each site. Open Sites, choose the site, open the CDN tab and find Security and bot control → AI crawlers:

  • No special treatment: AI crawlers are treated like any other visitor.
  • Allow them: AI crawlers are let through.
  • Challenge them or Block them: AI crawlers are challenged or refused at the edge. These two options come with the bot protection included in your plan; the screen shows if yours does not include it.

We never change this setting for you. Blocking fake bots does not touch it: a real AI crawler is only blocked if you choose to block it.

Something legitimate was refused

Uptime monitors, payment callbacks and SEO tools are not affected unless they pretend to be one of the crawlers above. If a service you use is refused, open a support ticket with the time and the address it came from and we will look at it.

Latest from the blog

What we have been writing about hosting, SEO and running sites at scale.

SEO and Link Building from the Hosting Layer: A 2026 Operator's View

How hosting shapes indexing and link equity in 2026: keeping pages indexed, vetting aged domains before you build on them, link building without a footprint, and an honest line on what infrastructure can and cannot do for SEO.

Read the post →

Making WordPress Fast and Secure: A Performance and Plugin Checklist

A practical checklist for fast, secure WordPress: server-level caching, a per-site object cache, the handful of plugins worth running, keeping the stack current, and the WooCommerce pages you must never cache.

Read the post →

How to Choose Managed Web Hosting in 2026: A Buyer's Guide

What actually separates good managed hosting from a cheap box with a control panel — migrations, backups, isolation, real caching and honest scaling — and how to judge it before you commit.

Read the post →

Read the blog →

Still stuck?

Support is included on every plan, the desk is open 24 hours a day, and you can write to us in any of our 58 languages — we answer you in yours.

Contact support All articles