SEO-crawler blocking

Block the backlink crawlers — never the search engines

A first-class, per-site Footprint-Free control: block the SEO and backlink crawlers — Ahrefs, Majestic, Semrush, Moz, Screaming Frog and the rest — while the search engines keep full access. On by default for the PBN line, enforced at the server by User-Agent, driven from the dashboard and the MCP. It is the footprint game: hide the link from competitors' tools, stay indexable in the SERPs.

  • Per-site toggleeach crawler or as a group
  • Default onfor the Footprint-Free line
  • Search engines safeGooglebot never blocked
  • Server-enforced.htaccess UA-deny + robots

The footprint game, in one control

The value of a private network is the link nobody else can see. The moment a competitor's backlink tool can crawl your PBN site and follow its outbound link to your money site, the network is exposed — and the advantage is gone.

So the job is precise: shut out the backlink-analysis crawlers that build those link graphs — Ahrefs, Majestic, Semrush, Moz, Screaming Frog, BLEXBot and their kin — while leaving the search engines completely untouched, because a site that Google cannot crawl cannot rank. Block the spies, keep the indexers.

This is a per-site toggle on the Footprint-Free line, on by default. Tick or untick each crawler, or a whole group, save, and the engine writes or removes the rules for you.

Enforced at the server, not just requested

robots.txt is advisory, and the aggressive crawlers ignore it — Majestic has been observed doing exactly that. So the real block lives at the server, with robots.txt as a courtesy layer on top.

User-Agent deny at the server

The enforcement is a .htaccess, LiteSpeed or nginx User-Agent deny, written by the hosting worker and the Zinn plugin from your saved toggles. This is the rule that actually stops a crawler, because it does not depend on the crawler's goodwill.

robots.txt for the compliant ones

We also write a robots.txt Disallow for the crawlers that respect it — belt and braces, so the well-behaved tools are turned away politely and the misbehaving ones are turned away firmly.

IP and ASN deny for the worst

Some crawlers hide behind fake user-agents. For the worst offenders we can add known-IP or ASN denies, so a crawler pretending to be a browser still does not get in.

Search engines always allowed

Googlebot, Bingbot and the other indexers are never on the block list. Blocking is aimed exclusively at the backlink tools, so indexation and rankings are untouched.

You keep your Trust Flow

A common worry is that blocking Majestic will make a domain's Trust Flow disappear. It will not. Trust Flow and Citation Flow are computed from a domain's inbound backlink graph — the links pointing to it, which Majestic finds by crawling other sites — not by crawling the domain itself.

Blocking MajesticBot on your PBN site stops Majestic from seeing that site's outbound links, which is precisely the footprint you want hidden. It does not remove the domain's Trust Flow or Citation Flow from Majestic's API, so our own SEO-data checks keep reading those metrics normally. The same holds for Ahrefs DR and Moz DA — all inbound-link measures.

FAQ

Which crawlers get blocked?

The SEO and backlink crawlers — Ahrefs, Majestic, Semrush, Moz, Screaming Frog, BLEXBot and the like. The search engines are never touched: Googlebot, Bingbot and the other indexers keep full access, because staying indexable is the entire point. You can tick and untick each crawler, or a group, and save.

Will this hurt my rankings?

No. Only the backlink-analysis crawlers are blocked; the search engines that rank you are left completely alone. The site stays indexable and rankable — you are hiding your link graph from competitors' tools, not from Google.

How is the block actually enforced?

At the server, by User-Agent — a .htaccess, LiteSpeed or nginx deny — because robots.txt is only advisory and the aggressive crawlers are known to ignore it. We write a robots.txt Disallow for the compliant ones as well, and can add known-IP or ASN denies for the worst offenders that crawl under fake user-agents. The engine writes and removes the rules automatically from your saved toggles.

If I block Majestic, do I lose my Trust Flow?

No. Trust Flow and Citation Flow are computed from a domain's inbound backlink graph — links to it, found by Majestic crawling other sites — not by crawling the domain itself. Blocking MajesticBot on the site stops Majestic seeing its outbound links, which is the footprint you want hidden, but leaves the domain's metrics in Majestic's API intact. The same logic applies to Ahrefs DR and Moz DA.

Is this on by default?

On the Footprint-Free line, yes — crawler blocking is on by default, because hiding the PBN-to-money-site link is the core of the footprint game. You keep full control to adjust which crawlers are blocked per site.

Hide your link graph, stay indexed

Start a card-free 7-day trial and block the backlink crawlers per site while the search engines keep full access — on by default for the Footprint-Free line, controlled from the dashboard and the MCP.

Start free