hosting
GET /v1/sites/{siteId}/crawler-block
Get a site's blocked-crawler selection.
Authentication
Send an API key as a bearer token. The key must carry the sites.view permission; a key without it is refused with 403, not 404.
This endpoint takes no organisation id. Your key already identifies the organisation it belongs to, and the response is scoped to it.
Try it
Replace anything in angle brackets with your own values, and the key placeholder with a key from your dashboard.
curl -X GET https://api.zinndigital.com/v1/sites/{siteId}/crawler-block \
-H "Authorization: Bearer zdk_live_…"Signed in? The API console in your dashboard fills in your real organisation id and your own key, and runs the request against the live API so you can see the actual response. Open this endpoint in the API console
Details
A second representation of the same policy as `crawler-policy`, not a second setting: this one carries just the customer-facing selection, while `crawler-policy` additionally returns the rendered server rules that MCP and the hosting worker consume. `available_bots` is the catalog of blockable SEO crawlers, which by construction contains no search engine — so a customer cannot de-index themselves here, because Googlebot and Bingbot are not in the list to choose from. Requires `sites.view`.
Parameters
| Name | Type | Required | What it is |
|---|---|---|---|
siteId (path) | Uuid | Yes | Site ID (UUIDv7). |
Response
| Name | Type | Required | What it is |
|---|---|---|---|
site_id | Uuid | Yes | UUIDv7 identifier — sortable by creation time (docs/02 §8). |
blocked_bots | string[] | Yes | Crawler keys currently blocked at the server level. |
available_bots | string[] | Yes | The crawlers the customer may block. Contains no search engine, which is what makes de-indexing yourself impossible through this surface. |
enforcement | CrawlerEnforcement | No | Whether the stored crawler policy is actually a rule on a web server, and if not why not (#1954, D8300). ⛔ These are two different questions and collapsing them is the defect th… |
Errors this endpoint can return
401 · 403 · 404 · 429