CloudyBot SEO Audit crawler

This page documents the HTTP crawler used by CloudyBot’s SEO Audit Pack and the free one-URL page check. It is operated by CloudAxis Labs LLC. We do not run this bot to index the open web, train models, or monitor sites without a user request.

User-Agent string

Look for this token in your access logs:

Mozilla/5.0 CloudyBot-SEO-Audit/1.0 (+https://cloudybot.ai/bot)

The +https://cloudybot.ai/bot suffix points here for operators who want policy details.

When we fetch your site

• Paid SEO Audit Pack — a customer enters their own domain and starts an audit. Our server may fetch robots.txt, sitemap.xml, llms.txt, and crawl internal HTML pages (default cap 120 URLs per run) to build inventory for the report.
• Free one-URL check — a visitor submits one public URL. We send a single GET, read title/meta/H1/canonical from the HTML, and return the result. No multi-page crawl.

We do not schedule background revisits. If nobody starts a check or audit against your domain, this bot should not appear in your logs.

robots.txt

We parse robots.txt for rules targeting CloudyBot-SEO-Audit and the generic * group before crawling. If you disallow us, paid crawls stop at disallowed paths and the free tool will not fetch a blocked URL.

Example block:

User-agent: CloudyBot-SEO-Audit
Disallow: /

You can also disallow * if that matches your policy for all bots. See your own cloudybot.ai robots.txt for how we treat other crawlers on our marketing site.

Crawl behavior (paid audits)

• Same registrable domain as the URL the customer submitted — we do not follow external sites as crawl targets.
• Fetch-based inventory (HTML + headers), not a headless browser for the bulk crawl.
• Rate-limited requests (server-side spacing between URLs) to avoid hammering small hosts.
• Standard HTTP redirects are followed; disallowed paths are skipped.

What this bot is not

• Not the CloudyBot product browser sessions customers use inside the dashboard (those are separate, user-driven automation).
• Not Googlebot, Bingbot, or an AI training crawler — we do not add your pages to a public search index.
• Not a marketing scraper — no continuous monitoring of competitor sites unless a paying customer runs an audit on their own property.

Contact & abuse

Questions about crawl traffic, false positives, or a mis-attributed request: info@cloudybot.ai with the domain, timestamp, and request path from your logs. General product support → contact. Security disclosures → security.