privateSEO
Join Waitlist
Crawler information

Meet PrivateSEOBot.

PrivateSEOBot is the identifiable, user-initiated crawler used by the privateSEO desktop application to audit websites for technical and on-page SEO problems.

User-agent string

Mozilla/5.0 (compatible; PrivateSEOBot/1.0; +https://privateseo.app/crawler)

Use the case-insensitive product token PrivateSEOBot in robots.txt rules.

Operated by privateSEO

Started by a user

PrivateSEOBot does not continuously discover and crawl the web. A scan begins only when a person enters a website in the desktop app.

Runs from the user's computer

Website requests originate from the SEO professional's own internet connection. PrivateSEOBot therefore has no central or fixed IP range.

Analyzed locally

Fetched content is analyzed and saved on the user's device. Website content is not uploaded to a central privateSEO crawling service.

What PrivateSEOBot requests

During an authorized website audit, PrivateSEOBot may request publicly reachable resources needed to discover pages and calculate the report:

  • HTML pages and the HTTP headers returned with them;
  • /robots.txt, declared sitemaps, and common sitemap locations;
  • same-domain URLs found in links, canonicals, and hreflang annotations;
  • image resources referenced by scanned pages, within the active scan limit;
  • /llms.txt when the AI Visibility audit is enabled.

External page links are recorded for reporting but are not followed as part of the website scan. An external image host can receive image requests when a scanned page embeds its images; that host's own robots.txt policy is checked separately.

robots.txt and request pacing

PrivateSEOBot follows the rules for PrivateSEOBot, falling back to the wildcard * group when no crawler-specific group exists. It evaluates Allow and Disallow rules before requesting a resource and checks redirect destinations before following them.

  • A maximum of two requests can run concurrently for the same host.
  • Request starts are separated by at least 500 milliseconds per host.
  • A valid Crawl-delay value can request a longer interval.
  • HTTP 429 and 503 responses trigger shared per-host backoff and retry behavior.
  • Disallowed URLs may be reported as discovered, but they are not requested.

Authorized override: the desktop app provides a clearly warned, one-scan override for owners and professionals who have explicit permission to audit a protected site. Throttling and technical safety limits remain active when that override is used.

How to block PrivateSEOBot

Website owners can block the crawler across the entire origin by adding this group to the origin's /robots.txt file:

User-agent: PrivateSEOBot
Disallow: /

To block only a section, replace / with its path. For example, Disallow: /private/. Standard Allow rules can create more specific exceptions.

Privacy and data handling

privateSEO is a desktop application. Scan reports and fetched website analysis are stored locally on the user's device unless that user intentionally exports a PDF or CSV file. License validation communicates plan entitlement information to the privateSEO licensing service, but does not send the scanned website's page content to that service.

PrivateSEOBot is intended for public website auditing. It does not support authenticated crawling and does not intentionally request local-network addresses, browser sessions, passwords, or private account content.