DatvixBot

User-Agent

Requests identify themselves with this exact string:

DatvixBot/2.0 (+https://www.datvixai.com/bot)

The string is self-declared. Datvix does not claim verified bot status with any search engine or CDN, including Google, Cloudflare, Akamai, or SEMrush.

What it fetches

• It requests robots.txt for the host being audited.
• It can request sitemap URLs named in that file when the robots rules allow those URLs.
• It requests HTML pages with the User-Agent above. Each redirect hop is checked before the next request, and the fetched markup is used for the audit.

In the crawler source, a standard crawl stops at depth 8 and 6,000 pages. A free-tier crawl stops at depth 3 and 200 pages. A job can pass its own page cap. These are source defaults, not a statement that every production process is running this revision, and they do not mean every URL on a site is fetched.

robots.txt, rate limits, and access denials

• Allow and Disallow are evaluated for this User-Agent, including wildcard rules. A group with a universal Disallow and no Allow is treated as a block of that host.
• Crawl-delay is applied and capped at 5 seconds. A fetched robots.txt file is reused for one hour.
• HTTP 429, 500, 502, 503, and 504 can be retried. HTTP 401 and 403 are not retried and are not rendered in a browser. A denied response is not a successful audit and does not produce a site health score.
• After a crawl that is blocked by robots or by access denial, the source default wait before another job is 24 hours. After a crawl that fetched nothing because of rate limiting, the default wait is 1 hour. Those waits can be changed by deployment configuration.

For HTTP 401 and 403, the current source can keep a short diagnostic record: status, a truncated final URL, selected response headers, and a fingerprint of the start of the body. The body itself is discarded. Cookies and authorization headers are not stored. That describes the source. It does not prove the running production process is already recording those fields.

Blocking and contact

To block DatvixBot, disallow DatvixBot in your robots.txt. A CDN or firewall rule is controlled by the site owner. Datvix cannot guarantee that an allow or a block will be accepted. Published crawler IP ranges are not available.

To ask for less frequent crawling, or to report a crawler concern, email sales@datvixai.com. That address is the mailbox named in the product contact mail settings. The contact page is a general product form, not a separate crawler desk, and sending mail from it depends on SMTP being configured in the deployment. Include the website, the time, the URL, and the User-Agent from your logs.