CitevanaBot

How to recognise it

CitevanaBot sends this User-Agent header on every request: CitevanaBot/0.1 (+https://citevana.com/bot)

The name to use in robots.txt is CitevanaBot.

IP address (current)

CitevanaBot currently connects from this IP address: 187.77.220.186. It connects over IPv4 only.

The address can change if we move or add servers. This page always lists the current one, so check it when you review an allow-list.

What CitevanaBot fetches

CitevanaBot fetches only URLs that people submit to Citevana tools, plus closely related public files on the same site, such as /robots.txt and /llms.txt, and, for site audits, websites whose owners have verified them in Citevana. It does not browse the web on its own.

robots.txt

Crawling features, such as site audits, respect robots.txt rules. A one-off check of a single URL is requested by a person, much like opening the page in a browser. To block CitevanaBot's crawling features, add this to your robots.txt: User-agent: CitevanaBot / Disallow: /

Why a firewall may block it

Bot-protection services, such as Cloudflare Bot Fight Mode, “Under Attack” mode and managed challenges, or other web application firewalls, may show CitevanaBot a challenge page instead of your page. CitevanaBot never tries to solve or get around these checks. The audit stops and tells the site owner what to change, and pages that weren't checked don't use any audit allowance.

How to allow CitevanaBot in Cloudflare

• In the Cloudflare dashboard, open your domain and go to Security → Security rules → Create rule → Custom rules (in older dashboards: Security → WAF → Custom rules).
• Use this expression: (http.user_agent contains "CitevanaBot"). For a stricter rule that also checks our current IP address, use: (http.user_agent contains "CitevanaBot" and ip.src in {187.77.220.186})
• Choose the Skip action and tick All remaining custom rules, All rate limiting rules, All Super Bot Fight Mode rules and All managed rules (and Browser Integrity Check, if offered). Deploy the rule and move it to the top.
• Bot Fight Mode, the free-plan setting, can't be skipped by any rule. Turn it off while an audit runs, or use Super Bot Fight Mode (Pro plan and above), which the Skip rule does cover.
• If “Under Attack” mode is on, every visitor gets a challenge: turn it off while an audit runs. Cloudflare Configuration Rules can change the Security Level for matching requests, so a rule for CitevanaBot with a lower Security Level may also work, but we haven't confirmed that.

Other firewalls and CDNs

In Akamai, Imperva, Sucuri, DataDome, HUMAN (PerimeterX), AWS WAF or your hosting provider's firewall, add an allow or bypass rule for requests whose User-Agent contains CitevanaBot. If you can, limit the rule to the IP address above, because anyone can copy a user agent.

Limits and safeguards

• It only connects to public internet addresses, over HTTP or HTTPS on standard ports.
• It follows at most 5 redirects, stops after 8 seconds, and reads at most 2 MB per response.
• Site audits send at most 2 requests at a time, wait between requests, follow Crawl-delay (up to 5 seconds) and slow down when a site answers “too many requests”.
• Free tool requests are rate-limited.
• It does not log in, submit forms, or try to get around CAPTCHAs or bot protection.
• It does not scrape Google search results.
• It treats page content as data only, never as instructions.

Contact

If CitevanaBot causes a problem for your site, email hello@citevana.com with the affected URLs and the time, and we will look into it.

Frequently asked questions