# https://act3ai.com/robots.txt # # Docusaurus copies site/static/ verbatim into the build, so this file is the # whole crawl policy for the site. Three jobs: # 1. point every crawler at the sitemap, # 2. keep the parked and variation pages out of the index, # 3. state, deliberately, that the AI answer engines are welcome. Our buyers # ask assistants which tool makes AI video; being readable by those # assistants is the point, not an accident of whatever the default is. User-agent: * Allow: / Disallow: /v/ Disallow: /backup # --------------------------------------------------------------------------- # AI crawlers and answer engines: explicitly allowed. # # These agents share ONE group on purpose. Under RFC 9309 a crawler obeys only # the most specific group that names it and ignores `User-agent: *`, so a bot # with its own `Allow: /` group would skip the Disallow lines above and crawl # the parked /backup pages, the /v/ variations and the stale local /docs/ copy. # Repeat those Disallow lines here whenever the `*` group changes. # --------------------------------------------------------------------------- # OpenAI User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User # Anthropic User-agent: ClaudeBot User-agent: Claude-SearchBot User-agent: Claude-User # Perplexity User-agent: PerplexityBot User-agent: Perplexity-User # Google Gemini grounding and training. Allowed on purpose: our audience asks # assistants for tool recommendations, and being groundable is worth more to us # than withholding a public marketing site. User-agent: Google-Extended # Apple, Amazon, Meta, Common Crawl User-agent: Applebot-Extended User-agent: Amazonbot User-agent: meta-externalagent User-agent: CCBot Allow: / Disallow: /v/ Disallow: /backup Disallow: /docs/ Sitemap: https://act3ai.com/sitemap.xml