# General crawlers User-agent: * Disallow: /admin Disallow: /login Disallow: /register Disallow: /profile Allow: / Sitemap: https://www.art-of-fart.com/sitemap.xml # AI crawlers — explicitly welcomed on public pages User-agent: GPTBot Allow: / Disallow: /admin Disallow: /login Disallow: /register Disallow: /profile User-agent: ClaudeBot Allow: / Disallow: /admin Disallow: /login Disallow: /register Disallow: /profile User-agent: PerplexityBot Allow: / Disallow: /admin Disallow: /login Disallow: /register Disallow: /profile User-agent: GoogleOther Allow: / Disallow: /admin Disallow: /login Disallow: /register Disallow: /profile User-agent: meta-externalagent Allow: / Disallow: /admin Disallow: /login Disallow: /register Disallow: /profile User-agent: CCBot Allow: / Disallow: /admin Disallow: /login Disallow: /register Disallow: /profile