# ByteDance / TikTok scraper: takes content, gives nothing back. Blocked here # as a signal; real enforcement is the Cloudflare WAF rule on this user-agent. User-agent: Bytespider Disallow: / # Everyone else (all search engines + the AI engines we want to be cited by: # Googlebot, Bingbot, GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, # PerplexityBot, Google-Extended, Applebot/Applebot-Extended) crawls freely, # except internal paths. User-agent: * Allow: / Disallow: /cdn-cgi/ Disallow: /admin/ Disallow: /api/ Disallow: /private/ Sitemap: https://furwise.app/sitemap-index.xml # llms.txt (non-standard, for discoverability): https://furwise.app/llms.txt