# ========================= # AI + SEO Optimized Robots.txt # Site: rfid-pro.com # ========================= # ===== Content Signals (EU directive compatible) ===== Content-Signal: search=yes,ai-train=yes User-agent: * Allow: / # ===== Allow AI search & indexing (important for AI visibility) ===== User-agent: GPTBot Allow: / User-agent: Google-Extended Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: meta-externalagent Allow: / # ===== Block aggressive / unwanted scrapers (optional safety layer) ===== User-agent: Amazonbot Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / # ===== WordPress Core (SEO best practice) ===== Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Disallow: /wp-includes/ # ===== Internal search / duplicate content control ===== Disallow: /*?s= Disallow: /*?replytocom= # ===== Low-value system pages ===== Disallow: /feed/ Disallow: /*/feed/ Disallow: /*/trackback/ Disallow: /*/embed/ Disallow: */page/ Disallow: /cookie-policy-eu/ # ===== Upload forms / sensitive backend paths ===== Disallow: /wp-content/uploads/wpforms/ # ===== Sitemap ===== Sitemap: https://rfid-pro.com/sitemap_index.xml