# Block specific user agents (unauthorized crawlers and website copiers) User-agent: HTTrack Disallow: / User-agent: wget Disallow: / User-agent: curl Disallow: / User-agent: Python Disallow: / User-agent: php Disallow: / User-agent: Nmap Disallow: / User-agent: libwww Disallow: / User-agent: Sogou Disallow: / User-agent: ZmEu Disallow: / # Allow all legitimate crawlers User-agent: * Allow: / # Specify sitemap for legitimate crawlers Sitemap: https://www.cybex.in/sitemap.xml