# This rule means it applies to all user-agents User-agent: * # Disallow all directories and files within Disallow: /cp/ Disallow: /layout_files/ Disallow: /rf-admin/ Disallow: /*login # The Googlebot is the main bots for Google Search Engine User-agent: Googlebot # Disallow all directories and files within Disallow: /cp/ Disallow: /layout_files/ Disallow: /rf-admin/ Disallow: /*login # Disallow all files with ? in url Disallow: /*?* Disallow: /*? # Googlebot-Image is the Google bots which crawling image User-agent: Googlebot-Image # Allow Everything Allow: /* # This is Google AdSense bots User-agent: Mediapartners-Google* # Allow Everything Allow: /*