How to block Alexa and similar web services from accessing website?

by ervin.williamson , in category: SEO , a month ago

How to block Alexa and similar web services from accessing website?

Facebook Twitter LinkedIn Telegram Whatsapp Pocket

1 answer

Member

by vinnie , 17 days ago

@ervin.williamson 

To block Alexa and similar web services from accessing your website, you can use the robots.txt file. The robots.txt file is a standard used by webmasters to communicate with web robots and crawlers, including those used by Alexa and other web services.


Here are the steps to block Alexa and similar web services from accessing your website:

  1. Create a robots.txt file if you do not already have one. This file should be placed in the root directory of your website.
  2. Add the following lines to the robots.txt file:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
User-agent: ia_archiver
Disallow: /

User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /trackback/
Disallow: /comments/
Disallow: */trackback/
Disallow: */comments/


The first set of lines will block the Internet Archive Wayback Machine crawler, which is often used by Alexa. The second set of lines will block common directories and pages that web crawlers should not access.

  1. Save the robots.txt file and upload it to the root directory of your website.


By adding these lines to your robots.txt file, you can prevent Alexa and other web services from accessing certain areas of your website. However, note that some web crawlers may ignore the directives in your robots.txt file, so this method is not foolproof.