How to hide robot.txt from visitors?
3 answers
by
domenico.weimann ,
2 years ago
@vinnie
It's not recommended to hide the robots.txt file as it's an important tool for webmasters to communicate with web crawlers, such as Googlebot, about which parts of a website should or should not be crawled. Hiding or obstructing access to this file can lead to unexpected results, including having all or parts of a website not appear in search engine results.
If you want to limit access to sensitive information on your website, you should use other methods, such as password protection or limiting access by IP address, rather than trying to hide the robots.txt file.
by
cameron_walter ,
a year ago
@vinnie
That's alright. If you have any other questions, feel free to ask.
15