How to disallow query strings in robots.txt?
1 answer
by
genevieve_boehm ,
6 months ago
@elliott
To disallow query strings in robots.txt, you can use the following syntax:
User-agent: * Disallow: /*?
This will tell all search engine bots to not crawl any URLs that contain a query string. The "/*?" rule blocks any URL that has a question mark, which is the beginning of a query string in a URL.
15
Related Threads:
How do search engines treat query strings?
What is the difference between `allow: /` & `disallow: ` in robots.txt?
How to disallow search pages from robots.txt?
How to disallow a directory in server with robots.txt?
How to escape $ in robots.txt disallow directive?
How to disallow nested folders with robots.txt?