Ask Question Sign In Sign Up

#bots

Member

How to assign specific sitemaps for specific crawler bots in robots.txt?
by jose_gulgowski , in category: SEO Tools , 2 years ago
2 answers last reply a year ago from declan_ritchie

How to assign specific sitemaps for specific crawler bots in robots.txt?

bots bot

Member

Where can I obtain a list of user agents for SEO bots?
by zion , in category: SEO , 2 years ago
3 answers last reply a year ago from cameron_walter

Where can I obtain a list of user agents for SEO bots?

agent seo bots bot useragent

Member

How to disallow bots from a single page or file?
by creola.ebert , in category: SEO , 2 years ago
2 answers last reply 2 years ago from pietro

How to disallow bots from a single page or file?

bots bot singlepage