Ask Question Sign In Sign Up

#robots

Member

How to stop robots from crawling pagination using robots.txt?
by cameron_walter , in category: SEO , 2 years ago
3 answers last reply a year ago from hanna

How to stop robots from crawling pagination using robots.txt?

robots robotic stoprobots

Member

How to optimize my website's meta robots?
by hanna , in category: SEO , 2 years ago
2 answers last reply a year ago from cameron_walter

How to optimize my website's meta robots?

site website robots robotic optimize

Member

How to make an exception for <meta name="robots" content="nofollow"> for several links?
by dejuan.watsica , in category: SEO , 2 years ago
2 answers last reply a year ago from aniyah.green

How to make an exception for <meta name="robots" content="nofollow"> for several links?

robots robot content links link

Member

How to stop robots crawling pagination using robots.txt?
by elmo.conroy , in category: SEO , 2 years ago
1 answer last reply 2 years ago from aniyah.green

How to stop robots crawling pagination using robots.txt?

stoprobots robots robotic

Member

What is a meta robot tag, and how does it affect SEO?
by zion , in category: SEO , 2 years ago
1 answer last reply 2 years ago from larry_orn

What is a meta robot tag, and how does it affect SEO?

seo robots tag

Member

How to block robots without robots.txt?
by creola.ebert , in category: SEO , 2 years ago
2 answers last reply 2 years ago from aniyah

How to block robots without robots.txt?

robots block robotic

Member

What http signatures are encountered by Google's web-crawling robots?
by virginie , in category: SEO , 2 years ago
2 answers last reply 2 years ago from pietro

What http signatures are encountered by Google's web-crawling robots?

google signature california web robots