What happen if we place robots.txt file in sub directory?

1 Comment

It will apply to that directory only – not sure if it would get crawled there. It would be best to implement in ur root folder and disallow any sub-directories you don’t want to be crawled. It’s all up to the crawlers to respect your directives then.
Ideally you should have it in the root file but if you have to place it in the sub-directory, then this is what we would recommend.

1. Specify the path in Google Webmasters

2. 301 Redirect the conventional path – domain.com/robots.txt to the new path.
If you have a multi-language website you should put every robots.txt in the first directory for every language sections of the website and not in the different sub-directory.

Delivering the best of our firm to each and every client as cost effectively as we can fine tunes our capabilities performing better and better.

About us and this blog

We are a digital marketing company with a focus on helping our customers achieve great results across several key areas.

Compare Buy SSL Certificates

More from our blog

See all posts