robots.txt Archives - Web Development and Digital Strategy Consultancy

Tag Archives: robots.txt


What is Robots.txt and The best way to Optimize It

Basically, robots.txt is a transmission between a website and the web search engine. It traces the guidelines for indexing your site and presents…
Continue reading

What happen if we place robots.txt file in sub directory?

It will apply to that directory only - not sure if it would get crawled there. It would be best to implement in…
Continue reading