In google what is Robots file

Hi techies,
Can anyone please help me with the robots file in Google and how is it helpful.....

Robot file is often called robot.txt file, its main use is to restrict the web-crawlers or robots from accessing certain parts or the complete website. Robots or Web-crawlers are often used by search engines to categorize and archive other websites which they visit.

It usually functions as a request made to the crawlers or robots to not touch the contents of specified file or directories or to ignore the files and folders.This might be, for example, out of a preference for privacy from search engine results, or the belief that the content of the selected directories might be misleading or irrelevant to the categorization of the site as a whole, or out of a desire that an application only operate on certain data.

Hi,
But its working depends on the domains and sub-domain you have...... If you have a domain called one.com and correspondingly a robot.txt file for it then it will work properly. But if you also have a sub-domain i.e, p.one.com then the robot.txt file will be valid fro only one.com and not p.one.com....you need to create different robot.txt for multiple domains.
:biggrin: