How does robot.txt works

[Eleeazar]

Hi,
How does a robot.ext file works. I need some information on this. What I now about this file is that it prevents an online service to view the entire site. It is also termed as bots. But does this guarantee privacy to a website. It is positive to carry out robot.txt on the site. Whey this protocol is implemented on the web and what is the disadvantages of the same. Also how to place a good hyper link in a website.

[Jackson2]

The robots.txt file is the first resource site which is used to spam search engines. It allows to adapt the robot's behavior during the course of the site prohibiting him from indexing certain directories. Like the file name should be lowercase. If this file does not exist, the robot continues his visit normally by indexing all the pages. This practice is considered as ban.

[Techno01]

It is prohibited to use robots.txt. This can protect a script directory (/ cgi-bin /) and confidential data (/ private /) against the referral. Thus, the pages prohibited not only appears in search engines, ven if the sites offer links to those pages. You have to ensure that nothing prevents an attacker to make his own robot and not respecting robots.txt. Furthermore, robots.txt is not related to a file. Htaccess file, it does not protect your pages against Internet access.

[Trio]

To create a good hyperlink several elements are important to consider: Like the address itself: avoid relative links, choose the links absolute so that they remain valid even if the page is moved or copied to the disk of a user. The link text must be explicit and must indicate the direction of the subject of the page or destination media (the "click here" should be eliminated).

[deveritt]

The link title should be explicit may be substituted for the text, which allows (to some extent) put links click here. The size of the title and text: it is better to use keywords that reflect the main theme of the target page. Reduce the size of your text and titles of links to an absolute minimum; Behavior link: the option to target a href tag opens the page in another browser window and thus does not lose the visitor if a link to another site.

[Techguru01]

This is also known as Robots Exclusion Protocol. The same is called as robots.txt. This is used to spam search engines. This is used to prevent the web crawlers and other robots in accessing the entire part of a website. Sometime the same are used by search engine to provide a category or archive to a website. The problem is that is option does not guarantee privacy to the site