How does a search robot
On this subject, there is a detailed faq on Yandex at http://help.yandex.ru/webmaster/?id=995296
A detailed, but not sufficiently informative. For example, to a direct question, and asked himself: «What is a robot search engine and what he does?» Yandex imagine, and answers:
Robot (English crawler) keeps a list of URL, it may index, and regularly download the corresponding documents. If the analysis instrument robot detects [...]
Page. 1 of 1 1
