A text file located in the root directory of a website that instructs search engine crawlers which pages or files should not be indexed.