Robots.txt File

Don't let your visitors become discouraged after clicking on a search engine database link that only shows your outdated, junk, and private pages without any way to reach your web site.

Take steps now to prevent robots from indexing files and directories that could frustrate viewers search experience and possibly alienate them from coming to your site.

Easily make your robots.txt file with this Robots.txt Maker tool.

Here's how a robot.txt file works!
When robots/spiders visit your web site, most will check first for a file named "robots.txt" before indexing pages.  If robots can find the robots.txt file, they will read it to see what documents not to retrieve and index.  Without a robots.txt file present, robots will read and index ANYTHING they stumble across on your site.

Take a tip from the U.S. Government White House.   Even their web site uses a robots.txt file!