The robots.txt is used to guide search engine robots or spiders which pages can be crawled, in the 10 effective SEO tips, robots.txt is very important. We can not only forbid the folders and pages which are not allowed to others, but also tell the search engines our sitemap.xml directly.

User-agent:*
#Sitemap
Sitemap:http://www.feelblog.com/sitemap.xml
Sitemap:http://www.feelblog.com/sitemap.xml.gz

#Directories
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /comments/feed/
Disallow: /page/
Disallow: /comments/

#Files
Disallow:/400.shtml
Disallow:/401.shtml
Disallow:/403.shtml
Disallow:/404.shtml
Disallow:/500.shtml

Here is my robots.txt for feelblog.com, copy and paste it in robots.txt, don’t forget change the URL. You can also use the Robots.txt Plugin for wordpress.