robots.txt

[V!P]

Hi Guys
i have wordpress and when i go to my robots.txt i found this :

User-agent: *
Disallow:
Sitemap: [url]http://xxxxxxxxxxxxxxx.org/sitemap.xml.gz 

is that normal ????
cause i installed a plugin and i have written this :

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /category/*/*
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /*?*
Disallow: /*?
Allow: /wp-content/uploads
 
# Google Image
User-agent: Googlebot-Image
Disallow:
Allow: /*
 
# Google AdSense
User-agent: Mediapartners-Google*
Disallow:
Allow: /*
 
# Internet Archiver Wayback Machine
User-agent: ia_archiver
Disallow: /
 
# digg mirror
User-agent: duggmirror
Disallow: / 

V!P Reviewed by V!P on 21st Jun 2012. robots.txt Hi Guys i have wordpress and when i go to my robots.txt i found this : User-agent: * Disallow: Sitemap: http://xxxxxxxxxxxxxxx.org/sitemap.xml.gz is that normal ???? cause i installed a plugin and i have written this : Rating: 5

[V!P]

answer plz ???

[Proaudiozone.info]

don't pumb thread is prohabited risk of bann

[Heath]

Looks like its just blocking/hiding your sitemap.

[zyberakuma]

what plugin and what site?

[Cometolearn]

its normal nothinf to worry , previously u are disallowing core wp files .

[tritex]

Thats Right. I Have The Same Rorbots File. everything Is Fine.

[Apathetic]

Remove the plugin and place your robots.txt file directly in your documentroot.

[V!P]

jow can i create a robots.txt ???

[Yead]

It's very easy. There are many robot.txt generator. Just google them and they will make it for u. Or u can create it manually by placing the urls u dont want to indexed by google after the disallow: text.