Help with robots.txt

[CK13]

This is my robots.txt

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/

What does that mean?
Is everything ok, do I have to add somethig else to it?

PS: Still learning .

CK13 Reviewed by CK13 on 5th Jun 2011. Help with robots.txt This is my robots.txt User-agent: * Disallow: /administrator/ Disallow: /cache/ Disallow: /components/ Disallow: /images/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /libraries/ Rating: 5

[Rox]

add ur sitemap and rss feeds

Example below:

Sitemap: http://www.yoursite.com/sitemap.xml
Sitemap: http://www.yoursite.com/category/movies/feed/rss
Sitemap: http://www.yoursite.com/category/tv-shows/feed/rss
Sitemap: http://www.yoursite.com/category/mov...codes/feed/rss

[Daniel]

Here's an example that is often used

User-agent: *
Disallow: 
Disallow: /cgi-bin/
Sitemap: http://www.yoursite.com/sitemap.gz

What you should do
Create a sitemap and add it to the bottom of the text file.

[Vince(AdJoin.Me)]

Here's an example that is often used

User-agent: *
Disallow: 
Disallow: /cgi-bin/
Sitemap: http://www.yoursite.com/sitemap.gz

What you should do
Create a sitemap and add it to the bottom of the text file.

you are helping him to do the right thing.

[CK13]

@ MrSandvik If I put you're code will this be allowed:


And is it safe to remove what is already there?

[Daniel]

You can easily use such a generator. No problem.

That's why they exist.

[CK13]

But what about my current robots file should i replace it or just copy the additional code?
Is it safe to remove what is already there?