Google DId not Crawl My Site :-((

[bluedevil]

Heeloo..
i added my forum to google search engine.. but it does not crawl my site with following errors..


how can i sort out this error..

bluedevil Reviewed by bluedevil on 17th Oct 2009. Google DId not Crawl My Site :-(( Heeloo.. i added my forum to google search engine.. but it does not crawl my site with following errors.. http://www.besthostingforums.com/images/cdn/besthostingforums.com/2009/10/6508k1-1.jpg how can i sort out this error.. Rating: 5

[desiboy]

How can you expect it to crawl when you are restricting it to crawl in your robots.txt ?
http://kk4tech.com/robots.txt

[bluedevil]

User-agent: *
sitemap: http://www.kk4tech.com/kk/sitemap_1.xml.gz
Disallow: /kk/archive/
Disallow: /kk/admincp/
Disallow: /kk/attachments/
Disallow: /kk/calendar.php
Disallow: /kk/clientscript/
Disallow: /kk/cpstyles/
Disallow: /kk/customavatars/
Disallow: /kk/customprofilepics/
Disallow: /kk/faq.php
Disallow: /kk/images/
Disallow: /kk/install/
Disallow: /kk/includes/
Disallow: /kk/login.php
Disallow: /kk/misc.php
Disallow: /kk/member.php
Disallow: /kk/modcp/
Disallow: /kk/memberlist.php
Disallow: /kk/newreply.php
Disallow: /kk/newthread.php
Disallow: /kk/private.php
Disallow: /kk/register.php
Disallow: /kk/sendmessage.php
Disallow: /kk/signaturepics/
Disallow: /kk/sendpm.php
Disallow: /kk/search.php
User-agent: Googlebot-Image
Disallow:


User-agent: Mediapartners-Google*
Disallow:

what to remove frm it??

[desiboy]

remove
Disallow: /kk/calendar.php
Disallow: /kk/faq.php
Disallow: /kk/memberlist.php

[bluedevil]

ohk thanks.. but for other pages.. will it get this error again???

[Mac]

It's normal to get those errors, Google isn't supposed to crawl the calendar etc anyways. Would be the best for you to use a special skin for crawling without those links...

[bluedevil]

ahan, which skin?

[Mr Happy]

ahan, which skin?

This thread may help a bit. http://www.besthostingforums.com/showthread.php?t=14904

[narutoroot]

I always thought google webmasters messes shit up. Yet again robots.txt does to