Spiders and Bots

[Mr Happy]

What areas do you let bots like googlebot around in your site. Does it just have access to your Guest area or does it have access to Members area or even VIP area?

Mr Happy Reviewed by Mr Happy on 21st Aug 2009. Spiders and Bots What areas do you let bots like googlebot around in your site. Does it just have access to your Guest area or does it have access to Members area or even VIP area? Rating: 5

[carnage]

Its like a regular member all it can do it is view all viewable forums and just leech it to the search engines.

[Mr Happy]

Its like a regular member all it can do it is view all viewable forums and just leech it to the search engines.

Ya but that means their are links which are indexed in Google but you have to register to see which is crap for visitors.

Just wondering as Warez-bb hasn't any. My site is Guest only but thinking of changing it to regular members to get more traffic.

[carnage]

Ya it does bring more traffic but less registrants.

[killer]

mr. happy what is your site?

[ACE]

just wanted to let you know. If you let the bots index your special sections like staff section or the VIP section, When it gets indexed it is viewable to the public if they look in the right place. I have seen it in a few places but i forget where.

[Mr Happy]

I have a script that let's me see a site as a googlebot so if you have your vip/staff area viewable to bots I can see it. That's why it can be dodgy

[desiboy]

this is a very good addon
https://addons.mozilla.org/en-US/firefox/addon/59