Search engines on my site have nearly the same rights as members. Dunno how vbulletin goes about it but in ipb you can put spiders/bots in a usergroup of choice. So guests still need to register but google etc can crawl and index all public member sections. Dowside is people can use the cache to avoid needing to register. But i think very few people do that.