ddlmafiia auto crawler

[immortalxx]

I have been working on and basically perfected an auto crawler that will crawl (currently) forum sites using ipb, once i recieve the first request to crawl vb i will also add that to my crawler.

To get listed, you must have our button and have submitted at least 20 links to ddlmafia.net/submit.php, using a valid email.

Once you have done this, use our contact page to get my email and send me one, requesting you would like your site to be auto crawled.

This is a beta feature atm, but working with imafia.net.

when indexing your site, it checks for button, so make sure its there. Your downloads also must be organized for example, all movie downloads in movie section etc, etc.

immortalxx Reviewed by immortalxx on 25th Jun 2009. ddlmafiia auto crawler I have been working on and basically perfected an auto crawler that will crawl (currently) forum sites using ipb, once i recieve the first request to crawl vb i will also add that to my crawler. To get listed, you must have our button and have submitted at least 20 links to ddlmafia.net/submit.php, using a valid email. Once you have done this, use our contact page to get my email and send me one, requesting you would like your site to be auto crawled. This is a beta feature atm, but Rating: 5

[JmZ]

Why would anyone want you to leech all their content?
It benefits the webmasters in no way what so ever, all it means is you get content free.

[immortalxx]

wat? lol it doesnt steal the content..lol it links to the pages, similar to katz, and phaze. Next time take a look at the site before u comment. www.ddlmafia.net

[wildfire95]

What about permissions e.g, guests cant view threads.

[JmZ]

So why not tell them to submit to you?
I still see no reason why anyone would do this.

I'm not trying to instantly deny what you're saying but i cant make any sense of it.

[Curtis129]

So essentially it will become a warez search engine? organic? for instance we submit our website link and create an acc and then you bot crawls regularly and index's threads or what?

[immortalxx]

What about permissions e.g, guests cant view threads.

It will not include those in the main site, once it see's that guests can not view the page, it will skip that link.

So why not tell them to submit to you?
I still see no reason why anyone would do this.

I'm not trying to instantly deny what you're saying but i cant make any sense of it.

aw ok, people can manually submit to me, and they have to, in order for an account be created. But after a while, if they are submitting good links i will includetheir site in the crawler that way they no longer have to submit links. Saves everyone time. Plus their site is still being indexed via our star levels.

So essentially it will become a warez search engine? organic? for instance we submit our website link and create an acc and then you bot crawls regularly and index's threads or what?

Yes it will, for now its a small one.

Feel free add our button on all pages, submit some links, then email me,or pm me letting me know you wish to be added to the crawler.