[PHP] How to Create a Rapidleech Download Plugin

[Devilz]

When you request a URL to download from Rapidleech, Rapidleech will act as a browser. It will send a so-called HTTP request to the host's server and receives the response. During this transmition, there are a few important components, which are referer, cookie and post. Please make sure you do understand them before continuing. In order not to make this article too long, I will not be explaining them. Some file hosts require the correct referer and cookie value sent in order to give you the download link to download, so it is essential to always give the correct value. Now, you might be a little confused here, don't worry, try to understand what I'm saying, eventhough you don't know, just understand first, and continue.
In this article, I will take the easiest file host as example: zshared.net. I believe that after reading this article, you will be able to write even more complicated plugins on your own. But of course, I will also explain some essential parts.
When you want to download from zshared, you will first enter the file url, then hit enter on the browser. During this process, the browser actually sends a GET request to the server which looks like this:

GET / HTTP/1.1
Host: zshare.net
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

Upon receiving this request, the server will return a response, which looks as below:

HTTP/1.1 200 OK
X-Powered-By: PHP/5.2.0-8+etch13
Content-type: text/html; charset=UTF-8
Last-Modified: Tue, 17 Mar 2009 10:32:36 -0400
Cache-Control: post-check=0, pre-check=0
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Set-Cookie: sid=ed161d014ec131d4146ab3239892ac45; expires=Wed, 17 Mar 2010 14:32:36 GMT; path=/; domain=.zshare.net
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 4232
Date: Tue, 17 Mar 2009 14:32:36 GMT
Server: lighttpd/1.5.0

Note that this is just the header of the response, there are still some more data below, which is the page which you will usually see when you open zshare.net in the browser.
During this request and response, you can see that the browser first requested a page '/' to the server, the server then responded with a cookie value which is defined by 'Set-Cookie'. This value is usually very important as many file hosts will check the cookie value before allowing you to download a file.
Writing the plugin

Rapidleech passes the download url as $LINK. But before you request a response, you must parse the url into a few components: Host, port, path and query. You can do that by using:

$Url = parse_url($LINK);

Then, when you get the components of a url, you can request and receive response in this way:

$page = geturl($Url["host"], $Url["port"] ? $Url["port"] : 80, $Url["path"].($Url["query"] ? "?".$Url["query"] : "") ,$Referer , $cookie, $post, 0, $_GET["proxy"],$pauth);

In this case, all headers and body contents will be in the variable $page. Next, you will want to check if there are any cookie values set, you can do that by:

preg_match_all('/Set-Cookie: (.*);/U',$page,$temp);
$cookie = $temp[1];
$cookie = implode(';',$cookie);
 

All cookies will be saved into $cookie. But you don't have to do this everytime, you will understand about that later.
Proceeding to download page

So next, if you want to download a file, you will need to click on the download button, right? But how do you do that in Rapidleech? Well, you can emulate a browser receiving clicks. But you will have to see what type is it.
There are basically 2 types of clicks, one is redirection, another is form submittion. Ok, so how do you differentiate it? In zshare, look for this line:

<form name="form1" method="post" action="">
<input name="referer2" type="hidden" id="referer2" value="">
<input name="download" type="hidden" id="download" value="1">
<input name="imageField" type="image" id="imageField" src="/images/download.gif" width="219" height="57" border="0">
</form>

Well, it seems obvious that they are using the post method. Form is the signature, the 'Download Now!' button you see in the browser is the line with name="imageField".
When you see file hosts using post method, you must be extremely careful. You must pass every value of the input. As you can see, there are 3 input tags in this form. But the ones with values are 'referer2' and 'download', so you must pass these 2 values to the server. Besides, you must also note the form action, see where it is posting to. In zshared, the action is empty, it means it will be posted to the same page. Some file hosts will post the content to other pages, in this case you will need to get the url in the action. You can use the following function to get the url:

$url_action = cut_str($page,'<form method="post" action="','"');

But this will not work on every site, some site might not work with it. For example, zshare will not work. Because, cut_str actually finds the string '<form method="post" action="' exactly, however, in zshare, we have '<form name="form1" method="post" action="'. Note the difference? So you can just change it to match.
However, if you know RegExp (Regular Expressions), you can use the following function to get the action value:

preg_match('/<form.*method="post" action="(.*)"/U',$page,$url_action);
$url_action = trim($url_action[1]);

Remember to add the second line, the results returned by preg_match is in array.
Both methods work the same but I prefer the RegExp method.
However, in zshared, the action value is empty, so you actually don't have to care about it. You can just post back to the same url.
So, in order to post some values, we construct an array as below:

$post['referer2'] = '';
$post['download'] = 1;

See that it is important to put fields without values too. Sometimes your plugin might not work just because you miss out this important post value.
Next, we send the post request to the server using the same command as previous:

$page = geturl($Url["host"], $Url["port"] ? $Url["port"] : 80, $Url["path"].($Url["query"] ? "?".$Url["query"] : "") ,$Referer , $cookie, $post, 0, $_GET["proxy"],$pauth);

But this time, you have a referer, which is the download link itself. In zshared, you can ignore it but in some other sites it might be important. In this request, the browser sends the following:

POST /download/57177513735828d7/ HTTP/1.1
Host: www.zshare.net
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.zshare.net/download/57177513735828d7/
Cookie: sid=ed161d014ec131d4146ab3239892ac45
Content-Type: application/x-www-form-urlencoded
Content-Length: 135

referer2=&download=1&imageField.x=90&imageField.y=35

It's always a good idea to use a tool for analysing headers that are received and sent during a download/upload, such as LiveHTTPHeaders plugin for Firefox, or WireShark if you are more familiar. I personally prefer HTTP Debugger Pro. Some sites also use JavaScript to either set cookies or send information, so when you look at the headers you might wonder where some of them came from (how they got sent or received) - so look in the javascript for things like 'onClick' to see if it sets a cookie, or posts information along with the rest of the <form>. Well, if you are familiar with HTML, you might notice. Whenever there is an image input type, you must post the name.X and name.Y values together. But what about the referer2? Well, you can leave that out as that actually should be the page before you reach this download page, which is empty.
So this is the correct post array:

$post['referer2'] = '';
$post['download'] = 1;
$post['imageField.x'] = 90;
$post['imageField.y'] = 35;
 

Ok, so in the browser, after we clicked the button, there will be a countdown before the link appears. Well, we will need to do that in Rapidleech too since the download won't be available until the countdown finishes...
Counting down

So, in order to countdown from Rapidleech, we must get how many seconds exactly do we need to countdown first. I have found the countdown value here:

|start|link|document|important||here|50|class|onClick|Click|

To get the countdown seconds, use the method same as getting the action link:

$countdown = cut_str($page,'|start|link|document|important||here|','|');

Or

 preg_match('/|start|link|document|important||here|(.*)|class|onClick|Click|/',$page,$countdown); 
 $countdown = trim($countdown[1]);

So now, $countdown contains the value of time you will need to countdown. In order to let Rapidleech countdown and wait for the download link, use this function:

insert_timer($countnum, "Counting down...");

Well, you can put anything instead of "Counting down...", just make sure it makes sense to the user.
Retrieving the link and download it

Ok, here comes the difficult part. You will need to know the exact link which points to the file you want to download. zshare seems to make it complicated enough like here:

var link_enc=new Array('h','t','t','p',':','/','/','d','l','0','1','2','.','z','s','h','a','r','e','.','n','e','t','/','d','o','w',
'n','l','o','a','d','/','f','0','5','1','d','8','9','2','e','d','c','3','0','8','8','2','1','f','f','7','b','0','d','5','a','2','7','3',
'2','b','6','9','/','1','2','3','7','3','4','3','8','0','3','/','5','7','1','7','7','5','1','3','/','c','o','r','e','_','d','e','c','.','p','h','p');

Notice the download link? They are all splitted by "','", so you know what I mean? So here's the easy way:
First, get rid of those extras like var link_enc etc:

$dllink = cut_str("var link_enc=new Array('","')");
 

Then, you just get rid of those "','" to get a valid download link!

$dllink = str_replace("','","",$dllink);

Here's if you know RegExp:

preg_match('/Array\((.+)\);link/', $page, $dllink);
$dllink = $dllink[1];
$dllink = preg_replace('/\'[, ]?[ ,]?/six', '', $dllink);
 

The last thing you will have to do, is tell Rapidleech that you have found the download link, and let it download the file. In order to do so, you must parse the download link again first and tell Rapidleech the filename:

$Url = parse_url($dllink);
$FileName = basename($Url["path"]);

Then, pass the url to Rapidleech this way:

$loc = "{$_SERVER['PHP_SELF']}?filename=" . urlencode ( $FileName ) . 
    "&host=" . $Url ["host"] . "&port=" . $Url ["port"] . "&path=" . 
    urlencode ( $Url ["path"] . ($Url ["query"] ? "?" . $Url ["query"] : "") ) . 
    "&referer=" . urlencode ( $Referer ) . "&email=" . ($_GET ["domail"] ? $_GET ["email"] : "") . 
    "&partSize=" . ($_GET ["split"] ? $_GET ["partSize"] : "") . "&method=" . $_GET ["method"] . 
    "&proxy=" . ($_GET ["useproxy"] ? $_GET ["proxy"] : "") . "&saveto=" . $_GET ["path"] . 
    "&link=" . urlencode ( $link ) . ($_GET ["add_comment"] == "on" ? "&comment=" . 
    urlencode ( $_GET ["comment"] ) : "") . $auth . ($pauth ? "&pauth=$pauth" : "") . 
    "&cookie=" . urlencode($cookie) .
    "&post=" . urlencode ( serialize ( $post ) );
insert_location ( $loc );

In some file hosts, you will still need to pass a cookie and some posts in order to download the file, so don't forget to put it there. Now, if you look into Rapidleech's plugin, you will see that all insert_location is done in 1 line. This is not really a good practise, I hope that you will follow this way to make the plugin more editable later on.
So, if you have been following closely all the way, you might have written a plugin for yourself already! But if you compare to the original zshare plugin, ther emight be some difference. There are a lot of things not needed in zshare which are taught in this article. But I hope that you are now able to write a plugin yourself.
Sites with captcha

However, in this article, I haven't been teaching how to write plugins for sites with captchas. Well, zshare doesn't have captcha but I will show you how to do it when you come up to a site with captchas.
When a site has captcha, you must split the process of downloading the file into 2 steps. So use an 'if' statement at the beginning like:

if ($_GET['step'] == 1) {

Inside this 'if' statement is step 2. Use the 'else' statement for step 1 like:

if ($_GET['step'] == 1) {
    // Step 2
} else {
    // Step 1
}

In step 1, you must get the captcha and display it to the user and let the user enter the captcha. In step 2, you will post the captcha the user entered to the file host and request the download link.
To get the captcha, you must find the captcha image in a page. There are 2 types of captchas, 1 is static with the link and another is dynamic. Static ones display the same letters and numbers in the captcha no matter how many time you refresh with the image link. The dynamic ones will renew itself each time it is accessed. To identify whether a captcha is dynamic or not, find the captcha's image link. Like for megaupload, the captcha is static. Because this

http://wwwq13.megaupload.com/gencap.php?89187dc555d37ffd.gif

is always DVC8, well, it's expired now but it won't be so fast when the user retrieves it, so it's ok to use that link as the captcha image. However, some file hosts change their image everytime you request it. In this case, you can only request it once from the server and save it to the server temporarily and present it to the user. For example, ziddu is a file host which is considered dynamic captcha. You will have to retrieve the url of the captcha image first. In ziddu, you can find the captcha image here:

<img src="/CaptchaSecurityImages.php?width=100&height=38&characters=5" align="absmiddle" id="image" name="image"/>

So you can retrieve it using the technique above. I'm not going to show you once more, if you still don't know how to retrieve it, you might need to restart all over again. After you got the url of the captcha and saved it into a variable like let's say $img, we then parse the url to download it and save it as a file:

 $Url = parse_url($img);
$page = geturl($Url["host"], $Url["port"] ? $Url["port"] : 80, $Url["path"].($Url["query"] ? "?".$Url["query"] : ""), $LINK, $cook, 0, 0, $_GET["proxy"],$pauth);
    
$headerend = strpos($page,"\r\n\r\n");
$pass_img = substr($page,$headerend+4);
write_file($download_dir."ziddu_captcha.jpg", $pass_img);
$randnum = rand(10000, 100000);
    
$img_data = explode("\r\n\r\n", $page);
$header_img = $img_data[0];

In some circumstances, the host will pass some cookie values together with the image, just remember to capture them. So you got the image as files/ziddu_captcha.jpg then you must display it to the user so he/she can enter it.
As I have mentioned above, these types of plugins have two stages, one occur before the captcha and one after it. These two stages does not share the same session so variables are not stored. In order to pass the variables, you must enter it together with the form. Like the ziddu plugin, it appears like this:

print     "<form method=\"post\" action=\"$PHP_SELF\">$nn";
print    "<b>Please enter code:</b><br>$nn";
print    "<img src=\"{$download_dir}ziddu_captcha.jpg?id=".$randnum."\" >$nn";
print    "<input name=\"link\" value=\"$LINK\" type=\"hidden\">$nn";
print    "<input name=\"referer\" value=\"$referer\" type=\"hidden\">$nn";
print    "<input name=\"flink\" value=\"$flink\" type=\"hidden\">$nn";
print    "<input type=hidden name=id value=$id[1]>$nn";
print    "<input name=\"zd\" value=\"ok\" type=\"hidden\">$nn";
print    "<input name=\"cookie\" value=\"$cook\" type=\"hidden\">$nn";
print    "<input name=\"name\" value=\"$name\" type=\"hidden\">$nn";
print    "<input name=\"fid\" value=\"$fid[1]\" type=\"hidden\">$nn";
print    "<input name=\"tid\" value=\"$tid[1]\" type=\"hidden\">$nn";
print    "<input name=\"fname\" value=\"$fname[1]\" type=\"hidden\">$nn";
print    "<input name=\"securecode\" value=\"$securecode[1]\" type=\"hidden\">$nn";
print    "<input name=\"keyword\" value=\"$keyword[1]\" type=\"hidden\">$nn";
print    "<input name=\"securitycode\" type=\"text\" >";
print    "<input name=\"submit\" value=\"Download\" type=\"submit\"></form>";

Some are the variables retrieved from forms earlier and some are required to let Rapidleech know that is in the next stage. Below is a checklist that you should always not forget to put together with this form:

1. First and foremost, the captcha image and the field
2. Next, it's important to put an input element with the name 'link' and the exact value that is entered by the user, this ensures that Rapidleech will later come back to this specific plugin
3. A variable to let the plugin know that it is entering the next stage. In ziddu, we have ziddu="ok".
4. POST values that may be passed to the host.

Next, when the user enters the captcha and submits the form, it is our time to submit the results to the file host.
Submitting the captcha

We need to know that we are now entering this after captcha stage. So you must see this in the code:

if ($_POST['zd'] == 'ok') {

In this stage, we must capture all values which are posted from the previous stage, although some are not needed. When we get those values, we can post them to the file host. In zshare, the download will be initiated once you post the captcha. So we collect the variables passed:

$post = Array();
$post["fid"] = $_POST['fid'];
$post["tid"] = $_POST['tid'];
$post["securitycode"] = $_POST['securitycode'];
$post["fname"] = $_POST['fname'];
$post["securecode"] = $_POST['securecode'];
$post["Keyword"] = 'Ok';
$post["submit"] = 'Download';
$cookie = $_POST['cookie'];
$Referer = $_POST["referer"];
$Href = $_POST["flink"];
$FileName = $_POST["name"];
$Url = parse_url($Href);

Make sure to capture cookie, referer and the download link, determine the filename and off we go posting it!
Conclusion

Writing a plugin for a file host is a very flexible thing. Process written here is not necessary followed and you will have to twist your mind to follow the file host not this tutorial. Besides, file host love to always change their script to break our plugins as a present, so the most important is that you frequently check. Users will report, but it is always good to fix before any reports turn up, isn't it?
Things to remember

Below is a list of things to remember:

1. File host change their script frequently to break our plugin, don't forget to fix.
2. Always remember to check for cookies and hidden input elements when posting forms.
3. Don't leave out the referrer when redirecting or posting form.
4. If you encounter some problems and want to debug so that you know what Rapidleech is receiving instead of what you're seeing in the browser, use this line:

   echo "<pre>";var_dump(nl2br(htmlentities($page)));echo "</pre>";exit;

   
   
   This will print out the html source of the page Rapidleech is receiving.

Devilz Reviewed by Devilz on 18th Jun 2010. [PHP] How to Create a Rapidleech Download Plugin When you request a URL to download from Rapidleech, Rapidleech will act as a browser. It will send a so-called HTTP request to the host's server and receives the response. During this transmition, there are a few important components, which are referer, cookie and post. Please make sure you do understand them before continuing. In order not to make this article too long, I will not be explaining them. Some file hosts require the correct referer and cookie value sent in order to give you Rating: 5