Activity Stream
48,167 MEMBERS
6905 ONLINE
besthostingforums On YouTube Subscribe to our Newsletter besthostingforums On Twitter besthostingforums On Facebook besthostingforums On facebook groups

Results 1 to 5 of 5
  1.     
    #1
    Member
    Website's:
    PasteBot.appspot.com

    Default Identify and Block autoposters, bots and scrapers

    How bots work
    A bot is basically a script written in PHP, C#, Python, or any other language.
    When run, it fetches a Web-page, looks for specific data, like download links, and optionally, leeches from it.
    Some bots, are just made to Post to sites.


    Disable API (Wordpress)
    If you have open registrations on a Wordpress site and are encountering SPAM, then you should disable
    Go to Admin> Settings> Writing> Remote Publishing and
    Uncheck the protocols for ‘Atom’, ‘XML-RPC’, or both.
    This is a n00bie technique, but will keep stupid posters away.
    This won't block screen scrapers.

    robots.txt
    You can use robots.txt and disallow bots.
    Most bots won't respect this but still, you should try this.

    Check Headers
    This is again, very stupid.
    You can check Headers like User-Agent, and Referer, and validate them Server-Side.
    Almost all posters and scrapers will fake the Headers, but still, some people might be stupid enough.

    Use images instead of data
    You can use Images instead of posting email addresses as text.


    Captcha
    Most people use this to scare away bots. This is a good method, but you can't use this for every post a user makes. It will also irritate genuine users away.
    You can use this for Registrations, or for Logins.
    If you had to use one, I would suggest Recaptcha, instead of other alternatives like Keycaptcha.

    Random Questions
    You can ask random questions while posting, once per 24 hours.
    But what some people do is have a pool of 5 -10 Questions that keep rotating.
    This can easily be bypassed.
    Instead you should, try changing the Question daily.
    You should ask this only once, not for every post. Otherwise people will leave your site.
    You can hire a coder who can implement this. This is very easy.

    Resources like JS, CSS, Favicon
    When you type in google.com in your browser, and hit enter, a lot of things happen behind the scenes.
    First your browser requests for the Google.com Resource.
    It then requests other resources like Javascript files and CSS files, Favicon file.

    When bots, request pages, they usually only request(GET) the url resource, and ignore other resources, since they are useless to them.
    You can look for this behavior and Block these IPs

    Mouse Movement
    You can try to detect mouse movement by using Javascript. Because bots don't use a browser, they don't register any mouse movement.
    The Mouse movement cannot be faked, but the AJAX calls that the site makes can easily be faked.
    Still, this is a good technique
    More Info

    Text as canvas
    You can render text or Data using the canvas element.
    This is impossible for bots to see.
    For more info, see:
    Dive into html5
    CufonSite CufonTutorial CufonSource

    Data representation using Javascript
    Use Graphs and Charts to Represent data using Javascript.
    Most scrapers are Javascript agnostic and fail miserably.

    nonce
    You can use security tokens, to make it harder for bots.
    This will force the bots to visit a previous page first.
    For e.g., to post to wordpress (not through xmlrpc), you need to visit the New Post page before you post the post

    Fail softly
    Another thing you can do is, once you recognize a bot, you can give them bogus data, with the same Markup of course.
    For e.g. If you have a wordpress site and the bot asks for the Home Page, and you know it's a bot, then you can return a response with Lorem Ipsum text with the same Markup and the bot will never know.
    It will make it a lot harder for the bot writer to debug.

    Time between requests
    Bots are made to work fast. They don't pause between requests.
    Humans, require time to read and process data, but bots do this at lightning speed.
    You can check for IPs making requests with little or almost no time in between and flag them as bots.
    Humans can make maybe 10 requests per minute, whereas bots can go to more than 100 requests.
    This behavior is easily recognizable.

    AJAX and Javascript
    I know I have said javascript before, but this solution is so good, that it should be mentioned again
    Javascript is invisible to Bots, so make huge use of Javascript.
    Make AJAX calls in between, to verify that it's a real human.

    Randomization
    You can do a lot of Random things, like present recaptchas after a Random Interval.
    Ask Random questions after Random intervals.

    Change code frequently
    Change code like ids and classes of your HTML tags frequently.
    Most importantly change names of forms and input elements every few days.
    Change the Code every few days, without changing the way a site looks.
    Every time you make a change, the script programmer will have to make a change in his code.


    EDIT:
    There's no foolproof way you can block a bot, but following the above steps will keep 99% bots away.

    All this being said
    If I would have owned a site, I would've just provided an API
    ajinkya9 Reviewed by ajinkya9 on . Identify and Block autoposters, bots and scrapers How bots work A bot is basically a script written in PHP, C#, Python, or any other language. When run, it fetches a Web-page, looks for specific data, like download links, and optionally, leeches from it. Some bots, are just made to Post to sites. Disable API (Wordpress) If you have open registrations on a Wordpress site and are encountering SPAM, then you should disable Go to Admin> Settings> Writing> Remote Publishing and Uncheck the protocols for ‘Atom’, ‘XML-RPC’, or both. Rating: 5

  2.   Sponsored Links

  3.     
    #2
    Member
    Website's:
    premium-links.net wanted-programs.com
    everything can be automated m8, and bot exist everywhere, its a matter of time and level of coding thats it!

  4.     
    #3
    Member
    Captcha
    Most people use this to scare away bots. This is a good method, but you can't use this for every post a user makes. It will also irritate genuine users away.
    You can use this for Registrations, or for Logins.
    If you had to use one, I would suggest Recaptcha, instead of other alternatives like Keycaptcha.
    i think we can use select the flower or scissor or image instead of text captcha.......any way we are not able to eliminate completely, but under control

  5.     
    #4
    Member
    Website's:
    freelinkz.info
    u can use picture captha thats the best way or u can use dnp firewall in ur forum and so when eveen the user enter forum he has to answer the question he cant enter without escaoping the dnp firewall . I have seen both these method are good for many forums and they are big forums

  6.     
    #5
    Member
    is dnp firewall thing is available for mybb......?? not able to find one

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Stop & Ban Autoposters
    By JGM in forum vBulletin
    Replies: 14
    Last Post: 16th Jan 2012, 06:08 AM
  2. Stop / Ban Autoposters
    By Perl in forum IP.Board
    Replies: 44
    Last Post: 26th Dec 2011, 07:21 PM
  3. Blacklist (block bad bots)
    By shahpar in forum Server Management
    Replies: 1
    Last Post: 26th Dec 2011, 12:38 AM
  4. Replies: 0
    Last Post: 25th Aug 2009, 10:38 PM
  5. How to Get Rid of F* AutoPosters ?!
    By Mecho in forum Webmaster Discussion
    Replies: 15
    Last Post: 22nd Jul 2009, 05:42 PM

Tags for this Thread

BE SOCIAL