[RFC] stop distributing worthless CAPTCHAS in 3.1

Note: We are moving the topics of this forum and it will be deleted at some point

Publish your own request for comments/change or patches for the next version of phpBB. Discuss the contributions and proposals of others. Upcoming releases are 3.2/Rhea and 3.3.
Post Reply
User avatar
/a3
Registered User
Posts: 97
Joined: Mon Sep 20, 2010 6:44 am

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by /a3 » Sat Nov 24, 2012 6:02 am

With regards to Q&A, it's possible to see if your question is on XRumer's list (XRumer is a popular forum spamming program).

You can download the trial of their program from their website (I would suggest not running any executables though). I used a fake email address to download it.

Last time I checked they had a file which had a few thousand questions/answers in it. There's probably a lot more in it now though. It's only a matter of time before someone uses the file to automatically check if a Q&A is safe or not... ;)
$ git commit -m "YOLO"

User avatar
Pony99CA
Registered User
Posts: 986
Joined: Sun Feb 08, 2009 2:35 am
Location: Hollister, CA
Contact:

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by Pony99CA » Wed Nov 28, 2012 10:15 pm

ecwpa wrote:So, Q&A without defaults is the way to go.
Probably, but you might want to read the hammered by newly registered members topic on phpBB.com. Q&A isn't as safe as it was before.

Steve
Silicon Valley Pocket PC (http://www.svpocketpc.com)
Creator of manage_bots and spoof_user (ask me)
Need hosting for a small forum with full cPanel & MySQL access? Contact me or PM me.

User avatar
brunoais
Registered User
Posts: 964
Joined: Fri Dec 18, 2009 3:55 pm

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by brunoais » Thu Nov 29, 2012 9:54 am

I think this is something we need while doing this:
Stef775 wrote:Why can't we log which question is answered right by a spammer, so you know the broken question
(source: https://www.phpbb.com/community/viewtop ... #p13201422)

User avatar
brunoais
Registered User
Posts: 964
Joined: Fri Dec 18, 2009 3:55 pm

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by brunoais » Thu Nov 29, 2012 11:53 am

K'ay. I got an idea that just might work.

So... What is the thing that all these forum "spammers" do? They register, activate and then start spamming. Right?

There's one interesting thing in here. There's seems to be an interesting pattern in the posts they make.
Sounds like, for all these users, they do not make any useful post (we can think of 3 posts, just to be sure) before they start spamming.

We could make a message checker system that would work with the same idea as thunderbird's message evaluator (it uses it to evaluate if an e-mail is junk or not).

Code: Select all

For each post from a user with less than 4 posts post:
	var words = break the post into individual words.
	var urls = also we look for all urls
	var urlImages = also we look for all urls that are images.	

	Foreach (words as word)
		Evaluate the probability that that word is related to spam.
		Compare it to the previous one
		Compare it to the next one
		Associate a number to it. (That number is the probability that that word is related to spam).
	
	Then do the same about the urls.
	Then do the same about the image urls.
	The objective is to see if it contains words like buy or it's a bit.ly (or something like that)
	
	Then gather the final result. It is a number between 0% and 100% (we can use other scales, if it makes sense)
	
	Using a threshold, we can then separate what to do with that post.
This is kinda resource heavy but it seems to be quite effective. At least with thunderbird, it works like a charm to me!

We can do this in a more accurate and strict way in the form to contact the admins.
We create a form that, while the user is not in a "accepted" state (has, at least, 3 accepted posts, or so) his request to the admins is filtered. Only certain kind of stuff is allowed.

Yeah... bots can go through these protections but they can only pass with content they don't want to insert. All these bots seem to be only interested into placing ads of some sort. Ads have patters. We may gain something in here! This will not prevent spam but it will prevent the kind of spam all these bots want to place in the BB's.

What do you think?

User avatar
callumacrae
Former Team Member
Posts: 1046
Joined: Tue Apr 27, 2010 9:37 am
Location: England
Contact:

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by callumacrae » Thu Nov 29, 2012 6:09 pm

brunoais wrote:K'ay. I got an idea that just might work.

So... What is the thing that all these forum "spammers" do? They register, activate and then start spamming. Right?

There's one interesting thing in here. There's seems to be an interesting pattern in the posts they make.
Sounds like, for all these users, they do not make any useful post (we can think of 3 posts, just to be sure) before they start spamming.

We could make a message checker system that would work with the same idea as thunderbird's message evaluator (it uses it to evaluate if an e-mail is junk or not).

Code: Select all

For each post from a user with less than 4 posts post:
	var words = break the post into individual words.
	var urls = also we look for all urls
	var urlImages = also we look for all urls that are images.	

	Foreach (words as word)
		Evaluate the probability that that word is related to spam.
		Compare it to the previous one
		Compare it to the next one
		Associate a number to it. (That number is the probability that that word is related to spam).
	
	Then do the same about the urls.
	Then do the same about the image urls.
	The objective is to see if it contains words like buy or it's a bit.ly (or something like that)
	
	Then gather the final result. It is a number between 0% and 100% (we can use other scales, if it makes sense)
	
	Using a threshold, we can then separate what to do with that post.
This is kinda resource heavy but it seems to be quite effective. At least with thunderbird, it works like a charm to me!

We can do this in a more accurate and strict way in the form to contact the admins.
We create a form that, while the user is not in a "accepted" state (has, at least, 3 accepted posts, or so) his request to the admins is filtered. Only certain kind of stuff is allowed.

Yeah... bots can go through these protections but they can only pass with content they don't want to insert. All these bots seem to be only interested into placing ads of some sort. Ads have patters. We may gain something in here! This will not prevent spam but it will prevent the kind of spam all these bots want to place in the BB's.

What do you think?
They will simply start duplicating old topics (such as introduction or GD topics) for a few topics before spamming. It won't work on small forums, but it will on bigs ones where the moderators don't read every single post.

Also, instead of doing this ourselves, why not just use Akismet?
Made by developers, for developers!
My blog

User avatar
Pony99CA
Registered User
Posts: 986
Joined: Sun Feb 08, 2009 2:35 am
Location: Hollister, CA
Contact:

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by Pony99CA » Thu Nov 29, 2012 11:33 pm

callumacrae wrote:Also, instead of doing this ourselves, why not just use Akismet?
:lol: I was going to ask the same thing myself. It sounds like brunoais is just trying to reinvent the spam filter. That's why E-mail spammers started including random snippets of prose (to lower their spamminess).

On another topic, does anybody know if spambots like Xrumer accept cookies?

Steve
Silicon Valley Pocket PC (http://www.svpocketpc.com)
Creator of manage_bots and spoof_user (ask me)
Need hosting for a small forum with full cPanel & MySQL access? Contact me or PM me.

User avatar
callumacrae
Former Team Member
Posts: 1046
Joined: Tue Apr 27, 2010 9:37 am
Location: England
Contact:

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by callumacrae » Fri Nov 30, 2012 7:05 am

Pony99CA wrote:On another topic, does anybody know if spambots like Xrumer accept cookies?
They would if they had to
Made by developers, for developers!
My blog

User avatar
Pony99CA
Registered User
Posts: 986
Joined: Sun Feb 08, 2009 2:35 am
Location: Hollister, CA
Contact:

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by Pony99CA » Sat Dec 01, 2012 2:33 am

callumacrae wrote:
Pony99CA wrote:On another topic, does anybody know if spambots like Xrumer accept cookies?
They would if they had to
Sure, but I'm wondering if they do now. If not, phpBB could probably set a random cookie on the Registration Agreement page and check that it was set on the Registration form page. If it wasn't set, display an error.

Yes, that would prevent people who turn cookies off from registering, but they could easily turn cookies on.

Steve
Silicon Valley Pocket PC (http://www.svpocketpc.com)
Creator of manage_bots and spoof_user (ask me)
Need hosting for a small forum with full cPanel & MySQL access? Contact me or PM me.

Oleg
Posts: 1150
Joined: Tue Feb 23, 2010 2:38 am
Contact:

Re: [RFC] stop distributing worthless CAPTCHAS in 3.1

Post by Oleg » Sat Dec 01, 2012 4:37 am

Pony99CA wrote: If not, phpBB could probably set a random cookie on the Registration Agreement page and check that it was set on the Registration form page. If it wasn't set, display an error.
It is so trivial to read/forward cookies that doing anything like this is very much pointless.

User avatar
Master_Cylinder
Registered User
Posts: 361
Joined: Wed Jul 31, 2013 9:54 pm

Remove broken captcha options...

Post by Master_Cylinder » Tue Dec 31, 2013 4:09 am

Since spambots have defeated most of the captcha options, shouldn't those broken options be removed from the core so new admins don't select a method that doesn't work?


Discuss... :D
These kids today...
Buy them books, send them to school and what do they do?

They eat the paste. :lol:

Post Reply