[RFC|Merged] Improve User Pruning feature

These requests for comments/change have lead to an implemented feature that has been successfully merged into the 3.1/Ascraeus branch. Everything listed in this forum will be available in phpBB 3.1.
User avatar
A_Jelly_Doughnut
Registered User
Posts: 1780
Joined: Wed Jun 04, 2003 4:23 pm

[RFC|Merged] Improve User Pruning feature

Post by A_Jelly_Doughnut » Thu Apr 29, 2010 3:48 am

The prune users feature currently in phpBB 3.0 has a few limitations. To wit:
  • Criteria is limited to a small subset of the data we have about users
  • The pruning process takes an exceptionally long time, on account of a huge number of database queries required
I am working on a patch to improve both of these problems, but before I do, does anyone have any further issues/suggestions?

---

Ticket: http://tracker.phpbb.com/browse/PHPBB3-9622
Code: https://github.com/p/phpbb3/compare/dev ... rune-users
A_Jelly_Doughnut

Oleg
Posts: 1150
Joined: Tue Feb 23, 2010 2:38 am
Contact:

Re: [RFC] Improve User Pruning feature

Post by Oleg » Thu Apr 29, 2010 4:42 am

For killing spam users I wrote a script that nuked users by user id range. It is inevitable that many boards start off with no anti-spam measures, get spammed, then start fighting spam, but by then they have a number of spam users that need to be removed.

A user-friendly way of addressing this problem is to have search for users return a list of results that have checkboxes next to each user, in a form, submitting which removes checked users.

At a minimum search criteria would need to include user group and date of registration.

Output should include username (spammers use auto-generated unreadable ones) and user post count, ideally with a link to view user's posts. Registration time, last post time and last visit time together could be a good indicator of a spammer.

There should be 'select all' and 'select none' js links.

It should be possible to configure page size or display all results on a single page.

User avatar
MichaelC
Development Team
Development Team
Posts: 889
Joined: Thu Jan 28, 2010 6:29 pm

Re: [RFC] Improve User Pruning feature

Post by MichaelC » Thu Apr 29, 2010 9:07 pm

EDIT: Misread.
Formerly known as Unknown Bliss
psoTFX wrote: I went with Olympus because as I said to the teams ... "It's been one hell of a hill to climb"
No unsolicited PMs please except for quotes.

User avatar
VSE
Extension Customisations
Extension Customisations
Posts: 670
Joined: Mon Mar 08, 2010 9:18 am

Re: [RFC] Improve User Pruning feature

Post by VSE » Mon May 10, 2010 6:38 pm

Perhaps an option to check against a hot-list of common SPAM names to prune users by?

My board is many years old, from early days of phpBB2 and there was almost no SPAM protection in the beginning. So I had a legacy of SPAM registrations... I deleted over 200 inactive users with these wildcards in their names:

*viagra*
*pharm* (as in pharmacy)
*britney*
*spears*
*casino*
*webcam*
*shave*
*doctor*
*search*
*herbal*
*p e n i s* (spaces added to prevent word censor here)
*online*
*adult*
*dick*
*web*
*ringtone*
*photo*
*cheap*
*free*
*sex*

What'd be cool is a button - Show possible SPAM registrations - which would show a list of all usernames with these (and other spammy) wildcards. Then you could select which one's to delete, considering their is a chance there might be some "real" members using some of these wild cards in their usernames.
Has an irascible disposition.

User avatar
A_Jelly_Doughnut
Registered User
Posts: 1780
Joined: Wed Jun 04, 2003 4:23 pm

Re: [RFC] Improve User Pruning feature

Post by A_Jelly_Doughnut » Tue May 11, 2010 3:49 am

Would that be especially useful?

Let's say I have two users valid: "sextant" and "NorelcoShaverFan" on my board, among 500 "spammy" registrants. It seems to me it would be a very bad user interface to present the user with a list of 502 users matching a wide swath of wildcards. And we already have wildcard-based pruning, so an admin could enter *viagra* and see who matches that.

However, I would see the utility in the following:
Admin enters *sex* in the "prune users matching this username" box. Admin sees the listing of matching users, showing certain statistics, like post count, number of posts in the queue, website URL, whatever. He can then remove certain users from the pruning list. So now he could prune *sex* and protect "sextant" from being pruned. (this user just likes circumnavigating the globe :mrgreen:). This would be a decent improvement on the current functionality, which is useless if there are legitimate users matching some wildcard.
A_Jelly_Doughnut

User avatar
VSE
Extension Customisations
Extension Customisations
Posts: 670
Joined: Mon Mar 08, 2010 9:18 am

Re: [RFC] Improve User Pruning feature

Post by VSE » Tue May 11, 2010 5:47 am

When I searched for these wildcards, it was also with the posts equal to 0 setting, so it was pretty obvious these were all inactive parked spam registrations.
Has an irascible disposition.

User avatar
A_Jelly_Doughnut
Registered User
Posts: 1780
Joined: Wed Jun 04, 2003 4:23 pm

Re: [RFC] Improve User Pruning feature

Post by A_Jelly_Doughnut » Tue May 25, 2010 4:43 pm

Code is now on GitHub:
http://github.com/jellydoughnut/phpbb3/ ... rune-users

Feedback is welcome, I'll be adding some sort of unit test before getting this into the main phpBB repo.
A_Jelly_Doughnut

MartinTruckenbrodt
Posts: 171
Joined: Sun Jan 29, 2006 1:00 pm
Location: Germany
Contact:

Re: [RFC|Accepted] Improve User Pruning feature

Post by MartinTruckenbrodt » Wed Jul 21, 2010 8:51 pm

Hello,
IMO it would better to include a true auto user prunning. I've realized it at the Advanced Double Activation Pack.
IMO the handling of the default phpBB3 prune users function is to strange and to hard to understand, at least for newbies. The risc for administrators to prune users and posts they don't want to delete is to high (in Olympus). If the administrators always will have done a database backup before? Although you have set the settings every time you are pruning users again. This is not very comfortable. I don't think the aim should be to use different settings every time you are pruning users.
I think you can reduce the settings and the combinations to the settings I'm using here: http://www.phpbb.com/customise/db/downl ... -mode_view .
I've included the pruning to the index.php. But the pruning will be started only one time per day or less as configured at the ACP.
Perhaps you could include a hierarchy for the settings. So that e.g. the lazy posters setting have to be higher or equal to the lost users setting.

Bye Martin
Advanced Block MOD 1.1.1 has been released! - Prevent spam on your phpBB3 board with Stop Forum Spam, BotScout, Akismet, Project Honey Pot and several IP-RBL and Domain-RBL DNS blacklists! - My MODs

Oleg
Posts: 1150
Joined: Tue Feb 23, 2010 2:38 am
Contact:

Re: [RFC|Accepted] Improve User Pruning feature

Post by Oleg » Sun May 08, 2011 7:26 am

I managed to get this up-to-date with develop via creative use of merging.

https://github.com/p/phpbb3/compare/dev ... rune-users

Redundant sql_in_set construction needs to be fixed and this should be ready for testing.

Oleg
Posts: 1150
Joined: Tue Feb 23, 2010 2:38 am
Contact:

Re: [RFC|Accepted] Improve User Pruning feature

Post by Oleg » Wed Jun 22, 2011 8:28 am

Thanks for testing. Myself I'm not sure I ever actually looked at this patch in action.

Going back to my previous post it looks like I did eliminate the redundancies mentioned back in May.

Therefore this patch should be ready for testing and merging, and testing is already under way.

My plan was to merge the patch and mark the RFC finished. The other ideas that have been proposed have no code. If people want them implemented RFCs should be created for 3.2+.

Post Reply