New Karma System (Theory)

Discuss features as they are added to the new version. Give us your feedback. Don't post bug reports, feature requests, support questions or suggestions here.
Forum rules
Discuss features as they are added to the new version. Give us your feedback. Don't post bug reports, feature requests, support questions or suggestions here. Feature requests are closed.
Locked
MKruer
Registered User
Posts: 156
Joined: Sun Jul 20, 2003 9:01 pm

Re: New Karma System (Theory)

Post by MKruer »

Rocketeer wrote:
*Expletive Deleted*

I just typed a really clever, complicated method that solves some of the issues... then accidently hit the "Back" button on my keyboard (cunning located next to the up arrow) and lost the whole thing... Darn. Hopefully, I'll get it right again this time...
I have done that before. I once figured out the answer to life the universe and everything but completely forgot it. For some reason the number 42 keeps coming to mind.
Rocketeer wrote:
MKruer wrote: If there is a history being kept for each user that you vote on (say 20+ posts) then you can calculate the weight of that person, specifically. It would be the combined weight that would result in the real rating.
That's a pretty good idea... I think it would work, and get around the "nice to friends, nasty to enemies" issue, but I don't know if there would be too much data for it to be successfully implemented. I don't know enough about MySQL performance to comment too much, but it could quickly grow out of control - anyone know better (say a 3 column table, with User1, User2 and Average Karma)?
I believe the max value for a record table and SQL is 4 billion (2^32) currently, so this could be a huge problem.
Rocketeer wrote:
MKruer wrote: So something like for ever positive response give a add a multiplier ratio value?
I was more thinking along these lines...
  • User A posts a message - no karma change
  • User B rates it 7 - no karma change, he could be being nice or nasty, no way of knowing
  • User C rates it 3 - still no karma change, because there's no way of knowing which of B and C was being accurate.
  • User D rates it 8 - now B and D should be rewarded, and/or C punished, because the consensus is clearly around the 7/8 mark, not 3.
One way of doing this would be to record the Min, Max and Average karma rating for each post. The average should ignore the minimum and maximum rating - so you'd need at least 3 ratings before the post had an average. If user X rates a post above the current Max or below the current Min, he should get no karma weight change. If he rates it within (say) 2 points of the average, he should get a positive karma weight change.
This way, if one person has an extreme opinion on the post, it will get ignored, but if there's an even split through the community, the average will reflect that.
The problem I see with this is that I feel you are inherently punishing people if anyone marks lower than the average. With what I was trying to do I was just trying to set up a ratio between good and bad but not punish people for making those marks.
Rocketeer wrote:
MKruer wrote: The big trick is to make it impossible to go above 10 or below 0.
True... but you can cheat by limiting the values in code, so that anything above 10 is reduced to 10, and anything below 0 is increased to 0... or you could just have an infinite scale...
Two things first by coding the system of not go above 10 or below 0 in my opinion is a brute force solution. If the system requires us to me this is obviously a wrong system. The system should be inherently limited to the range. This may be more difficult but in the end it will be much more elegant and most likely much simpler to code. The trick is to find a way to accomplish the desired results simplistically.
Rocketeer wrote: Hmmm... you've got me thinking a lot about this... which is silly, because I would probably never implement a karma system on any of my forums (my users would try to abuse it, fail, then get bored and not bother with it at all).
I feel this is partially true. However that is why I specified just by posting a reply the minimum value the preceding post would receive is a five. This would mean that the post was considered average and not worth judging it either way however if someone didn't take the time to rate the post this rating could then be used to find relevant information based upon what other people have read.

if you'd like to help perhaps it would be best to start building all the equations for the system from the ground up this would mean going back to the very fundamental rating system and how to determine weight of authority.

MKruer
Registered User
Posts: 156
Joined: Sun Jul 20, 2003 9:01 pm

Re: New Karma System (Theory)

Post by MKruer »

before the very first code is even written for the system this system needs to be thought out thoroughly and that is why I am discussing it here on this forum.

PS I would not get any credit for the post above because according to my system I reply to myself which means that I cannot use this as a way to inflate my weight on the system

User avatar
Rocketeer
Registered User
Posts: 20
Joined: Thu Sep 06, 2001 11:43 pm
Location: Ascot, UK
Contact:

Re: New Karma System (Theory)

Post by Rocketeer »

MKruer wrote:
Rocketeer wrote: I don't know enough about MySQL performance to comment too much, but it could quickly grow out of control - anyone know better (say a 3 column table, with User1, User2 and Average Karma)?
I believe the max value for a record table and SQL is 4 billion (2^32) currently, so this could be a huge problem.
It's not so much the
limits
of SQL (which would allow 2^16, or 65,000 fully active users to rate everyone), as the performance I'm worried about... although that said, there are 177,507 users on http://www.phpBB.com" target="_blank, so if people are even 50% active it could fall over 8O :(
MKruer wrote: The problem I see with this is that I feel you are inherently punishing people if anyone marks lower than the average. With what I was trying to do I was just trying to set up a ratio between good and bad but not punish people for making those marks.
Not so... remember, the "Karma Weight" of a person is hidden, and is not the same as their acutal "Karma" (if I've understood you correctly)... thus all I'm doing is giving more "weight" (or responsibility, as their votes will count for more) to users who "fit in" well with other users, and not rewarding those that feel a need to be extreme. My way doesn't punish people for marking low - it only punishes them for marking
really
low, so it encourages taking a middle road rather than the extreme path...
MKruer wrote: Two things first by coding the system of not go above 10 or below 0 in my opinion is a brute force solution. ... The system should be inherently limited to the range.
Of course... the other option is there though...
MKruer wrote: if you'd like to help perhaps it would be best to start building all the equations for the system from the ground up this would mean going back to the very fundamental rating system and how to determine weight of authority.
To be honest, I think we need to go back one step further... Having read through this a couple more times, I have a feeling we might be at cross purposes slightly...

As I see it, there are two separate uses for the "Karma" system.
First, is a way of indicating how useful a given post is - good Karma should mean it's relevant to the topic, bad Karma indicating that it's spam, off topic or generally useless.
Secondly, Karma also seems to be an indicator of whether a given user is generally useful, on topic, and helpful...

Let me see if I can list and define all the terms we're using - once we agree on terminology, we can move on to specifics...
(User) Karma Weight: How important/trustworthy are the user's Karma ratings. I think we've defined this as something along the lines of the average Karma rating a user gives (possibly to a specific other user to avoid the Friends/Enemies problem)
Weight of Authority: This (which I have to admit I haven't really mentioned in my posts) seems to be a second factor used to calculate how important a given rating is, based on posts and user level rather than other Karma ratings the user has given.

Post Karma Rating: This would be the average Karma rating for the post.
Post Karma Weight: This (I guess) would be used to work out the average karma for a thread.

Thread Karma Rating: This would be the average Karma rating for an entire thread.
Thread Karma Weight: I don't quite get what this would be used for, but I assume it works like the User and Post Karma weights...

User Karma Rating: This would be the Karma of the actual user. I don't think we've defined how this is calculated, but I also don't think it's used for anything other than as an extra field in the user profile...

I think I've covered all of the terms we're using... let's make sure we're all agreed on the meaning of the terms, then we can start on what affects each of the values, and finally the exact methods used to affect them.
The Hippocrates Oath - First, do no harm - applies to support as well as medicine!
University of Warwick - Samurai Jiu Jitsu

BioALIEN
Registered User
Posts: 120
Joined: Tue Jan 25, 2005 3:46 pm
Location: London, UK

Re: New Karma System (Theory)

Post by BioALIEN »

And the question is:

Why should this be included in the vanila phpBB3? If this is a MOD, then it will be very difficult to write (hence the amount of text used to explain its functionality) ;)

It is impossible to determine its use. As mentioned in one of the posts in this thread, users will likely mark their friend's posts with a high rating making this mod an unfair play for the helpful users.
BioALIEN
█ Secure your name before someone else does: TheMillionDollarAdvert.com
█ Get your permanent listing from $10 USD. You too can own a piece of internet history.
The moment you master the art of programming is the moment you program the art itself! - BioALIEN

SamG
Registered User
Posts: 1241
Joined: Fri Aug 31, 2001 6:35 pm

Re: New Karma System (Theory)

Post by SamG »

I might be wrong, but I believe the developers said the karma system that once was in 2.1 was deferred, and the reason it was deferred was because the existing implementation of karma wasn't satisfactory - hence the potential value of a discussion such as this.
"I hate trolls!" - Willow Ufgood

MKruer
Registered User
Posts: 156
Joined: Sun Jul 20, 2003 9:01 pm

Re: New Karma System (Theory)

Post by MKruer »

BioALIEN wrote: And the question is:

Why should this be included in the vanila phpBB3? If this is a MOD, then it will be very difficult to write (hence the amount of text used to explain its functionality) ;)
It's not in the plain vanilla phpBB3. and as to the difficulty in trying to write, it is always more difficult to spell to the function then to write the equation, hence the reason why math was invented. Don’t worry it will work out.
BioALIEN wrote: It is impossible to determine its use. As mentioned in one of the posts in this thread, users will likely mark their friend's posts with a high rating making this mod an unfair play for the helpful users.
Not true. Because unrated, but replied posts, are assumed to be average, any change even by one user can have an impact. Like everything there will always be someone trying to break the system, but I think your assuming the only reason why someone would rate anything is that they think it is a game, and they want the highest score. I am and saying that the max score is 10 and you will ALWAYS fall in-between it. After the initial frenzy of trying to break the system, I think that it then will be used in a more correct way, and as stated earlier can be used as a metric to help find relevant information in a post.

MKruer
Registered User
Posts: 156
Joined: Sun Jul 20, 2003 9:01 pm

Re: New Karma System (Theory)

Post by MKruer »

SamG wrote: I might be wrong, but I believe the developers said the karma system that once was in 2.1 was deferred, and the reason it was deferred was because the existing implementation of karma wasn't satisfactory - hence the potential value of a discussion such as this.
Exactly. I think everyone can see the possible benefits form a rating system. however like most systems, when originally designed, the creator would never think that the system would be used in “bad” ways, and when they go to make a change to the system they don’t see the full ramification of the change and often end up making it worse.

This thread was started to try to figure out how people might break the system, without having to write the system. It end up that there are quite a few simple ideas that when working together can eliminate most if not all the possible problem from previous systems.

P.S. Thanks for not locking this thread I was not sure how it would be received.

MKruer
Registered User
Posts: 156
Joined: Sun Jul 20, 2003 9:01 pm

Re: New Karma System (Theory)

Post by MKruer »

Rocketeer wrote:
MKruer wrote:
Rocketeer wrote: I don't know enough about MySQL performance to comment too much, but it could quickly grow out of control - anyone know better (say a 3 column table, with User1, User2 and Average Karma)?
I believe the max value for a record table and SQL is 4 billion (2^32) currently, so this could be a huge problem.
It's not so much the
limits

of SQL (which would allow 2^16, or 65,000 fully active users to rate everyone), as the
performance

I'm worried about... although that said, there are 177,507 users on http://www.phpBB.com" target="_blank, so if people are even 50% active it could fall over 8O :(
Well that’s a constraint that we have to work within. It might be nice to keep the history indefinitely, but if not oh well.
Rocketeer wrote:
MKruer wrote: The problem I see with this is that I feel you are inherently punishing people if anyone marks lower than the average. With what I was trying to do I was just trying to set up a ratio between good and bad but not punish people for making those marks.
Not so... remember, the "Karma Weight" of a person is hidden, and is not the same as their actual "Karma" (if I've understood you correctly)... thus all I'm doing is giving more "weight" (or responsibility, as their votes will count for more) to users who "fit in" well with other users, and not rewarding those that feel a need to be extreme. My way doesn't punish people for marking low - it only punishes them for marking
really

low, so it encourages taking a middle road rather than the extreme path...
I still think that this is a form of punishment, because what happens if they are making a valid point. This comes from the idea that you need to avoid “group think” or perhaps a better example would be a whistle blower. Why should their claim be less valid then everyone else’s.
Rocketeer wrote:
MKruer wrote: if you'd like to help perhaps it would be best to start building all the equations for the system from the ground up this would mean going back to the very fundamental rating system and how to determine weight of authority.
To be honest, I think we need to go back one step further... Having read through this a couple more times, I have a feeling we might be at cross purposes slightly...
As I see it, there are two separate uses for the "Karma" system.
First, is a way of indicating how useful a given post is - good Karma should mean it's relevant to the topic, bad Karma indicating that it's spam, off topic or generally useless.
Secondly, Karma also seems to be an indicator of whether a given user is generally useful, on topic, and helpful…
Yes I know that I made it somewhat confusing by calling everything Karma this Karma that… For this I am sorry.
Rocketeer wrote: Let me see if I can list and define all the terms we're using - once we agree on terminology, we can move on to specifics...

(User) Karma Weight

: How important/trustworthy are the user's Karma ratings. I think we've defined this as something along the lines of the average Karma rating a user gives (possibly to a specific other user to avoid the Friends/Enemies problem)
Correct.
Rocketeer wrote:
Weight of Authority

: This (which I have to admit I haven't really mentioned in my posts) seems to be a second factor used to calculate how important a given rating is, based on posts and user level rather than other Karma ratings the user has given.
Correct where I guess now (User) Karma Weight: would be on a user by user basis, Weight of Authority: is a global basis. Meaning that what they say, because they post more, should matter more, however because of how the (User) Karma Weight: is built, even those they have more power, if they consistently mark someone low the (User) Karma Weight: nullifies or reduces the effect to that person.
Rocketeer wrote:
Post Karma Rating

: This would be the average Karma rating for the post.
Correct.
Rocketeer wrote:
Post Karma Weight

: This (I guess) would be used to work out the average karma for a thread.
Sort of. Its by itself its more used to determine the relevancy of the rating. The more Weight, the more relevant.
Rocketeer wrote:
Thread Karma Rating

: This would be the average Karma rating for an entire thread.
Correct
Rocketeer wrote:
Thread Karma Weight

: I don't quite get what this would be used for, but I assume it works like the User and Post Karma weights...
Again, the more Weight, the more relevant the thread would be.
Rocketeer wrote:
User Karma Rating

: This would be the Karma of the actual user. I don't think we've defined how this is calculated, but I also don't think it's used for anything other than as an extra field in the user profile...
This is a base line indicator when the user creates a thread, or a post. It is used based upon past history (other users feedback) to determine if this is something relevant to the discussion. Example: Joe Blow creates a thread but has a rating of 2. This means that historically he has not had many relevant discussions, however is Sally Something replies to him, the thread rating goes up. (2+5)/2= 3.5 this is a 150% increase just by acknowledging that what Joe Blow posted is worth something. Now Sally could override the default 5 by rating Joe Blow post directly, in which case this would increase or the post rating. This is where it probably gets confusing, all the people that rate Joe Blow, either directly or indirectly change his default user Karma Rating. (I.E. this is a stating point when the person posts.)
Rocketeer wrote: I think I've covered all of the terms we're using... let's make sure we're all agreed on the meaning of the terms, then we can start on what affects each of the values, and finally the exact methods used to affect them.
I would like to append
User Karma Rating(S): => Post Karma Rating:
Post Karma Rating(S): => Thread Karma Rating:
Same as
User Karma Weight(S) => Post Karma Weight:
Post Karma Weight(S) => Thread Karma Weight:
And
User(s) Karma Rating(S): for a specific user, become that User’s (NEW) Karma Rating for their next post.
Hope that make sense.

User avatar
Rocketeer
Registered User
Posts: 20
Joined: Thu Sep 06, 2001 11:43 pm
Location: Ascot, UK
Contact:

Re: New Karma System (Theory)

Post by Rocketeer »

MKruer wrote:
Rocketeer wrote:
Thread Karma Weight
: I don't quite get what this would be used for, but I assume it works like the User and Post Karma weights...
Again, the more Weight, the more relevant the thread would be.
This is the one I don't quite get - the more relevant to *what*?
The Hippocrates Oath - First, do no harm - applies to support as well as medicine!
University of Warwick - Samurai Jiu Jitsu

DragonlordP
Registered User
Posts: 180
Joined: Sun Nov 16, 2003 10:51 am

Re: New Karma System (Theory)

Post by DragonlordP »

karma systems are as lame as it gets, i hope the developers prefer to work on the 432544 features that are more important than this. just my 10 dollars.
you live it or lie it

http://www.electricrequiem.com" target="_blank
greek metal forums and more

Locked