New Karma System (Theory)

[DragonlordP]

[dhn]

Dhn (I think) proposed a system that is not based in points, but on specific attributes (funny, helpful, etc.; much like slashdot) which could help guide the way people use a rating-system.

Indeed. This came up in the open feature discussion almost two years ago, and I still think that this would be the most useful implementation for most communities.

Anyway, my problem with Karma is that whenever you allow people to down-vote other posts, it will be abused. If users don't like a certain post, not voting for them is enough. If you like it, give it a thumbs up or a funny or informative rating. There is no need for a thumbs down. A post is either informative, or it isn't.

[vph]

In a Karma System what exactly should be rated:
- The like/dislike of a user? (personality-based?)
- The agreements/disagreement-factor? (meaning of the content of his posts)
- The quality of the discussion that his posts create?

There is no clear cut answer for this. On support forums it probably should be the second one, where the meaning/content will be the most interesting (is it correct? is it complete?). And on forums that try to be a a high quality platform for discussions, you want people to rate according to the third: Did the user not use personal attacks or unfair rhetorics and did he give a well reasoned explanation and arguments for his opinion? On forums that are mainly oriented to build communities of people and where talking offtopic and killing time is the main attraction, the ratings will tend to be personality-based.

The problem is, that this is often not a clear-cut case, and the even bigger problem is, that users will even be a lot less likely to give any thought to it. What you will get, is a lot of users rating for a lot of different reasons. And what is the meaning of the average of that? I'd say there isn't any. Some pointed out that "karma" fits so well, because it is so vague, IMHO that's the reason why it will not work very well.

Dhn (I think) proposed a system that is not based in points, but on specific attributes (funny, helpful, etc.; much like slashdot) which could help guide the way people use a rating-system. Another helpful thing IMO is (and I know a lot of people will disagree), that only a closed group of people are allowed to rate. "Forum-Membership" alone is to loose a thing to define who can vote who can't, because it is too easy to create sockpuppets and the like. With a closed group that gets appointed e.g. by the the forums' admin/mod-team, you can more or less be sure that if you only choose trusted and mature members, that there will be little gaming int the system. A side-effect that some people will not like (but which I find is a positive feature) will be that that appointed group will in a way set the tone of the forum - it will "choose" what is exemplary and/or accepted behaviour and what is not.

You raised some good points. But I'd like to argue that the "vague" notion of what "karma" is, is actually a good thing. First, let's address a question. Is it fair to assume that the theme of a (sub)forum is unique? If it is so, then the judgement of threads in that forum will be based majorly on one attribute. Is this a fair assumption?

The advantage of 'karma' is that it is a meta-attribute; it could be anything. If, on the other hand, we clearly vectorize rating in terms of concrete attributes, like usefulness, funniness, etc., I'd like to argue that it's either complicated (multi-dimensional) or redundant (most of the time only one dimension is rated). The advantage of 'karma' is in its polymorphism. We dont have to make a judgement what the criteria upon which topics in a given forum should be judged (and hence attempt to break them down into discrete attributes). As long as, the assumption that topics in a forum follow a theme holds, then the metamorphic, singly-attributed notion of 'karma' works just fine, I'd like to argue.

So then, the next question is exactly what a karma rating of a user is. It is not usefulness, nor funniness. It is exactly the degree of significance based on the unique theme/character of that forum. This is to me very reasonable. If a forum is simply chit-chat forum, then karma is perhaps a measure of being liked or not. If a forum is a technical forum, then karma is perhaps a measure of usefulness. And so on and so forth.

The most important question of all is whether all users of a forum will judge topics according to ONE attribute (whatever that is). It's a difficult question. When the theme of a forum is well defined, and when there is only one attribute to judge the topics by, I tend to be optimistic that users will come to terms with themselves to judge according to ONE common attribute, whatever that is. To them that is ok. That is how they conceptualize what 'karma' is.



PS: just for side technical issues. I think karma weight shouldn't be a interger, but rather a number, say between 0 and 1. The advantage is you can view is as probability and update karma weights according bayes' theorem or something like that. Also, karma rating should be an optional feature.

[rivaldo]

Because a karma rating system is not implemented on this board or in this fourm, all I can say is "Excellent post, vph!"

[Magnotta]

Here's my ideas for Karma system. First, you can vote on posts, or users, whichever, and you can give the good or bad ratings. Seems to be just what everyone else has said so far. However, keeping in mind the actual idea of karma, if you vote someone's post bad, your karma should also decrease a bit, while if you vote good, your karma increases a bit. Make the amount of karma the voter gains/loses also change. For instance, if a voter gives a vote of 6 to a post, give him say 3 karma points, but if he gives it a 9, give him only 1. This would reduce the idea of voting high just to try and get karma points, but then say someone figures it out and starts voting low. Well, then, compare it to the other votes given, specifically against those who vote low. Say for instance when you give someone a rating of 4, you loose 1 point. But say then 3 others also vote a 4, and then a user comes along and gives a 6. The guy giving a 6 is clearly doing so to get karma points, so then take some karma points away from him, and give some karma points to those who originally lost by voting bad, as since so many people voted bad and willingly gave up karma points to admit it's bad, chances are that they're telling the truth. This way, when someone votes, they have to be honest if they actually care about whatt heir own karma is, because there is no definite "it will go up if I do this, or down if I do this".

It be very complicated thing to implament, since there would have to be so much checking in order to tally the final karma points given, but it also stays more true to what karma is: if you wish bad things upon people, then bad things will happen to you(loosing karma when voting bad), and if your greedy bad things happen(voting extremely high, or high when everyone else says low, just to get karma points), and also good things go to those who are honest(if you vote bad, you loose points, but if a bunch of people, say more than 10, vote bad, then it probably deserved to be voted bad and you were being honest, compared to the lone guy who was greedy and tried to get points by voting good, and so instead you now gain karma points).

If anyone doesn't exactly follow what I mean, let me know and I'll try and make it clearer.

[MrTufty]

That sounds way too complicated....

[R. U. Serious]

I think we will have to agree to disagree. Just to illustrate my POV:

But I'd like to argue that the "vague" notion of what "karma" is, is actually a good thing. First, let's address a question. Is it fair to assume that the theme of a (sub)forum is unique?

I think for the majority of forums this assumption seems valid.

If it is so, then the judgement of threads in that forum will be based majorly on one attribute. Is this a fair assumption?

IMHO not at all. Most people will lump together the three factors I tried to diffferentiate. A lot of people have a very simplistic world view of black vs. white, good vs. evil, with me vs. against me. In a hypothetical scenario (which is how those people see thew world) you would have two groups of people:

Group1:
- nice and well behaved people,
- friendly to others
- can make a well reasoned argument
- illustrates his opinion carefully
- does not use rhetorical mud slinging etc.
- is on the only true/right side (that is "my" side) of the topic

Group2:
- annoying people, unfriendly
- like to troll
- yell and scream
- either unable to articulate any thought, or slick weasel that uses rhethorical tricks to put words in your mouth, attack you etc.
- is on the wrong (that is "their" side) of the topic

Better illustrated here:
"Don't talk while I'm interrupting"
http://radio.weblogs.com/0107946/storie ... pting.html" target="_blank

In reality of course people "score" very different on each of those characteristics. And more so, for single posts. Unfortunately I don't know many people that will give credit to someone defending the opposite position of oneself.

IMHO you make the mistake of confusing what "should be" with "what is"/"what will be". People get emotional fast, and then a simple good/bad rating will be used in a very different way than what it "should" be used for in general in that forum.

If, on the other hand, we clearly vectorize rating in terms of concrete attributes, like usefulness, funniness, etc., I'd like to argue that it's either complicated (multi-dimensional) or redundant (most of the time only one dimension is rated).

I agree that mutlidimensional would be too complicated. But I think one-dimensional with a specific attribute is still better than one-dimensional with no attribute except "The Good" vs. "The Evil".

The advantage of 'karma' is in its polymorphism. We dont have to make a judgement what the criteria upon which topics in a given forum should be judged (and hence attempt to break them down into discrete attributes). As long as, the assumption that topics in a forum follow a theme holds, then the metamorphic, singly-attributed notion of 'karma' works just fine, I'd like to argue.

That is just a hyped up way of saying, we don't try to give it a meaning, hence it will mean whatever it means. Which IMHO is a serious flaw. If all members were equally rational and homogeneous in that they would interpret and use it in the same way, than I might agree with you, however that assumption is not a valid one to take.

The most important question of all is whether all users of a forum will judge topics according to ONE attribute (whatever that is). It's a difficult question.

It is only difficult if you try to answer the question yes. Even if the forum is well defined, you won't get to well-define the people that take part.

When the theme of a forum is well defined, and when there is only one attribute to judge the topics by, I tend to be optimistic that users will come to terms with themselves to judge according to ONE common attribute, whatever that is. To them that is ok. That is how they conceptualize what 'karma' is.

I have solved the problem of most efficient milk producing cows. First we assume a perfectly spherical cow... ( )

[theamazingTWOeyedman]

So what would happen if i only rated bad posts?

[Magnotta]

So what would happen if i only rated bad posts?

Well, in the system I mentioned above, your karma would go down, unless you and a bunch of people vote bad for that topic, and then someone comes along and votes really high in an attempt to boost their own karma, in which case, yours would go up while their's would go down(in the true sense of the word "karma").

[MKruer]

PS: just for side technical issues. I think karma weight shouldn't be a interger, but rather a number, say between 0 and 1. The advantage is you can view is as probability and update karma weights according bayes' theorem or something like that. Also, karma rating should be an optional feature.

The original Spec was that everyone started off at 1 (minimum value) and would get maybe 0.1 per rank, moderators and Admins would be higher then any standard rank however it was not out of the realm of possibility that many people could overrule an Admin or moderator. Your idea would fit quite well inside the system outlined.