Post Ratings: Based on Previous Reputation Suggestion

Even if it was possible to fully emulate vB's rep system in Xenforo, there are a number of users who enjoy the current like system of Xenforo just like there are a number of users who preferred the old reputation system on vB. The Post Ratings add-on is, as Misty mentioned, supposed to be a compromise between the two.

 

I know this is highly subjective, but I and many others felt that rep helped set the atmosphere of KHV. New KHV... And just about every change that was made, was negative, in my opinion. In general, the atmosphere became less community-oriented and a lot of people got pushed away...

I think the calls for the old rep system are happening because people wanted the familiar.

I realize you won't change for me, but I have two very important requests.

Please hide ratings on specific content and limits per user and per day. Do not allow just anyone to view ratings on a post. See my earlier post for reasons. I think both of them concern both the people who enjoy likes and the people who enjoy rep. They are two things we can agree on.

 

I like the idea of seeing who rated a post, publicly at that. For negative ratings, as we said, we felt that it being public would deter abuse of the system, which was far easier with negative rep -- it was hidden away in profiles, so people felt there was little consequence to making petty dereps because no one but the receiving member would see it. As I told you, there is no way to limit it and I don't see much of a reason to. You are right that it would encourage people to give them out more freely but I do not think that cheapens the value of the ratings. The rest of what you have said is extremely subjective and, to be frank, a rather narrow view as you have been away from the community for a time, yet it was a common complaint from a few users when we moved to Xenforo. I will reiterate what I said then: we changed software for a lot of very good reasons. We are not trying and do not wish to clone vBulletin or keep changing things in deference to the past. It's important to consider what we liked about past systems, but we will not be bound by them, particularly when they had some very glaring issues.

 

Why do you like the idea?

About staff moderation: it is easy to filter visibility below posts by user group. Feel free to edit a template so that it shows only to staff. I have no problems so long as it doesn't cause the Matthew effect.

What does it offer that the Matthew effect does not turn into a cognitive bias inducing tendency?

I don't want to upset you, but I think you have gotten caught up in staff interests and left the user interests behind. I have not heard anyone but staff say that the changes were wholesale, good. But do not take my word for it.

It would help both of us if these points were put to a poll. Relevant questions include:

Should we limit ratings per user/per day?

Should other non-staff members be able to see your ratings on specific posts, either given or received?

It would also help if these polls did not include 'don't care'. I believe 'not caring' is the actual reason for the acceptance we see. Not allowing that and forcing people to take a stance will give a more accurate read of user preference.

 

There's an add-on I mentioned earlier that can limit the amount of likes you can give per day & is also functional with Post Ratings. It's only $10 for a lifetime copy of it, so I can foot the bill on it myself. That said, it only limits the number you can do per day. It can't be limited on a user basis. However, all that considered, if there is evidence of someone liking only the content of a certain person and/or people repeatedly, it will be pretty simple to take action against them if necessary.

Like Misty said, the negative rating counts are only displayed on that sidebar on user profiles, which can be enabled/disabled whenever.

I do have to say that I don't think halving it wouldn't be too necessary due to how the add-on functions. Even if someone were to give someone a negative rating, that would not effect the amount positive ratings they received:



They will still have the same number of positive ratings, but they'll also have an amount of the negative ratings. The ratings aren't in a ratio, but more along the lines of standalone values.

I'm not sure if this is possible within the functionality of the add-on/other add-ons, but I'll try to look into it. I do see where you're coming from, however, I do want to go back to points Misty and Nova raised:

With the daily limit and the public element applied, I think there could be a greater tendency to put more thought into what you're giving a rating to. Like I said though, I'll look into it.

Misty's right. You can only do one rating per post. It's possible to undo a rating (pretty much how you can undo a like now), but it's impossible to 'Like, Useful, Informative, etc.' all at once on a single post.

This I am more strongly against. At the moment we're trying to keep the ratings down to a minimum size so we can see how they work on the forum, and about a month later I was planning on doing a check in (possibly survey) to see what everyone thinks & possibly open up for more ratings suggestions later on down the road. I also think having a large amount of ratings in general could end up potentially being overwhelming in the amount of choices avaliable.

I'd be fine with renaming Like and Dislike to something else, but segmenting them into a collection of different phrases would be too much currently.

We aren't using the default icons that come with it because they are ugly and unsightly. Llave is currently designing ones that will fit more in line with the site itself (and future skins for it). Without the icons it can look pretty empty (can add in a photo of this later), and imo designs like the current heart for Likes do look fit in pretty seamlessly with everything else around it.

 

Wait... Is there a reason we are NOT showing the user's overall ratings in the mini-profile, like with old rep?

That was one of the main things I wanted...

I'm sure I have a prepaid card with $5.00 and some change sitting around if that's really necessary.

 

I'm not too sure what you mean by this. What mini profile? o.o

And nah, it's fine. I still have money leftover that I'm not really using currently, so might as well dump it somehow, lol. Definitely appreciate the offer, though!

 

The mini-profile is the little bar beside each post with your username, avatar and post count.

Are we removing the bar and/or numbers in that mini-profile (sidebar)?

 

Currently, no, we'll still have the bar there when the add-on is applied to the main site. It could possibly be removed or changed in some fashion later on, but for the present it's gonna be there on the side below pins.

 

That's what I wanted.

I think that that is the only way that it should be visible to non-staff and uninvolved parties. For the reasons I stated.

 

This was probably geared towards Misty, but SNOOZE YA LOSE, MISTY.

I also like having them public; I think it would help to illicit a sense of accountability on the part of people who negatively (or positively) rate something. With Rep, the accountability was practically negligible because it was behind closed curtains -- admittedly, much more so back when it was fully anonymous. With public ratings, there will be that sense of "I neg./pos. rated X member for the 15th time in a row, maybe I should stop," as well as knowing full well while giving someone a rating, everyone will be able to see it If someone sees Joey297 constantly neg rating What?, they can easily report that to the staff. Additionally, if someone sees said hypothetical member pos. rating every post of What?'s, they can also report that. There has been at least one case in the past where someone was actively overusing the 'Like' button (for a joke, however), and someone else did report it and it was easily resolved. I like allowing all members to be able to see and report such abuse in the event staff (be it everyone or those on at the time) do end up catching some and missing others.

And on the Matthew Effect, imo even if that is accounted for, there will still be other cognitive biases at work behind the scenes. There have been many times in the various different Skype, MSN, Facebook, and so on chats I've been in when there have been other people who share a link they find funny or infuriating, and people will flock to that link and check it out. When this has happened in the case of KHV posts, people have mass repped or mass derepped posts. These mass rep sprees were organized offsite, and it is an effect of deindividuation, of acting as the group around you does. What stops this bias from happening when ratings are private? Even with the Matthew Effect, it will still be in effect, just through a different medium. If enough, (specific) people around someone in a chat or in a call say they find it funny, that someone can still be effected, regardless of the way they received that information. I agree that trying to account for different biases is good, but in this case, I think regardless of whether they're private or not, they'll still be present in some fashion.

And at the risk of getting off topic, I disagree. I don't like forcing people into a decision for something they may honestly not care for. To rectify this, typically (and what I had decided to do a few days after my initial post of the User Awards), I just count the 'Don't Care' votes towards both options in the poll and go from there since it's clear they don't really mind either way, which is fine imo.

 

Public ratings... Okay. I can see what you mean. My reason for hiding them is primarily because I do not want people to feel pressured not to give negative ratings. Instead, I would prefer it if people could rate as they want with negative ratings counting for half.

On the flip side, you are hiding negative ratings. If you don't care about the Matthew effect and want people to face scrutiny for their ratings, why only hide the negative? People should be accountable for positive ratings but not negatives? I don't get it.

All of you points are good but the last one. I figure that if they don't care, they are not forced to vote.

Letting them say that they don't care is the same as voting to delegate the decision to others. In other words, if we have three options (Yes, No, and Don't Care), Don't Care is the same as 'Let the staff decide,' which is redundant to the point of the vote.

Do you see what I mean there?[DOUBLEPOST=1402092885][/DOUBLEPOST]I can answer one of your questions.

If I link someone to a post I rated while ratings are public, they will see that I rated it right away and be inclined to follow suit because they are following my lead. They will feel inclined to give an opinion on it in the same way I did. If they saw that I posted instead of a rating, they would be inclined to post as well. You only jump on the wagon if you think there is one.

If the rating I gave is not public, they will feel less inclined to rate it period because they won't have a reason to think I expect them to rate it at all, let alone like I did. If I did not post, they will be less likely to post in my stead. It's the same bias.

That's more of the Matthew effect at play.

 

You can still see the negative ratings you and other people give to each other. By being hidden, it's more that the amount of negative ratings you have won't reflect in your rating bar/number showing in the postbit:


taken before my previous template edits

The Rep Bar was similar in that it didn't show dereps, but rather just the total amount of positive rep you had, which is an idea that I wanted to keep with this. The bar also, imo, looked worse with all ratings reflected in the bar than only positive.

And yeah, I see what you mean too. I do fear that people may feel like they're forced to vote for something even if they don't care (or worse, be swayed by friends/etc.), but I suppose a note saying if they don't feel strongly they don't, nor shouldn't, feel pressured to vote could be good.

 

Aha, I like that... But how do you tell how much rep the person has from the bar? Is it always a solid green bar all the way across?

Please read and reply to my double post; it's relevant.

 

The only pressure to not give negative ratings is the awareness that other people can see it; one would not give a negative rating if they didn't want the community to know they gave negative ratings. I don't consider that form of transparency an issue. The idea is having public consequence for your actions; if people would think differently of you for doing something, and you don't want that, then maybe you should reconsider what you're about to do. If you don't care what others think, or if you feel the negative rating is still justified, then you can still do it. It's a system of self-regulation, which can work in tune with staff moderation. If anything, I find that fairly similar to the vBulletin reputation system, just an extrapolation of it -- your "reputation" within the community is decided upon how other people view you (the ratings you receive) and how you treat others (the ratings you give out).

The negative ratings are still visible in user profiles, just not the member card or message user info. Your "negative" reputation still exists and is visible, but you're not being branded all over the site.

If it bothers you so much then it is fairly easy to turn off the display of ratings on your end. I'd be happy to give users the option to within their user profile, like how you can toggle the back to top button, but I think that the benefits of public display (mentioned above for negative ratings, there are further advantages for the positive ratings) outweigh what you're mentioning, especially because the matthew effect tends to apply to opportunity and institution more than a social setting.

 

As I said before, cognitive bias affects us all. It's not about the person who gives the first positive or negative rating. It's about those who come after. When you rate a post that already has ratings, you unconsciously take those ratings into account when rating. You are no longer simply rating the content on its own merit. If I rate something first, I don't want my rating to influence the ratings others give.

I strongly believe the effect of this will cause many people to like, dislike, or not rate things based partially on the ratings they see already on the content. I believe this will happen extremely often, to the tune that 1/5 to 1/3 of all ratings will be influenced. Some will go with the flow completely.

By comparison, I do not expect people to think critically about the ratings they see. I can't see people actually thinking about ratings and calling each other out for them unless an extreme example arises like with someone liking everything or someone disliking everything a specific user posts. That kind of targeting was rare even when rep was private. If people don't call each other out, the potential bonus is meaningless. Even so, we still get people being affected by the ratings they see.

In summary, it's not me. I am one of the few who are aware of the bias and I still cannot control it completely. The vast majority don't think it exists. This is for them, too.[DOUBLEPOST=1402097054][/DOUBLEPOST]How do you think it will affect their decision when someone sees a post they agree with, goes to rate it, and sees a lot of dislikes on it? What about when someone is offended or dislikes a post, goes to rate it, and sees a lot of likes on it?

I think it will backfire on you.

 

Cognitive bias is an issue when applied to making rational decisions, not giving out small ratings on a user forum. I never disagreed that it would happen, but people's ratings will never be influenced by logic or the "quality" of the post -- quality is subjective. People will be influenced by bias, emotion, and a whole host of other things. I feel you are simply overthinking the system's purpose by attempting to turn into some sort of advanced system of responding to a post. It's not. It's a little value that does, to some degree, display your reputation within the community. On the first page we established that if you have thoughts on someone's post -- that is, you disagree with it -- you are encouraged to reply to it, whether publicly or privately, as that facilitates discussion. Any reputation system -- vB's version, post ratings, or even likes -- are meant to express approval or disapproval. Nothing more, nothing less. Making ratings secret would rob people of information and remove that element of self-regulation previously mentioned, even if some of people's ratings may be influenced by the phenomena you're describing.

It's worth mentioning as well that the bias you're describing is in no way exclusive to these ratings. What if I linked you to a post and told you I negatively repped it? What if I was in a group chat and we were all talking about how ****** a post is? What if I saw a bunch of people replying to a post and disagreeing? You are never going to eliminate cognitive bias in a social setting, as people are not objective & do not exist in a vacuum, nor will they ever.

I didn't say anything about "calling people out," but if I noticed that a particular user seemed to be giving out a lot of negative ratings, that would influence my perception of them.

 

I want people to vote their first impulse based on what they think of the post, not what others think of it. Cognitive bias adulterates personal opinions. It changes your subjective view of what is quality. These things build up over time, which is where the effect comes in.

I already gave a specific reason why private ratings would be preferable in the case of pointing someone to a post on the forum. It's hard to see the logic behind the argument, 'If it's going to happen anyway, might as well make it worse.'

We might end up using these ratings as requirements for various things in the future.

 

I think this would be a good compromise between private and public ratings. It gives everyone the coice to opt out whether they want to have them private or not, which in regards to this particular thing, I think would be more effective than forcing people one way or another.

There aren't any plans now, or will there ever be I think, to use Rep/Likes/Ratings or anything of the sort as a requirement for going-ons on the site. The only thing they are linked to is Materials for pins (which Llave is currently working on revamping).

 

Maybe not from the staff side. There has been a lot of pressure in the past to add a rep requirement for premium or a third tier group above premium, for example. Those were left to die despite getting a lot of effort and agreement put into them by both old members and new, presumably because we didn't have a rep system on XenForo.

To make a blanket statement like 'this is what rep is for and it should not be taken more seriously than that' implicitly discourages people from making suggestions like that. I don't think we should be making such blanket dismissals of things users care about.

That compromise is fine with me. Much better than nothing. I don't think I can do any more to sway you, so I will back out of this conversation (unless Misty pulls me in) until the system is reviewed after the trial. See you then.