Changes To Newschoolers Rating System

Today Newschoolers has made some changes to how members rate News and Media. From now on, the old 10-star rating tool is no more, but instead simply Up and Down votes...


Our old rating system used a rating scale of 1-10, and although at a glance it seems to work very well, it had several problems.

The biggest problem we faced was how to order items by their average rating, which was nearly impossible to do well. The reason for this was that an average rating does not take into account how many people voted. So for example, a picture with a single "10" rating would be rated higher than a picture averaging "9.99/10" and rated 500 times. As a human, you could tell that the latter should obviously be rated higher, but this is a difficult problem to solve with computers. There are ways to mitigate this problem, but no way to properly solve it. A perfect example of this could be found in any section of the site. For example, the screenshot below was taken from the Member Video gallery...


The second major problem was that it's very difficult to objectively rate items properly in a 1-10 scale. What we found was that most members would rate something a 9 or 10 if they liked it, or a 1 if they didn't. Members were essentially rating things Up or Down, just like on YouTube or Reddit. Check out the graph below to see the breakdown of ratings given on Newschoolers, the division between "Like" and "Don't Like" ratings is pretty obvious.


Our new rating system solves both of these problems, and it should already feel familiar. We've ditched the 1-10 scale in favor of Up and Down buttons, which more closely reflects how members were already rating.

The real magic though comes from how the results can be used for generating a relevant rating. The new rating system accounts for the number of ratings when calculating an overall ranking order. So an item that has 70% positive ratings from 20 ratings would be rated higher than something with 100% positive ratings with only two ratings. The reason for this is that the former has a higher "confidence" of being an accurate score due to its higher number of samples. (Math nerd-out located here.)

So now lets look at the original screenshot of the 5 videos above. Assuming any rating between 6-10 is an Up vote, and anything between 0-5 is a Down vote, the new order and score would be...


It's clear that the new order makes far more sense, and we're excited to finally see this change to go live!

And in addition to all of that, you are now also able to change your vote after rating something, so you can change your mind at any time.

Finally, please note that there is a 'score' associated with each media item. This score does not affect sorting order. This is an important thing to realize, as the publicly displayed score is the up votes minus the down. This score is designed to catch your eye when looking at media in the 'Latest Added' view, where we have also added bars that show you both the scale of number of votes as well as green and red for up and down.

This will be a drastically better way of rating media, and we are confident that its going to bring a whole new level of exposure to the talent that is bursting at the seams of Newschoolers. All of our featured media algorithms have been updated to use the new system, and will do a drastically better job at getting what is the best stuff on the site.

So dig up some of those shots you got this winter and give 'em a home to breathe. And for those of you who are just looking, get in there and do some voting!