Changes To Newschoolers Rating System
Jul 17th 2012Newschoolers is making some changes to the Rating system for News and Media. Read more for the details...
- Views: 12167
- Cmts: 83
Online
members online
Discussion about new upco - Site Discussion - Forums
Subject: Discussion about new upcoming rating system
There will be a full announcement when it launches, but I want to give you guys a touch of a heads up so you can start wrapping your mind around it.
We scoured the earth trying to figure out how to build a better rating system for Newschoolers. 1-10 was fundamentally broken, as very few people used it properly. Most people voted 9-10 for everything, and then those that wanted to harm other people's media just gave it a 1.
Sadly, there was no way to fix this system, which is based on everyone giving critical and fair ratings.
Our research led us to Reddit, which basically has the best ratings system in the game right now.
It relies on an up/down voting system, a 'score' which things get and a fancy ranking equation in the back-end. This importantly focuses more on the 'hive mind' aspect of things, so no one person has the power to affect the outcome so drastically. Do note that your 'score' will not mean you're necessarily ranked the best. The ranking equation is what decides how good you are, applied to a time scale.
What you have to get used to is that everything on NS that gets rated is about to be converted to an up/down rating system. So you can bump something up, you can bump it down or you can leave it alone with a 'meh'. Your score will be based on this, and your ranking will be based on the specific ordering equation.
Note that the only way to do this is to back-convert all previous ratings, which we have done in a testing environment. All current POD/VOD/etc. will stay as is, but everything else that is sorted by rating will be completely changed to be based off of the new system. I'm looking at all the media right now, and I think this ranking is going to be a huge amount better.
Sit tight though, as we're not going to launch it until its ready to go. Ideally that doesn't take much more than next week to get finished, but you never know.
Thanks for sticking with us, and please don't downrate in the intrim.
We scoured the earth trying to figure out how to build a better rating system for Newschoolers. 1-10 was fundamentally broken, as very few people used it properly. Most people voted 9-10 for everything, and then those that wanted to harm other people's media just gave it a 1.
Sadly, there was no way to fix this system, which is based on everyone giving critical and fair ratings.
Our research led us to Reddit, which basically has the best ratings system in the game right now.
It relies on an up/down voting system, a 'score' which things get and a fancy ranking equation in the back-end. This importantly focuses more on the 'hive mind' aspect of things, so no one person has the power to affect the outcome so drastically. Do note that your 'score' will not mean you're necessarily ranked the best. The ranking equation is what decides how good you are, applied to a time scale.
What you have to get used to is that everything on NS that gets rated is about to be converted to an up/down rating system. So you can bump something up, you can bump it down or you can leave it alone with a 'meh'. Your score will be based on this, and your ranking will be based on the specific ordering equation.
Note that the only way to do this is to back-convert all previous ratings, which we have done in a testing environment. All current POD/VOD/etc. will stay as is, but everything else that is sorted by rating will be completely changed to be based off of the new system. I'm looking at all the media right now, and I think this ranking is going to be a huge amount better.
Sit tight though, as we're not going to launch it until its ready to go. Ideally that doesn't take much more than next week to get finished, but you never know.
Thanks for sticking with us, and please don't downrate in the intrim.
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
Quoting Mr.Bishop
from Jul 18 2012 2:25:28:
The score that is displayed is the number of upvotes minus the number of downvotes. We liked the idea of one number, as it made finding current good stuff within the 'latest added' view really easy.
I understand that the score is the number of up votes minus the number of down votes. I also agree with your point in only displaying the score on the thumbnail versions in all the different categories a viewer can be in such as latest added, top rated, etc to keep the clutter down with one number to accurately display a videos quality. But on the actual video page, where the video can be voted up or down, I'd like to see the actual number of up votes (without the downs subtracted) and then the actual number of down votes similar to on youtube (except they don't do a score, only total likes and total dislikes). I find it annoying that I can see the red bar on the graph but can't find out the actual number of down votes. I made a sample version to help explain what I am getting at. from Jul 18 2012 2:25:28:The score that is displayed is the number of upvotes minus the number of downvotes. We liked the idea of one number, as it made finding current good stuff within the 'latest added' view really easy.
Youtube version:
;)
Current NS version:
;)
What I'd like to see happen:
;)
WisCompton.
Abstract Reality.
Abstract Reality.
^
+1 on that idea. I think it'd be beneficial and not that hard to add!
+1 on that idea. I think it'd be beneficial and not that hard to add!
-- Jamie
§ http://surfaceskis.com § http://el33tskier.tumblr.com §
--EASTERN SAMPLE----B.M.I.T.R.--
if i wanna see dudes and chicks doing drugs and each other, while tearing their lives apart from the inside, ill watch antiques roadshow
sammyj
☭
§ http://surfaceskis.com § http://el33tskier.tumblr.com §
--EASTERN SAMPLE----B.M.I.T.R.--
if i wanna see dudes and chicks doing drugs and each other, while tearing their lives apart from the inside, ill watch antiques roadshow
sammyj
☭
Quoting BcL$ from Jul 19 2012 12:12:51:
I understand that the score is the number of up votes minus the number of down votes. I also agree with your point in only displaying the score on the thumbnail versions in all the different categories a viewer can be in such as latest added, top rated, etc to keep the clutter down with one number to accurately display a videos quality. But on the actual video page, where the video can be voted up or down, I'd like to see the actual number of up votes (without the downs subtracted) and then the actual number of down votes similar to on youtube (except they don't do a score, only total likes and total dislikes). I find it annoying that I can see the red bar on the graph but can't find out the actual number of down votes. I made a sample version to help explain what I am getting at. Youtube version:
Current NS version:
What I'd like to see happen:
This was how it was originally built. We specifically removed this and changed it from likes/dislikes (and thumbs up and thumbs down) to Up/Down voting and score.
I understand that the score is the number of up votes minus the number of down votes. I also agree with your point in only displaying the score on the thumbnail versions in all the different categories a viewer can be in such as latest added, top rated, etc to keep the clutter down with one number to accurately display a videos quality. But on the actual video page, where the video can be voted up or down, I'd like to see the actual number of up votes (without the downs subtracted) and then the actual number of down votes similar to on youtube (except they don't do a score, only total likes and total dislikes). I find it annoying that I can see the red bar on the graph but can't find out the actual number of down votes. I made a sample version to help explain what I am getting at. Youtube version:
Current NS version:
What I'd like to see happen:
We debated this for quite some time. Frankly yes, we understand that YouTube uses the like/dislike system and its pretty decent. However, Reddit uses the up/down and score system, which also is fantastic.
The primary decision came from something that has been an issue on every single launch we've ever done, which is any changes to the 'latest added' views. Very specifically the Latest added video view.
http://www.newschoolers.com/videos/latest.0/Latest_Videos%20
This view is where our most vocal, loyal and involved video critiques hang out. They check everything that is new, and do it with a quick scroll scan of that page. Mid-winter, everything in there moves very quickly, and each time we've messed with anything in that view its been met with massive outrage.
We tested both views extensively. What we were looking for was an optimal way to scan that list as fast as is possible, while easily finding quality videos to watch in an instant. Quick scroll, minimal data and maximum goodness.
The Like/Dislike model put more data on the screen, and wasn't quite as easy to compute quickly as was the score model. Score gives you one super easy number to look at, which gives you the data you need.
We even went one step further with the bars, which the bar sizes dependant on the number of votes applied as well as displays red/green.
So from two angles we did what we could to give a quick scan of the new stuff and maximum speed of picking quality out of new. This view is basically only designed for people who troll the 'latest added' view, as the ranking algorithm and Video Of the Day is a much better way for your average person to look at the list.
I understand that one might want to know how many dislikes they've gotten, but I think that the Reddit-style system is much more intelligent and elegant. Really this isn't a like / dislike situation, this is a score that your video gets. We don't dislike things, we simply give them a down rating. That doesn't mean we don't like it, we just don't like it as much as we like something we upvoted.
Its classier, smarter, gives your brain quicker access to summarized data and overall just more efficient. So I vote that we stick with the up/down system, and move onto some more features of the site that need some attention.
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
Quoting Mr.Bishop from Jul 19 2012 3:19:38:
This was how it was originally built. We specifically removed this and changed it from likes/dislikes (and thumbs up and thumbs down) to Up/Down voting and score. We debated this for quite some time. Frankly yes, we understand that YouTube uses the like/dislike system and its pretty decent. However, Reddit uses the up/down and score system, which also is fantastic. The primary decision came from something that has been an issue on every single launch we've ever done, which is any changes to the 'latest added' views. Very specifically the Latest added video view. http://www.newschoolers.com/videos/latest.0/Latest_Videos%20 This view is where our most vocal, loyal and involved video critiques hang out. They check everything that is new, and do it with a quick scroll scan of that page. Mid-winter, everything in there moves very quickly, and each time we've messed with anything in that view its been met with massive outrage. We tested both views extensively. What we were looking for was an optimal way to scan that list as fast as is possible, while easily finding quality videos to watch in an instant. Quick scroll, minimal data and maximum goodness. The Like/Dislike model put more data on the screen, and wasn't quite as easy to compute quickly as was the score model. Score gives you one super easy number to look at, which gives you the data you need. We even went one step further with the bars, which the bar sizes dependant on the number of votes applied as well as displays red/green. So from two angles we did what we could to give a quick scan of the new stuff and maximum speed of picking quality out of new. This view is basically only designed for people who troll the 'latest added' view, as the ranking algorithm and Video Of the Day is a much better way for your average person to look at the list. I understand that one might want to know how many dislikes they've gotten, but I think that the Reddit-style system is much more intelligent and elegant. Really this isn't a like / dislike situation, this is a score that your video gets. We don't dislike things, we simply give them a down rating. That doesn't mean we don't like it, we just don't like it as much as we like something we upvoted. Its classier, smarter, gives your brain quicker access to summarized data and overall just more efficient. So I vote that we stick with the up/down system, and move onto some more features of the site that need some attention.
from Jul 19 2012 3:19:38:This was how it was originally built. We specifically removed this and changed it from likes/dislikes (and thumbs up and thumbs down) to Up/Down voting and score. We debated this for quite some time. Frankly yes, we understand that YouTube uses the like/dislike system and its pretty decent. However, Reddit uses the up/down and score system, which also is fantastic. The primary decision came from something that has been an issue on every single launch we've ever done, which is any changes to the 'latest added' views. Very specifically the Latest added video view. http://www.newschoolers.com/videos/latest.0/Latest_Videos%20 This view is where our most vocal, loyal and involved video critiques hang out. They check everything that is new, and do it with a quick scroll scan of that page. Mid-winter, everything in there moves very quickly, and each time we've messed with anything in that view its been met with massive outrage. We tested both views extensively. What we were looking for was an optimal way to scan that list as fast as is possible, while easily finding quality videos to watch in an instant. Quick scroll, minimal data and maximum goodness. The Like/Dislike model put more data on the screen, and wasn't quite as easy to compute quickly as was the score model. Score gives you one super easy number to look at, which gives you the data you need. We even went one step further with the bars, which the bar sizes dependant on the number of votes applied as well as displays red/green. So from two angles we did what we could to give a quick scan of the new stuff and maximum speed of picking quality out of new. This view is basically only designed for people who troll the 'latest added' view, as the ranking algorithm and Video Of the Day is a much better way for your average person to look at the list. I understand that one might want to know how many dislikes they've gotten, but I think that the Reddit-style system is much more intelligent and elegant. Really this isn't a like / dislike situation, this is a score that your video gets. We don't dislike things, we simply give them a down rating. That doesn't mean we don't like it, we just don't like it as much as we like something we upvoted. Its classier, smarter, gives your brain quicker access to summarized data and overall just more efficient. So I vote that we stick with the up/down system, and move onto some more features of the site that need some attention.
I don't disagree with showing only the score on the recently added, but why not show the amount of down votes ONLY on the page where the video is embedded? The graph seems somewhat pointless unless you are going to show the numbers that created it. A general idea of the numbers that made it can be guessed by the total score, but still I'd like to see the actual numbers behind each video. I don't think the picture I posted of he modified graph scores makes the video page anymore cluttered (i mean c'mon, it's a total of 6 letters and 2 numbers in an empty space below the graph) or hard to understand and I'm sure many people would like to see the number of down votes (as others have already stated in this thread).
WisCompton.
Abstract Reality.
Abstract Reality.
One thing about this new system is the fact that I was just in the newly uploaded videos and this was rated -1. So many people aren't going to click on it now because it is a negative rating right off the bat. and it does not deserve a -1 at all. I mean, not the best clip ever uploaded to newschoolers, but for real straight up disrespectful for a -1.
WisCompton.
Abstract Reality.
Abstract Reality.
Quoting BcL$ from Jul 19 2012 3:40:19:
I don't disagree with showing only the score on the recently added, but why not show the amount of down votes ONLY on the page where the video is embedded? The graph seems somewhat pointless unless you are going to show the numbers that created it. A general idea of the numbers that made it can be guessed by the total score, but still I'd like to see the actual numbers behind each video. I don't think the picture I posted of he modified graph scores makes the video page anymore cluttered (i mean c'mon, it's a total of 6 letters and 2 numbers in an empty space below the graph) or hard to understand and I'm sure many people would like to see the number of down votes (as others have already stated in this thread).
I don't like the up and down being displayed, I find it puts too much of an emphasis on the negative aspect of voting. Simply having a score only focuses on one thing.
I don't disagree with showing only the score on the recently added, but why not show the amount of down votes ONLY on the page where the video is embedded? The graph seems somewhat pointless unless you are going to show the numbers that created it. A general idea of the numbers that made it can be guessed by the total score, but still I'd like to see the actual numbers behind each video. I don't think the picture I posted of he modified graph scores makes the video page anymore cluttered (i mean c'mon, it's a total of 6 letters and 2 numbers in an empty space below the graph) or hard to understand and I'm sure many people would like to see the number of down votes (as others have already stated in this thread).
It also does not even matter in terms of your ranking. That is the key... ranking by up and down is meaningless. Its your wilson score that really matters.
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
Quoting BcL$ from Jul 19 2012 4:53:13:
One thing about this new system is the fact that I was just in the newly uploaded videos and this was rated -1. So many people aren't going to click on it now because it is a negative rating right off the bat. and it does not deserve a -1 at all. I mean, not the best clip ever uploaded to newschoolers, but for real straight up disrespectful for a -1.
You uploaded it twice, and I deleted one of them (obviously the wrong one).
One thing about this new system is the fact that I was just in the newly uploaded videos and this was rated -1. So many people aren't going to click on it now because it is a negative rating right off the bat. and it does not deserve a -1 at all. I mean, not the best clip ever uploaded to newschoolers, but for real straight up disrespectful for a -1.
However, your video had already gotten a few votes and made it up to a 2.
You can't judge on what happens after a few minutes. Things will not disappear if they deserve to move upwards. One downvote is not the end of the world.
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
Quoting Mr.Bishop from Jul 19 2012 5:29:31:
I don't like the up and down being displayed, I find it puts too much of an emphasis on the negative aspect of voting. Simply having a score only focuses on one thing. It also does not even matter in terms of your ranking. That is the key... ranking by up and down is meaningless. Its your wilson score that really matters.
meh, we aren't going to agree on this haha. from Jul 19 2012 5:29:31:I don't like the up and down being displayed, I find it puts too much of an emphasis on the negative aspect of voting. Simply having a score only focuses on one thing. It also does not even matter in terms of your ranking. That is the key... ranking by up and down is meaningless. Its your wilson score that really matters.
And that wasn't my video, just a random one I found in the latest added. I wasn't aware it was posted twice so maybe it was the repeat and that is why it was down voted.
WisCompton.
Abstract Reality.
Abstract Reality.
Quoting BcL$ from Jul 19 2012 8:41:28:
meh, we aren't going to agree on this haha. And that wasn't my video, just a random one I found in the latest added. I wasn't aware it was posted twice so maybe it was the repeat and that is why it was down voted.
Just do me this favor -
meh, we aren't going to agree on this haha. And that wasn't my video, just a random one I found in the latest added. I wasn't aware it was posted twice so maybe it was the repeat and that is why it was down voted.
Please really look at the new system and give it a chance. I've been researching this topic for more than a year now, and have looked at every aspect in unfathomable detail.
At first I was really about the like/dislike, but the more I researched, tested and experienced the more my mind changed in favor of up/down and score. Open your mind to it, and I think over time it will sink in.
Also, use youtube/facebook less and Reddit more. :)
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
Quoting *kF_pauLo-$$ from Jul 18 2012 8:16:44:
No hate, but...is this more important than the search bar ? Cheers
I forgot to thank the staff for implementing the new rating system though !No hate, but...is this more important than the search bar ? Cheers
Always nice to see new features.
French in Oslo
am i the only one who's never really had that much trouble about the searchbar?
this rating system is pretty good from what I've experienced, which honestly hasn't been much so far. I'll have to dive into the videos/pictures soon and go wild with it.
I like it though...it's a like a uniquely NS twist on something a lot of other sites use.
You guys don't seem to make major changes to the site often, but all in all, when you do, it just about always turns out to be an awesome improvement. So props on being probably the only site i use that is capable of advancing like you do.
this rating system is pretty good from what I've experienced, which honestly hasn't been much so far. I'll have to dive into the videos/pictures soon and go wild with it.
I like it though...it's a like a uniquely NS twist on something a lot of other sites use.
You guys don't seem to make major changes to the site often, but all in all, when you do, it just about always turns out to be an awesome improvement. So props on being probably the only site i use that is capable of advancing like you do.
Proud owner of an original NS lanyard.
Quoting Mr.Bishop from Jul 20 2012 9:06:42:
Just do me this favor - Please really look at the new system and give it a chance. I've been researching this topic for more than a year now, and have looked at every aspect in unfathomable detail. At first I was really about the like/dislike, but the more I researched, tested and experienced the more my mind changed in favor of up/down and score. Open your mind to it, and I think over time it will sink in. Also, use youtube/facebook less and Reddit more. :)
i think you guys have done a great job on this and should keep it that way aslong as it works well from Jul 20 2012 9:06:42:Just do me this favor - Please really look at the new system and give it a chance. I've been researching this topic for more than a year now, and have looked at every aspect in unfathomable detail. At first I was really about the like/dislike, but the more I researched, tested and experienced the more my mind changed in favor of up/down and score. Open your mind to it, and I think over time it will sink in. Also, use youtube/facebook less and Reddit more. :)
we call front swap front 2s ellen d's cause they involve a lot of scissoring haha - .squirrely
I like it more
RIP Jack Wade
The only hip-hop singa to tell America to kiss his killa bee stinga
LT4
The only hip-hop singa to tell America to kiss his killa bee stinga
LT4
Quoting Mr.Bishop from Jul 20 2012 9:06:42:
Just do me this favor - Please really look at the new system and give it a chance. I've been researching this topic for more than a year now, and have looked at every aspect in unfathomable detail. At first I was really about the like/dislike, but the more I researched, tested and experienced the more my mind changed in favor of up/down and score. Open your mind to it, and I think over time it will sink in. Also, use youtube/facebook less and Reddit more. :)
Word. Right now it seems funky to me but I'm sure I'll learn.
from Jul 20 2012 9:06:42:Just do me this favor - Please really look at the new system and give it a chance. I've been researching this topic for more than a year now, and have looked at every aspect in unfathomable detail. At first I was really about the like/dislike, but the more I researched, tested and experienced the more my mind changed in favor of up/down and score. Open your mind to it, and I think over time it will sink in. Also, use youtube/facebook less and Reddit more. :)
Much respect, Bishop, you know your shit.
OP: "If you suddenly skied off a huge fuckin cliff and you only had time to throw one last trick before you died, what would it be?"
Malcolm: "id probably just flail my self to death"
"If the overall quality of your film depends on what you shot it on, you aren't ready to make a film." - Barry Green
http://www.vimeo.com/milosatz
:)
Malcolm: "id probably just flail my self to death"
"If the overall quality of your film depends on what you shot it on, you aren't ready to make a film." - Barry Green
http://www.vimeo.com/milosatz
:)
Quoting blondie.$ from Jul 21 2012 1:58:48:
Word. Right now it seems funky to me but I'm sure I'll learn. Much respect, Bishop, you know your shit.
The deal I'll make you is that if you use it for a couple of months and still feel the same way, I'll re-assess and re-open for voting whether people want it this way or like/dislike.
Word. Right now it seems funky to me but I'm sure I'll learn. Much respect, Bishop, you know your shit.
Give it a bit though, I really do think this will feel natural after some time.
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
I'd also love to point out that the recent rankings of photos have been damn good. Its a far cry from where they were even a week ago.
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
the karma system for members is pretty much the same thing as the old rating system for videos/pics. What if the new up/down system was implemented for member karma? The problem you talked about was that people usually always rated 10 or 1 on videos and that is the same issue that comes up on all the karma threads. the what others rate you could still be on the profile with a graph except instead of it being the average rating out of 10, it would be the average of up/down. or it could be the bar graph like the new system.
WisCompton.
Abstract Reality.
Abstract Reality.
I like this rating system!
Purplejumpsuit Apparel : http://purplejumpsuit.wordpress.com/
for all your dope hoodie needs!^
for all your dope hoodie needs!^
Quoting BcL$ from Jul 27 2012 12:26:06:
the karma system for members is pretty much the same thing as the old rating system for videos/pics. What if the new up/down system was implemented for member karma? The problem you talked about was that people usually always rated 10 or 1 on videos and that is the same issue that comes up on all the karma threads. the what others rate you could still be on the profile with a graph except instead of it being the average rating out of 10, it would be the average of up/down. or it could be the bar graph like the new system.
Karma takes into account WAY more than just the 1-10 member ratings, but you are very correct that there is a flaw in the 1-10 system no matter where it is.
the karma system for members is pretty much the same thing as the old rating system for videos/pics. What if the new up/down system was implemented for member karma? The problem you talked about was that people usually always rated 10 or 1 on videos and that is the same issue that comes up on all the karma threads. the what others rate you could still be on the profile with a graph except instead of it being the average rating out of 10, it would be the average of up/down. or it could be the bar graph like the new system.
We actually already have some plans sketched out for some upgrades to the Karma system, but we're not prepared to announce them yet. As always, karma is a highly guarded secret here at the office.
The Up/Down lesson though is definitely what we're basing all of our new ratings logic off of though. The success is absolutely insane.
newschool is a state of mind, not a style of skiing.
-hendrik. '05
-hendrik. '05
Hopefully this was not answered yet. Does the current or future karma system show you who has given you karma?
I'm really loving this.
"Does Crichton smoke? Does a bear shit in the woods?" -Rex
"A debate between two newschoolers is very much like a wizard's duel, except they're not wizards and magic isn't real. So what remains are two socially awkward dudes yelling jibberjabber towards each other." -I_Am_Mod
"So what do you say? Wanna put your IP where your mouth is?" -Huck_Norris
The Undeniable Blog
http://newschoolers.com/ns/content/readblog/member_id/18689/
"A debate between two newschoolers is very much like a wizard's duel, except they're not wizards and magic isn't real. So what remains are two socially awkward dudes yelling jibberjabber towards each other." -I_Am_Mod
"So what do you say? Wanna put your IP where your mouth is?" -Huck_Norris
The Undeniable Blog
http://newschoolers.com/ns/content/readblog/member_id/18689/
All times are EST (-5); The current time is 7:36AM
<< Back to thread listing - Previous Page
Related News Articles
Judging?
Mar 20th 2012Competition means discussion in every sport and here on NS members like nothing more than to discuss results. In this blog post I shall discuss ways to make the judging system dare I say 'better'?
- Views: 856
- Cmts: 0
Getting closer!
Oct 15th 2010Snowmaking system is 100% ready to go to get us shredding again for the upcoming season. Park crew, Stu and Julian, have been busy putting the system through it’s preseason tests the last two weeks. Today every machine on Castle Peak and Nugget passed their final pressure tests with flying colors. We are set up [...][IMG]http://media.nscdn.com/uploads/images/1/00/01/34/78/13478.png[/IMG]
- Views: 413
- Cmts: 0
The Bare Minimum
Aug 3rd 2012A hilarious and nicknamed-filled discussion with Good Enough's Jason Arens, Noah Curry and Garrett Jurach on the mantra behind their webisode series and their upcoming short film, The Bare Minimum.
- Views: 7546
- Cmts: 22
