Curious finding in charts
Printed From: Progarchives.com
Category: Site News, Newbies, Help and Improvements
Forum Name: Report bugs here
Forum Description: Help us improve the site from a tech standpoint
URL: http://www.progarchives.com/forum/forum_posts.asp?TID=85108
Printed Date: July 19 2025 at 09:36
Software Version: Web Wiz Forums 11.01 - http://www.webwizforums.com
Topic: Curious finding in charts
Posted By: Marty McFly
Subject: Curious finding in charts
Date Posted: February 22 2012 at 08:32
|
I was looking through Neo Prog top DVD's charts and found many errors (release that is higher than it should be from its ratings). Among the most weird ones are these three depicted on the screenshot: EDIT: Probably needless to say, but I'll say it anyway - 45 should be 46, 46 should be 47 and 47 should be 45. All albums have reviews. And why I think it should be like I am saying ? Well, that's my understanding of how charts should work - the higher rating AND the more ratings, the higher the entry should be. Some complex equation of these two ... maybe number of reviews is accounted too, maybe just having a review (with its weighted ratio) is enough, I don't know. (Full) Link to this list is here: http://www.progarchives.com/top-prog-albums.asp?ssubgenres=18&salbumtypes=2&syears=&scountries=&sminratings=1&smaxratings=0&sminavgratings=1&smaxresults=100&x=73&y=6#list But this is just a one problem of many I am encountering every time I use charts. Here, you have basic link from main page: http://www.progarchives.com/top-prog-albums.asp?salbumtypes=1#list OK ? We all know that ... Well, when I trim the genres to just Symphonic and Neo Prog (yeah, with CTRL), it will show me Symph and Neo - BUT then the weird thing happens. It will automatically select Tech/Extreme Prog Metal, Eclectic, Experimental and Heavy Prog from some reason I can oblivious to. I just don't get it ... this bug is here as long as I know, maybe 3 years at least ? Is there something that can be done about that, so that charts would work like they should ? ------------- There's a point where "avant-garde" and "experimental" becomes "terrible" and "pointless," -Andyman1125 on Lulu  Even my |
Replies:
Posted By: Dean
Date Posted: February 22 2012 at 11:14
|
Chart position and Average value are calculated using two different algorithms -
/edit: correction the collab/review weighting is used for both. ------------- What? |
Posted By: Marty McFly
Date Posted: February 22 2012 at 12:08
|
Well, Pendragon album has 4 reviews in total (3 of them collab - 4-4-3 stars), while Fish and Haze have just 1 (4 and 3 stars respectively). Not sure if it changes anything. ------------- There's a point where "avant-garde" and "experimental" becomes "terrible" and "pointless," -Andyman1125 on Lulu Even my |
Posted By: Dean
Date Posted: February 22 2012 at 13:13
|
Underneath the Average rating, which is the value calculated using Collaborator and Review weighting you will see the QWR average ... or Query Weighted Rating, this is used to calculate chart position and uses an algorithm like this:
Which when you pick any two DVDs you can easily solve to find the avg_num_votes and avg_rating score for this site, which I have and they are:
avg_num_votes = 15.9779
avg_rating = 3.9979
What this means is that any disc that has a rating higher than the site average rating will be weighted down, and any disc that has a rating lower will be weighted up. Similarly, any disc that has fewer ratings than the site average number will be weighted up and those with more will be weighted down.
Weighted averages are prone to producing counter-intuitive results (such as these here)... as you said yourself in the OP ... "the more ratings, the higher the entry should be" - that's the intuitive result and that would be a false result, the weighted algorithm corrects that "popularity" skew.
/edit: collab/review weighting average is used in this formula, not the arithmetic average
------------- What? |
Posted By: TheGazzardian
Date Posted: February 22 2012 at 13:28
| Hey Marty, the number used to calculate the position, as Dean said, is the weighted average. You can see that number next to each album, under the average and number of ratings, where it says QWR =. And by those numbers, the list you showed is in the correct order. |
Posted By: Marty McFly
Date Posted: February 22 2012 at 13:38
|
OK, this explained a lot to me, I completely (unfortunately) forgot about QWR. So even this explain a lot (little rated ones getting boosted up and high rated ones being beaten down, which is a case of Steven Wilson's new album as we can see + [correct me if I am wrong] high ratings are disadvantage and low ratings can work as advantage to an extent, or this switched logic works only on number of ratings, not on its values?) ... but it's not just against my intuition, it just doesn't make sense. Well, obviously it does, because there are equations that calculates it. But what I mean is simply that 46 shouldn't be higher than 47. It has far less ratings, less reviews, lesser rating (by almost whole point, which in our 4 point system is a lot). If it's not a bug, then it is not right. Why I think so ? My logic is simple - as it is now, having such album higher than my Pendragon example doesn't help anyone. I don't know any of these three DVD's in question, but logic, intuiting and common sense all tells me that it should be differently. ------------- There's a point where "avant-garde" and "experimental" becomes "terrible" and "pointless," -Andyman1125 on Lulu Even my |
Posted By: Triceratopsoil
Date Posted: February 22 2012 at 14:05
| I think that's why the top lists, by default, filter out releases with fewer than X ratings |
Posted By: Dean
Date Posted: February 22 2012 at 14:14
|
What the QWR is calculating is not the average score of the album from all the votes it received, but the average score of the album relative to all the votes for all albums.
for example - if one album has an average of 4.5 another has 4.2 then the mean of the two would be 4.35, which would be fine if both albums received the same number of votes - however if the first had 5 votes and the second 10 votes then the real average would have been (4.5*5+4.2*10)/15 = 4.3 and the average number of votes cast per album would be 7.5.
So what weighted averages do is calculate the average score for an album in relation to that 4.3 score and that 7.5 average number of votes, so when we get another album that has an average of 5 over 1 vote we can plug it into the forumula and calculate it's relative position - ie ( (7.5 * 4.3) + (1 * 5) ) / (7.5 + 1) = 4.38, rather than "5" and would slot this new album inbetween the two rather than it becoming the top album.
------------- What? |
Posted By: Dean
Date Posted: February 22 2012 at 14:53
|
CORRECTION Sorry Marty, I've just calculated the numbers for these three albums and I've discovered I have told you a big fat fib. The collaborators weightings are used to calculate the chart position with the weighted average formula. So the reason why Pendragon has a lower position is the lower percentage of collaborator/review ratings (11.8%) compared to Fish (12.5%) and Haze (100%).
I'm going to have to think about whether that is "the right thing to do" or not... ------------- What? |