missing Albums on top 2014 list?! 
Post Reply  Page <12 
Author  
Mind_Drive
Forum Senior Member Joined: September 06 2009 Location: Germany Status: Offline Points: 374 
Posted: May 18 2014 at 17:16  
hey Dean,
thanks for reading through all my complainments the QWR is in the right order but it doesnīt seem to be correct in every case. letīs compare: PFMs per un amico and anglagards hybris in the overall toplist: PFMs album has 4.42 and 1041 ratings and 96 reviews ranking at nr 14 Anglagards album has 4.42 and 1144 ratings and 130 reviews ranking at nr 15 although i donīt have studied the rating system to a degree, that i could be sure this is not correct  my feeling says, that they should rank the other way round, you donīt think so? i mean it would only make sense to me if the weight of reviews would have an impact on the QWR and PFMs albm had significantly more collaborator reviews. but i think the reviews weight is only used to calculate the average rating and from then its only the average rating and the number of ratings in relation to the total number of ratings (and maybe overall average rating) to calculate the QWR. in the higher areas of the toplists there seem to be only slightly wrong placed albums but if you look into the unpopular parts of this list e.g: http://www.progarchives.com/topprogalbums.asp?ssubgenres=&salbumtypes=1&syears=2014&scountries=&sminratings=1&smaxratings=0&sminavgratings=1&smaxresults=250&x=67&y=3#list you will encounter lots of questionable orders of albums (the QWR ofc here is coherent, too)... 152: Lascailleīs Shroud 3.04  4 ratings 153: Not Overwise Specified 3.29  7 ratings 154: DaccorD 3.58  31 ratings i hope you can get my point and i donīt bring too much confusion to you right now Edited by Mind_Drive  May 18 2014 at 17:31 

It's just a ride... <3


Sponsored Links  
HolyMoly
Special Collaborator Retired Admin Joined: April 01 2009 Location: Atlanta Status: Offline Points: 25951 
Posted: May 18 2014 at 18:07  
I wish I had time right now to delve into this, because I love maths. But on the surface, I don't think it necessarily follows that two albums with an identical average (e.g. PFM and Anglagard example above) should be ranked by which one has more reviews. Perhaps the distribution of ratings (and weightings) has some bearing on it  the "standard deviation" concept used often in statistics. I don't know though, so I'll shut up for now.


My other avatar is a Porsche
It is easier for a camel to pass through the eye of a needle if it is lightly greased. Kehlog Albran 

Dean
Special Collaborator Retired Admin and Amateur Layabout Joined: May 13 2007 Location: Europe Status: Offline Points: 37575 
Posted: May 18 2014 at 19:42  
The only confusion is you said you understood QWR but kept saying there was a bug. The QWR formula is: QWR = (NR + nr) / (N + n) where QWR is the weighted rating, R is the average rating of all albums, N is the average number of votes per album for all albums, r is the album average and n is the number of ratings. The effect of calculating the QWR is twofold:
So, in simpler terms, albums with very few ratings/votes will have a rating weighted towards the average across all albums, while albums with many ratings/votes will have a rating weighted towards their own average rating. Therefore once the number of ratings for an album gets really big (ie n is much larger than N) then the actual value of the average rating r will have more effect on the formula than the chart average rating R and the QWR value will approach the actual average rating value r. QWR is a prediction, it is a prediction of what the average ratings would be if all the albums had similar numbers of ratings so it allows us to compare an album with 2 ratings with one that has 2000 ratings, therefore it is more related to probability.
No, not really. The numbers displayed are not accurate enough to make that kind of judgement call. The Average rating value and the QWR value for each album is calculated to more decimal places (dp) accuracy than what is displayed on the chart. So while both are displayed with an average of 4.42 (to 2 dp), their actual average ratings will be different  one could be 4.420001 and the other 4.4249999 for example. [example 2: The Abscinded Universe has an unweighted average of 3.25 and a weighted average of 3.04, but if you compute the weighted value to its fullprecision it is 3.04347826086957] To explain that further, consider the twofold effect of QWR (above) In both these cases the number of ratings n (1041 and 1144) is significantly larger than N, which I guess is somewhere around the 20 mark. This is effect (1): because n >> N the QWR values are very close to the Average rating values r for these albums. This suggests that if we could see the averages to more decimal places, we would see that Per Un Amico has a slightly higher average than Hybris. For example, assuming that N =17 and R=4.2 (I'm not saying they are, this is just an example), if we backcalculate the values of r from the displayed values of QWR for each album we get:
Now these examples are the result of effect (2) of the QWR formula because the number of ratings for each is very small (ie n < N) so the QWR values approach the value R and this value now has more effect on the result. As I said earlier, the site average for the number of ratings per album is around 20, so while 7 ratings is small and does approach zero, it is still quite close to N, (more so than the 1041 ratings for Per Un Amico for example). This means that we are close to the situation where R and r have equal effect (as would be the case if n=N). While D'AccorD III is > N, Projective Instruments and The Abscinded Universe are < N. This means that for D'AccorD III r has more effect on the QWR and for Projective Instruments and The Abscinded Universe R has more effect. You can think of this as the value of N acting like a fulcrum on a seesaw: values above N move down and values below N move up, which leads to a counterintuitive result. (ie it looks wrong to you). This is why the main charts do not show albums that have n < N. The problem with any statistical analysis of album ratings on the PA is that a relatively small population that rate any one album. This means that very slight changes in the data can have a dramatic (and subsequently meaningless) effect on the result. The charts are just for fun and should not be taken seriously. Edited by Dean  May 18 2014 at 20:02 

What?


Guldbamsen
Special Collaborator Retired Admin Joined: January 22 2009 Location: Magic Theatre Status: Offline Points: 23098 
Posted: May 19 2014 at 06:22  
Dean to the rescue!
Thanks. Like I said, to me this is mostly gibberish. 

The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.
 Douglas Adams 

Mind_Drive
Forum Senior Member Joined: September 06 2009 Location: Germany Status: Offline Points: 374 
Posted: May 19 2014 at 07:14  
thank you Dean!! you restored my sanity and my trust in the archives
i was not aware of this formular and you explained it super nice! especially the following sentences where strinking: "QWR is a prediction, it is a prediction of what the average ratings would be if all the albums had similar numbers of ratings so it allows us to compare an album with 2 ratings with one that has 2000 ratings, therefore it is more related to probability." "...which leads to a counterintuitive result." itīs indeed counterintuitive but now i got it thanks again for your efford! prog on! 

It's just a ride... <3


Post Reply  Page <12 
Forum Jump  Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum 