Forum Home Forum Home > Site News, Newbies, Help and Improvements > Report bugs here
  New Posts New Posts RSS Feed - missing Albums on top 2014 list?!
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Topic Closedmissing Albums on top 2014 list?!

 Post Reply Post Reply Page  12>
Author
Message
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Topic: missing Albums on top 2014 list?!
    Posted: May 16 2014 at 02:08
hey guys im not sure if this was asked before but

i love to check out the top albums of 2014 section and i wondered why there are still albums not listed there, although they have way more than 20 ratings. for example panic rooms incarnate (31 ratings) and i came across another one from this year with over 60 ratings so far (cant remember it right now).

whats the reason? are the albums added manually to the toplist? i dont know if its much of a problem but as i said i like to check out the best albums so far and with some missing it feels not 100% correct ;)

thanks!
It's just a ride... <3
Back to Top
chopper View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: July 13 2005
Location: Essex, UK
Status: Offline
Points: 19942
Direct Link To This Post Posted: May 16 2014 at 04:09
I'm no expert and you probably need someone better qualified to comment, but the rating for Incarnate is 3.69. The rating for album number 39 in the list at the moment is 3.86. Although the list is not just in rating order (presumably because of an algorithm), all the ratings in the list are higher than 3.69.
Back to Top
someone_else View Drop Down
Forum Senior Member
Forum Senior Member
Avatar
VIP Member

Joined: May 02 2008
Location: Going Bananas
Status: Offline
Points: 23972
Direct Link To This Post Posted: May 16 2014 at 05:31
The algorithm uses Query Weighted Ratings for the calculation of the ranking, which is based on the ratings, as is explained on top of the page.

To obtain the most complete list of 2014 studio albums (currently 171), this link can be used:

Only albums without ratings or reviews should be missed on this list.
Back to Top
chopper View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: July 13 2005
Location: Essex, UK
Status: Offline
Points: 19942
Direct Link To This Post Posted: May 16 2014 at 05:32
Originally posted by someone_else someone_else wrote:

The algorithm uses Query Weighted Ratings for the calculation of the ranking, which is based on the ratings, as is explained on top of the page.

To obtain the most complete list of 2014 studio albums (currently 171), this link can be used:

Only albums without ratings or reviews should be missed on this list.

Clap
And this puts Incarnate at number 131.
Back to Top
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Posted: May 16 2014 at 09:59
oh wow Nr 39 you said? if i check the top albums of 2014 i go to top albums of 2013 and then change the filter of year to 2014 that gives me a list only 17 albums oO

someone_else you shattered my worldview!
nice and i got it! :D just have to change minimum number of ratings and minimum average rating value! how awesome xD

thank you very much!!
It's just a ride... <3
Back to Top
someone_else View Drop Down
Forum Senior Member
Forum Senior Member
Avatar
VIP Member

Joined: May 02 2008
Location: Going Bananas
Status: Offline
Points: 23972
Direct Link To This Post Posted: May 16 2014 at 10:07
^You're welcome .
Back to Top
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Posted: May 16 2014 at 12:11
sorry that im bothering you again but

lets look at the example of panic rooms incarnate (3.69 - 31 ratings) nr 133 on this list:
http://www.progarchives.com/top-prog-albums.asp?ssubgenres=&syears=2014&scountries=&sminratings=1&smaxratings=0&sminavgratings=1&smaxresults=250&x=78&y=8#list


so why is it, that Syndone´s Odysseás is nr 132 with less ratings (22) and less average points (3.68) ?
nr 134 is btw another album wich has 1 rating and 2 stars oO i just.. i am confused
is there something totally screwed up in the later parts of the toplists?

or take for example 2013:
nr149: 3,35 - 36 ratings
nr150: 3,46 - 432 ratings.... what?
http://www.progarchives.com/top-prog-albums.asp?ssubgenres=&salbumtypes=1&syears=2013&scountries=&sminratings=0&smaxratings=0&sminavgratings=1&smaxresults=250&x=74&y=8#list

do i just dont get the weighted mean? (weight of review/collaborators review should have been already taken into account for the average rating, no?)
help!^^


Edited by Mind_Drive - May 16 2014 at 16:09
It's just a ride... <3
Back to Top
someone_else View Drop Down
Forum Senior Member
Forum Senior Member
Avatar
VIP Member

Joined: May 02 2008
Location: Going Bananas
Status: Offline
Points: 23972
Direct Link To This Post Posted: May 17 2014 at 01:56
A rating without a review counts for 1.
A rating with a review by a non-collab PA member counts for 10.
A rating with a review by a PA collaborator counts for 20.

For example, an album with 10 reviews, of which 4 are written by collabs, and 40 ratings without a review, puts in more weight than an album with 2 reviews and 132 ratings.

Then there is the Query Weighted Ratings system. There is a link to a Wiki entry about this on top of the ranking page. I have to study it sometime because I don't know the ins and outs of it yet. But the Query Weighted Rating may differ depending on the ratings of the albums participating in the list.
Back to Top
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Posted: May 17 2014 at 04:05
yes, i know that.
but the weighted rating system applies before the qwr. e.g: 4.09 | 2 ratings | 1 reviews | 50% 5 stars

only then the QWR is calculated wich depends on average ratings, number of ratings compared to all the other albums in the list - the weight of reviews doesn´t play a role in this calculation anymore because it was already used to get to the average rating.

please look at the example of syndone,
panic room and flor de loto - nr 134,135 and 136 now in this list
http://www.progarchives.com/top-prog-albums.asp?ssubgenres=&syears=2014&scountries=&sminratings=1&smaxratings=0&sminavgratings=1&smaxresults=250&x=78&y=8#list

again: panic room has MORE average rating, More ratings and MORE reviews(if it would even count) than syndone´s album! then comes flor de loto with 1 lousy rating of 2 stars - after this album there are some 40 albums ALL of which have again ofc more average rating (between 2,79 and 3,63) then this album on position 136.
you can find more examples of this if you search the list in the later positions...

does noone see here a bug? or am i just too dumb now to get that there is everything ok in the list?^^

ok now i am totally doubting my brains!
i always thought the higher the rating the better and the more ratings/reviews the better (becuase its more representative)
looking in the all time toplist:
red (nr 8) and animals (nr 9) both 4.52, but red has less ratings
get to nr 11 to 15 (all 4.42):
number of ratings goes down from 11 to 12 to 13 to 14.. but 15 has more ratings (and way more reviews, dont tell me this is to take into account here anymore! ;) ) than position 14.

am i on to something or just plain stupid?
thank you so far... :)
It's just a ride... <3
Back to Top
Guldbamsen View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin

Joined: January 22 2009
Location: Magic Theatre
Status: Offline
Points: 23098
Direct Link To This Post Posted: May 17 2014 at 04:29
I am not the most proficient in PA ratings system, but I do know that we erase a lot of bogus accounts, rating manipulators and the likes, but before the affected release receives a new rating after we've fondled with it, it still looks warped.
Anyway don't get too hung up on ratings - most of us aren't
“The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.”

- Douglas Adams
Back to Top
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Posted: May 17 2014 at 04:51
ok but in the above mentioned cases they must have directly altered the QWR? did you look up the obvious spots i linked? it still seems like a bug to me...

and your right, ratings are not that important especially when it comes to the albums one personally loves most..
and yet i still like to see what most of you like and my sense of perfection wants a conclusive toplist Wink
It's just a ride... <3
Back to Top
Guldbamsen View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin

Joined: January 22 2009
Location: Magic Theatre
Status: Offline
Points: 23098
Direct Link To This Post Posted: May 17 2014 at 05:19
I am on my useless phone for the next couple of days, but I actually wanted to do a search of Dean's posts on this particular matter, because I know I have read him on numerous occasions explaining it to perfection. Btw Dean is a retired admin on PA.
I think he would say that there is no bug, and the way you look at the rating algorithms basically is faulty. Sorry, but that is my analysis;-)

Maths interest me about as much as parking lots, so I have never looked into the mechanics of PA's rating system. Tell you the truth, most of the time I feel it's more of a hassle than anything else. The whole "ratings' thang:P
I guess it's nice to have some kind of musical bearing on PA users - you can always look at his/her ratings and read a little bit of personality from it. It's only when we get to the 'sportsmanship' part of the deal where the wheels start falling off the cart. Those people who suddenly decide to equate music with sports, and then proceed to go for the top slot, the wonderful gold medal on PA!!!!

Yep, completely ridiculous, but you'd be surprised to learn just how many who actually do this, whether intentionally or unintetionally. Some literally do believe that you can put sports and music in the same bag, and that, I feel, is a frightening thought. Music is supposed to be something else right? At least that's what I hope it is.

Edited by Guldbamsen - May 17 2014 at 05:45
“The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.”

- Douglas Adams
Back to Top
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Posted: May 17 2014 at 05:43
thanks for you answer!
i also tend to think that i just got something wrong but i wont be satisfied until i figure out or someone tells me what exatcly is right about a higher qwr with 1 rating 2 stars compared to a lower qwr with 31 ratings and 3,69 stars.. sorry Wink but you gave your best!

i hope some1 else is willing to explain this issue to me annoying pedant Tongue
It's just a ride... <3
Back to Top
Guldbamsen View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin

Joined: January 22 2009
Location: Magic Theatre
Status: Offline
Points: 23098
Direct Link To This Post Posted: May 17 2014 at 05:51
No problem:-)

Next time I'm on my laptop, I will probably be able cook something up for you. Though, I'm guessing, not until next week. Denmark's biggest karneval launches on Saturday, so the weekend is definitely out for me:-)
“The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.”

- Douglas Adams
Back to Top
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Posted: May 17 2014 at 09:47
ok guys this problem is not solved yet! i am still convinced that we have huge bugs in the rating system!


It's just a ride... <3
Back to Top
Andy Webb View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin

Joined: June 04 2010
Location: Terria
Status: Offline
Points: 13298
Direct Link To This Post Posted: May 17 2014 at 17:07
Ok - if this was a test I would definitely get partial credit for this answer, as I'm not 100% sure as to what the algorithm for the QWR is, but I'm fairly certain I have the general idea behind it, so I can conjecture a good guess. I think I covered all my bases there when all the math I'm about to do is horribly wrong.

Take a look here for a microcosmic look at how the QWR is calculated, kind of. 

There are a few data points you need to keep in mind when fiddling with these numbers. This query, of the best prog albums of 1955 (lol) is very small with a small number of ratings for only 2 albums. We need to keep in mind the average number of ratings for all albums in this query (that being 8 ((13+3)/2)), the average rating for the query (4.04), and the rating and number of ratings for each album. Got those? Ok, lemme explain some stuff first -

The QWR is a method that PA uses to ensure that albums with a couple of 5 star reviews from collabs don't become the number one album on the list. When the top of the list says it uses the weighted average theory, it's kind of a rough weighted average - that is, the average number of ratings and average ratings are literally a 'weight' that holds down higher ascending highly rated albums, as you can see by Miles Davis and Milt Jackson having a QWR that's .02 lower than it's actual rating and even lower that the weighted average of the two (4.065).

So, with those four numbers in mind, think of it this way - if we weight the average of each individual album with the average of the aggregate query (8 average ratings, 4.04 average score), we can get a fairly fair QWR. One equation I found when doing some research on this topic was this:

(NxR)+(nxr)/(N+n)

N = average number of ratings
R = average score

n = individual number of ratings
r = individual score

If you punch in the four numbers for our 1955 example, it works, to a degree. The actual QWR varies slightly which could be due to either aggregation issues or slight adjustments to the curve, which is what I suspect it is. Max has tweaked the algorithm countless times over the years, and the elimination of ratings abusers has left some of the stats a little scarred. 

When you apply this to your issue with 2013, of course we first come to the issue that it's difficult to ascertain the N and R for all of 2013, and even when solving the nasty systems of equations you can make with two albums, the numbers are uncertain. I got N=1193 and R=3.8608 for one combination of albums, which is clearly not correct seeing as SWilson's TRTRTS only has 995 reviews. My assumption (from the opinion of someone who's math education only goes up to Calc II) is that the N and R follow a curve that's probably logarithmic in some sense. When I dug through the source code in attempt to find the .js query that solves all this I couldn't find anything, so Max would have to be the one to ask about that. And interestingly enough, I think Dream Theater's album, the one that you said was #150 with 400+ ratings, has been booted from the list. That's another thing about this list - it's very volatile.

For the 2014 list and the album with 2 stars chilling in the top 150, I'll have to ask Max about that - it may just be a computational error or something was deleted that used to be there. Even with the formula and the curve that shouldn't be there. 

I don't think anyone admits the system is perfect. With the dozens of little tweaks to the formula Max has made over the past ten years, there are bound to be errors in the variables.

Hope this answered your general question - I love spelunking through the caves of this site's history, so please say so if something was left untouched.

Back to Top
Guldbamsen View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin

Joined: January 22 2009
Location: Magic Theatre
Status: Offline
Points: 23098
Direct Link To This Post Posted: May 17 2014 at 17:19
Thanks Andy.-)

I get a head ache just looking at that stuff.
If anyone is interested in Max Ernst or Jack Kerouac though, then I can be of service. If it's to do with algorithms and codes, then I'm out:-P
“The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.”

- Douglas Adams
Back to Top
Mind_Drive View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 06 2009
Location: Germany
Status: Offline
Points: 389
Direct Link To This Post Posted: May 17 2014 at 17:32
hey, thanks for your detailed answer!

i think i really understand the complex rating system, which i think is pretty awesome
but still: DTs album is still there when i look at the list and some strange rankings in other places i don´t feel that they are correct..

i guess i just have to deal with it and as i said it´s just my perfectionism thats speaks here^^ so most of you probably wont bother about some misplaced albums - and so should i Wink
what counts is the music and wow! i can´t tell how much i love the archives for it has enriched my life so much with so much inspiration for new music...

thanks so far! Smile


It's just a ride... <3
Back to Top
DamoXt7942 View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Avant/Cross/Neo/Post Teams

Joined: October 15 2008
Location: Okayama, Japan
Status: Offline
Points: 17447
Direct Link To This Post Posted: May 18 2014 at 06:34
Sorry I'm not familiar with PA rating system (and of course QWR ... oh my, that gives me headache like David LOL ), but let me say we Admins sometimes kick some ratings for hyping or manipulating out, but upon such a case, the rating system cannot usually work precisely ... there are a couple of errors or specifications also in PA, actually. Unhappy
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575
Direct Link To This Post Posted: May 18 2014 at 12:55
Originally posted by Mind_Drive Mind_Drive wrote:

hey, thanks for your detailed answer!

i think i really understand the complex rating system, which i think is pretty awesome
but still: DTs album is still there when i look at the list and some strange rankings in other places i don´t feel that they are correct..

i guess i just have to deal with it and as i said it´s just my perfectionism thats speaks here^^ so most of you probably wont bother about some misplaced albums - and so should i Wink
what counts is the music and wow! i can´t tell how much i love the archives for it has enriched my life so much with so much inspiration for new music...

thanks so far! Smile


Hi Constantin,

Hmm.... I'm confused. I've read all your posts in this thread and I don't see what the problem is or why you think it is a bug.

I've looked at the chart and everything looks okay to me. The albums are ranked by their QWR rating, and that value is shown in the listing.


Can you explain exactly what you think the problem is.
What?
Back to Top
 Post Reply Post Reply Page  12>

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.586 seconds.
Donate monthly and keep PA fast-loading and ad-free forever.