Data on score development

Logan wrote: To go completely on a tangent: One big problem, though, is that there is not a standard rating methodology being used by enough people (I think there should be more detailed criteria for doing a rating, I know that reviews can justify one's rating, but...).  Perhaps a table with various criteria, each with a different rating which is then averaged: Originality, creativity, technicality progressiveness, how it rates versus other albums by the artist (if there are any and the person has heard others), how it rates against other artists in the category, how it rates generally compared to the rest of Prog, subjective rating, "objective" rating, or whatever... and then an average of those.  Whatever criteria one chooses,; just ideas and some redundancy).  One can justify the rating in a review, but the rating process itself should be meaningful.

Anyhow, I was playing around, and found out you can use geometric series. The formula S(n)=4,9-0,005*(0,995^n-1)/(0,995-1) fits pretty well with the rating development of Fear of a Blank Planet, for example, where S is score and n is number of votes. If you study this, you will be able to find the best k (0,995) and a1 (0,005). 4,9 here is supposed to indicate which level the album was expected to begin scoring at.

Edited by Sofagrisen - December 14 2007 at 07:08

MikeEnRegalia wrote: ^ I think if you explore this further then you'll discover that the development varies a lot for different genres ... for example Prog Metal albums tend to get bashed more heavily than Symphonic Prog albums.

Yeah, I am sure about it, but theoretically you could find the function that fits each album best. I don't know if there is a relatively easy way, or if it will be very demanding. I am not a mathematician, but that being said, I think geometric series could be applied to basically any album. That was the main idea, not the details of the formula. And just because all of this isn’t figured out and it is difficult, doesn’t mean we should just ignore it, and keep what we have. What we know is that ratings decline. When we make top lists, we compare albums with different numbers of voters. But because ratings decline, it’s not enough to exclusively just look at rating, you have to take the number of votes into account, or else it would be unfair to the albums with more votes. And I know it is already taken into account on this site, but being taken into account, is not the same as being well enough taken into account.

Part of the problem is the whole issue is considered from a bad perspective. It is often considered from the perspective one can be more certain about the “true level” of an album with many votes, while an album with fewer votes will be less stable, and you can’t really know if it will go up or down. It has yet to prove itself. To be honest, this is ridiculous, because the rating of the album will decline. There really is no uncertainty about that. Then there is yet another alternative perspective, that number of votes is an indicator of popularity, and that this popularity should be awarded. The last perspective gives formulas that counteract the effect of declining ratings, but not at all as precisely as I would like. The focus is wrong.

Edited by Sofagrisen - December 14 2007 at 09:26

nobody should take top N lists *that* seriously ... especially not the topmost positions - it doesn't really matter whether an album is #1, #2, #3 etc.. Ratings will always fluctuate ... and like I said above: If you try to compensate for these fluctuation people will simply adapt their voting behaviour.

BTW: don't think there's anything wrong with the algorithm ... people can see that FoaBP has much more votes than the other albums and decide for themselves what to make of it. The simple truth is that there is *no* way to combine two separate/independent numbers (# of ratings, avg rating) in one number/"score".

MikeEnRegalia wrote: ^ how could this phenomenon be taken into account? It's only natural that as an album "ages", reality starts kicking in. Another possibility is that once an album reaches a critical number of review which put it in the top 100 or at the spot of "album of the week", bashers begin submitting 1 star ratings. Regardless of the reasons, how can this be taken into account ... are you suggesting that we should artificially tune album averages depending on how old they are in relation to the release date?

Sofagrisen wrote: I have collected some more data now. Does anyone see the pattern? ::snip::

Sofagrisen wrote: darqdean wrote: Sofagrisen wrote: I have collected some more data now. Does anyone see the pattern? ::snip:: All data will do that, even randomly generated data will demonstrate a similar trend as the calculated average score will tend towards the predicted average as the population increases. All this shows is that the more ratings an album gets the more accurate the average score is. I mentioned this perspective earlier, and I do agree the rating over time will tend towards the predicted average. That being said, what you see is that that how this happens is not random. It’s not 50-50 between the ratings going up and down. The ratings are basically always going down, but at a decreasing speed. Sure the data does not conclusively prove anything, but it is all the data I have, and none of these albums starts out low and then climbs, and I really think that is no coincidence at all. I would like to have more data, and I am pretty sure Prog Archives themselves could generate it, but that’s not an option for me. That being said, I do not base my opinions solely on this data. I have observed this affect myself over and over again in many albums I never collected any data for. Only reason I collected this data, was to make a stronger argument for my case in a thread like this, because obviously, people won’t believe my observations, they have to see it for themselves.

Why does it require interference? People are being honest. Now, if a specific person is trying to abuse the system, then we can usually tell, but I don't understand why we would or how we could accurately "combat" it.