Jethro Tull’s "Thick As a Brick" #1 Prog?

Many thanks for your comments, Easy Livin. My replies are given below:

Easy Livin wrote: “I'm not convinced by that argument. The fact that Album B has an addtional 50 4 star ratings does not make it either more popular, or better.”

It makes it more popular in my book (and who said anything about "better", which is a qualitative term and subjective in this case?). If Album A has the following:

50 x 5-stars, 0 x 4-stars, 0 x 3-stars, 0 x 2-stars, 0 x 1-star, 0 x 0-stars

and Album B has the following:

50 x 5-stars, 50 x 4-stars, 0 x 3-stars, 0 x 2-stars, 0 x 1-star, 0 x 0-stars

then I cannot see how anyone could possibly argue that Album B is not more popular than Album A. Popularity is a function both of how much an individual likes something and how many people are of the same, or similar, opinion.

Easy Livin wrote: “Let's assume that the same 50 people gave five stars to albums A and B. If the 50 peopel who also rated album B then listened to album A, and 49 gave 4 stars, it would only take one one to give 5 stars for album A to be deemed the more popular or better.”

You’ve lost me there, Easy. If you are saying that the same 50 people rate both albums at 5-stars and then 49 of them go back and add 49 4-star ratings to Album A, that could not be done because a person quite rightly cannot 'vote' twice, i.e. they cannot go back and also add another 49 4-star votes to A. However, if you are trying to say:

Album A has 50 5-star ratings from Group 1 and 1 5-star rating from Group 2 and 49 4-star ratings from Group 2

Album B has 50 5-star ratings from Group 1 and 50 4-star ratings from Group 2

then the sum of ratings for Album A would be 451 (average 4.51 stars) and the sum of the ratings for Album B would be 450 (average 4.5 stars). To me, Album A *is* more popular: notice that the sum of the ratings reflects that and, in this specific instance, so does the average.

Eaay Livin wrote: “A don't like the idea of volume of ratings beign factored into the caculation. You say that for album B, twice as many people "liked it". I agree that 4 stas implies that they did like it, but what about 3, 2 and 1 star. 2 stars, and 1 star, and indeed no stars, imply that people did not like the album, yet any algorithm which takes account of the volume of ratings will indicate that an album is "popular", because it ahs a lot of reviews, even if the majority of the ratings are low.”

Volume must be taken into account in popularity statistics. If only 50 people have rated Album A, and let’s say they all rate it at 5 stars, and only 200 people have rated Album B, all at 5 stars, the average rating for both albums is 5 stars. But which is the most popular? I did not say "better", which is a matter of personal opinion. Clearly Album B. I cannot see how anyone could possible argue differently.

Easy Livin wrote: “Examples, the Angalard albums were high in the collaborators chart because they had a lot of ratings, not necessarilty a lot of high ratings. A couple of months ago, a collaborator omplained that they had given an Angalard album a low rating (2 stars), yet this had caused it to move up a place in the collaborators chart! That is ludicrous.”

I agree with you, as I did in a previous thread, which is why I have proposed a different algorithm to M@X. However, averages also produce ludicrous results, as my previous example shows (and as many other examples also show - I have sent a few to M@X).

Easy Livin wrote: “Charts based on analysis of the ratings will always be for amusement purposes only.”

I agree. However, that does not mean that patently deficient ways of calculating them should be used, and both averaging and the previous algorithm are patently deficient (although the overall result when using averages is worse in my opinion). However I am not advocating a return to the previous algorithm, but changing to a new algorithm.

Easy Livin wrote: “Personally, I prefer the averaging method to one which takes account of volume. I accept though that there should be a minimum number of reviews (say 10) before an album is considered to have enough reviews for the average to be meaningful. If volume is to be taken into account, the algorithm must be designed in such a way as to ensure that a low rating does not lead to the album being reported as more "popular".”

As I have said before, volume must be taken into account; to ignore volume is to ignore an important factor in popularity assessment. However (and this is where I think you misunderstand my motives because I believe you may be thinking I am advocating a return to the previous algorithm when I am not) the new algorithm I have proposed to M@X does take into account that low ratings are, effectively, votes of unpopularity.

Averaging is most definitely not the correct way to assess rank in this situation. Let’s wait and see the result of M@X’s trial with the new algorithm. But I can assure you that using averages alone is a waste of time and can lead to completely wrong rankings. They are not helpful unless the statistical sample follows a Normal Distribution and, even then, can easily offer an incorrect assessment of popularity when the number of ratings per album varies greatly.

Best Regards.

Edited by Fitzcarraldo

Easy Livin wrote: “I maintain that any chart which appears on the home page should attmept to guide people to the albums we as a group find to be the best of the genre. These are not necessarily the most "popular" ones. For example, there are a lot of reviews of Genesis later albums, so by implication they are "popular". The reviews are almost universally adverse though.”

Easy Livin, my definition of “best” is not yours when discussing qualitative and subjective art. What you consider “best” albums are not necessarily what I would consider “best” albums, so my list of “best” albums would not necessarily be yours. Therefore popularity *is* a useful measure, particularly as we (and by “we” I refer to all reviewers, not just Collaborators) are a relatively small group of discerning fans of Prog Rock. Furthermore, I do want to know what are the most popular Prog Rock albums. I don’t believe I am the only one. That's what Top 10 (or 50 or 100 or 200) lists are all about, after all.

Easy Livin wrote: “Sorry, I had some diffculty expressing this coherently! What i'm trying to say is, we have 100 people. 50 of them have rated both album A and album B, and given them 5 stars. The other 50 have only rated album B and given it 4 stars. What I'm trying to say is that your logic says that album B is more popular. What I'm saying it that we don't know what rating the other 50 would give album A. They might all give it 5 stars, making it more popular.”

I’m sorry, Easy, but predicting the future is pointless. If, and when, the “other 50” were to rate Album A then the ranking would change in accordance with their ratings. But to base your argument on what people would or might do is ridiculous. I want to know the ranking based on existing ratings, not what people might rate in future. The Billboard Top 200 is based on actual sales, not on future sales that might, or might not, happen.

Easy Livin wrote: “But if one of the 200 gave 4 stars, for me album A should be higher in the chart, even if it has less ratings.”

You’ve lost me again, Easy. If you saying the following:

Album A: 50 ratings only, all 5-stars

Album B: 200 ratings only, 199 being 5-stars and 1 being 4-stars

then I totally disagree with you. To me Album B is the more popular. You could only say the opposite if the number of ratings were the same per album, viz.:

Album A: 200 ratings only, all 5-stars.

Album B: 200 ratings only, 199 being 5-stars and 1 being 4-stars

Then, and only then, would I agree that Album A is more popular than Album B.

Easy Livin wrote: “Suppose 100 of the 200 gave 5 stars, and 100 of the 200 gave 1 star (unlikely and extreme example to illustrate the point.) Is album B still more "popular". Yes more people have rated it, but 50% hated it!”

Again, I’m not sure what you are saying. If you are saying the following:

Album A: 200 ratings only, comprising 100 5-stars ratings and 100 1-stars ratings

Album B: 100 ratings only, comprising 100 5-stars ratings

then I understand where you are coming from (as I have said in the past) which is why, as I have said, I have proposed a new algorithm to M@X that takes low ratings into account.

Easy Livin wrote: “Indeed "Better" is a personal opinion, but that is what the ratings reflect, people's opinions. The chart should reflect which albums people think are the best, not the ones most people have heard.”

To me you appear to be contradicting yourself here, as you earlier wrote that, in your opinion, a Top 50 list should reflect what is “better” not what is popular. My contention is that a popularity chart – when the sample population (i.e. number of fans) is relatively large (i.e. not a handful of Collaborators) – does tend to reflect what people think are the best and not just what they have heard. As several people have written in these forums in the past, some people (most?) do not bother to review albums that they do not like or have not bought (because they don’t like them being one possible reason, although they may of course never have come across it). And, in any case, I am interested in seeing a Top 50 popularity list. I want to know what is popular with the collective fan base. If, say, 200 people rate e.g. a DREAM THEATER album highly and only 50 people (say) rate a TULL album highly, I want to see that reflected in the Top 50. That’s what a Top 10 (or 50, or whatever) is all about!

Easy Livin wrote: “We do need to bear in mind that there have been compaints about the site being slow recently. Max wondered if this was becuase of the complex calculions being done on some of the pages, so we do need to keep a balance between a complex algorithm, and an efficient one.”

To argue that we should keep what you call an “efficient” algorithm (I would use the term inaccurate, not efficient) just because you believe it to be efficient, even if it is misleading, is nonsensical in my opinion. I have seen the SQL query for the current algorithm. The new algorithm should not make much difference (in fact it might even be faster as it involves no floating point division (which calculating an average requires) and it does not need to check if there are less than 30 reviews for each album. In fact, as I mentioned, the current SQL query involves floating point arithmetic but the algorithm I propose could be constructed using faster integer arithmetic (although I have couched it in floating point terms in my explanation to M@X). In any case, the algorithm could be run using a daily or hourly database trigger or an aperiodic trigger, i.e. it does not need to be run every time someone visits the Home Page (if M@X wanted to have belt and braces, he could put a note to that effect on the Home Page).

Easy Livin wrote: “That's good to hear Fitz, What about 3 stars, is that a neutral rating or a positive one? What do others think (do they care? )?”

No, Easy, I would not classify 3 stars as a neutral rating, for two reasons: firstly the meaning of the ratings (which I have taken into account): 3 stars signifies “Good, but non-essential” and secondly, the current system is a 6-star, not a 5-star system:

1 -- 5 stars: Essential: a masterpiece of progressive music.

2 -- 4 stars: Excellent addition to any progressive music collection.

3 -- 3 stars: Good, but not essential.

4 -- 2 stars: Collectors/fans only.

5 -- 1 star: Poor. Only for completists.

6 -- 0 stars: Bad. Do not buy!

Ratings of 3 stars and above are ‘positive votes’ and ratings of 2 stars and below are ‘negative votes’. My new algorithm takes this into account. Only positive votes could raise ranking and, conversely, only negative votes could lower ranking.

Easy Livin wrote: “I agree averages are deeply flawed, but cannot myself think of any way of improving upon them. Happy to be proved wrong though!”

You may not be able to, Easy, but I can. My new algorithm is better than both the use of averages and the original algorithm (which was to compare the sum of each album’s ratings). I believe that market researchers, professional statisticians and medical researchers – who all conduct surveys which result in ordinal data like that produced by ProgArchives ratings - would baulk at using averages. richardh’s statement that the current ranking system is “quite clearly useless” is absolutely correct. I think that you are too tied to the use of averages and should be prepared to try something else. (Sorry if that sounds pejorative, it isn’t intended to be).

 

Best Regards.

Edited by Fitzcarraldo

what a silly post

bluetailfly wrote: I am tired of seeing Jethro Tull's "Thick as a Brick" as the #1 prog album week after week Why is "Brick" #1?

Well, after 10 total posts it looks like you got what you wanted, blueballsfay. Thick as a Brick dropped down to #18. I guess your other self pushes a lot of weight around here.

Fitzcarraldo wrote: All: M@X has changed the Top 50 to use a new algorithm, and I am checking it. This will take some time. However, bear in mind that the SQL query may not have implemented my algorithm correctly, so don't jump to conclusions - good or bad - just yet. "Thick As A Brick", for example, appears to be too low on the list on the Home Page (I have just calculated the ranking for a couple of albums using my algorithm in an Excel spreadsheet at home and TAAB would come above "Tales From Topographic Oceans", not below it as shown on the Home Page). So bear with us for a while until the correctness of the SQL query can be checked.

I don't know what algorithm is used now, but it doesn't seem to reflect the top 50 as I understand it should be (not implying we should create an algorithm that would manipulate the outcome to suit our thoughts).

 

I think the mistake is being made that the number of reviews/ratings has an important say in the outcome of the calculation. I think (humbly) the number of ratings should only be used to decide whether an album is allegeble for inclusion on the top 50 list.

 

So, an album should have at least 30 ratings and 5 reviews (arbitrary numbers of course), if the album passes that test, than the absolute average (perhaps minus top 5 and bottom 5 % ratings for normalisation) should be used, and not again weighing in the amount of ratings.

 

situation now, allows Dream Theater to become second, because it has +240 ratings, while only 47% rates the album 5 stars.

Maybe I'm to early with my comment, as you said it's still under investegation, but this is as good a time as any to reply

Great so M@x wastes his time over stupid opinion threads like this instead of adding a new virbrant group as to the email rules he posts. Hope you guys are happy because this is stupid. WHo really cares what album shows up as number 1? Go work for some pop radio station!

Train Of Thought NO2 ???
Thick As a Brick NO18 ???

I think most of us agree that this system isn't working. But let's give it time if it's still being investigated.

Fitzcarraldo, I agree. The only thing that bothers me here that "Train Of Thought" has almost 25% of 1 or 2 star ratings and still it is at number two position. The only reason why it's there is because it has so many reviews.

The fact that DT's "Train of thought" is number 2 does indicate to me that the algorithm is flawed. As has already been said, it is controversial even among DT fans. I think the whole issue revolves around the definition of "poplular".

In very simple terms, Fitz, I think I am right in saying that if:

Album A has 200 ratings, all 4 stars, and Album B has 50 ratings, all 5 stars

you would propose that album A is more "popular". I maintain that album B is more popular. Album B should appear higher up in any chart which puports to represent the opinions of the site participants. While a lot of people thought album A was very good, everyone who has rated album B says it is a masterpiece. The present algorithm results in pretty good albums charting higher than masterpieces, just because more people have rated them. 

An algorithm which places so much emphasis on the number of ratings would be more valid if everyone who rates albums confirmed they had rated all the albums they are ever going to do. This of course is not, and will not be the case. I'm sure we all have many albums still to review, the ones we have done are not necessarily the most popular even to ourselves.

What I mean by that is, I might have listened to album C twice, and written a review for it. I listen to album D every day, but have not yet reviewed and rated it. The present algorithm says that to me Album C is more popular than album D.

As a compromise, how about an algorithm which has less emphasis on the number of ratings, and more on the ratings themselves?

Garion: Max is a bg boy, and can decide for himself whether he is "wasting his time". I reckon he's keen to have a representative chart on the home page, as it offers casual visitors a snapshot of what we as a community like (Train of thought excepted!).

 

Edited by Easy Livin