Forum Home Forum Home > Progressive Music Lounges > Prog Blogs
  New Posts New Posts RSS Feed - Graphing the PA Top 250
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Graphing the PA Top 250

 Post Reply Post Reply
Author
Message
Zugzwang View Drop Down
Forum Groupie
Forum Groupie
Avatar

Joined: March 01 2017
Location: United States
Status: Offline
Points: 79
Post Options Post Options   Thanks (4) Thanks(4)   Quote Zugzwang Quote  Post ReplyReply Direct Link To This Post Topic: Graphing the PA Top 250
    Posted: October 27 2017 at 12:30
Back in March, I recorded the top 250 albums on Progarchives in Google Sheets to mess about with various graphs. I stumbled across my spreadsheet earlier this week and figured some of you might be interested. For full sized images, the album is here: https://imgur.com/a/z0GXj
Raw data (as comma-separated values) is here: http:///pastebin.com/kVczKXaN

First: A line graph of each year compared to both number of albums for that year and number of ratings for that year. The red line represents the top 25 albums, the blue line represents the top 250 albums.


As I'm sure everyone here could guess, there's a large spike in the early seventies. The spike is even larger when measured by number ratings instead of album count. The earliest album is Miles Davis' Kind of Blue (1959), the latest is Mike Oldfield's Return to Ommadawn (2017) which is now no longer in the top 250.

Second: A stacked area chart of country representation.

As you can see, the first non-British album to show up is Canadian (Rush's Moving Pictures). The United States has a slow start, but grows to be almost as large as the UK. Italy, of course, maintains a respectable size throughout.

Third: QWR rating, compared to both album rank and by number of ratings



There's a pretty large divide between the top 5-10 albums and everything afterwards. As the first graph shows, QWR rating starts to level out a bit more after the top 50. As the second graph shows, the highest rated albums tend to also have more ratings. Lower dots show albums that are rated higher than their number of ratings would imply, higher dots show the opposite. I annotated the graph with some album covers to demonstrate:


Fourth: A world map, colored by density of albums in the top 250 (darker color = more albums):


Finally: A genre pie chart, measured by both number of albums and number of ratings.


The more popular genres have larger slices in the second graph than in the first. For example, Symphonic prog goes from 15.6% of the number of albums in the top 250 to 24.1% of the number of ratings of top 250 albums. RIO/Avant, Zeuhl, and Fusion both drop in half.

That's all nothing too surprising, but I figured there are some other data nerds on this forum who'd appreciate this. Smile

Edited by Zugzwang - October 27 2017 at 12:45
Back to Top
Larkstongue41 View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: July 07 2015
Location: Eastern Canada
Status: Offline
Points: 1271
Post Options Post Options   Thanks (0) Thanks(0)   Quote Larkstongue41 Quote  Post ReplyReply Direct Link To This Post Posted: October 27 2017 at 12:36
Nice! Thank you for this Thumbs Up
"Larks' tongues. Wrens' livers. Chaffinch brains. Jaguars' earlobes. Wolf nipple chips. Get 'em while they're hot. They're lovely. Dromedary pretzels, only half a denar."
Back to Top
The.Crimson.King View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: March 29 2013
Location: WA
Status: Offline
Points: 3900
Post Options Post Options   Thanks (0) Thanks(0)   Quote The.Crimson.King Quote  Post ReplyReply Direct Link To This Post Posted: October 27 2017 at 12:47
Great job and fun to read.  Looks like '82, '86, '90 & 2010 were pretty slow years Wink
Back to Top
Quinino View Drop Down
Special Collaborator
Special Collaborator
Avatar
Errors & Omissions Team

Joined: June 26 2011
Location: Portugal
Status: Offline
Points: 2604
Post Options Post Options   Thanks (1) Thanks(1)   Quote Quinino Quote  Post ReplyReply Direct Link To This Post Posted: October 27 2017 at 15:08
Thanks, Bryson, good job !
I once did professional statistical work and simply adore graphic treatment of data, this was a real treat for me Geek
"Before beauty disappears entirely from the earth, it will go on existing for a while by accident" - Milan Kundera
Back to Top
Squonk19 View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: April 03 2015
Location: Darlington, UK
Status: Offline
Points: 1329
Post Options Post Options   Thanks (0) Thanks(0)   Quote Squonk19 Quote  Post ReplyReply Direct Link To This Post Posted: October 27 2017 at 17:26
Wonderful stuff! The kind of thing I loved doing when I was in Research and Development, but now I'm a Physics teacher - I don't have the time!

What stands out for me is - what a great decade the 70s was for our music! Also what a dark age period the 90s were for us, and how nice to see the prog resurgence from the start of the Millennium onwards.

Were we all young and vibrant in the 70s, got married with kids and got a mortgage in the 90s and did we refind our mojo with prog again in the last decade or so? .... or did the new breed of musicians revitalise us! Hopefully the latter.

Edited by Squonk19 - October 27 2017 at 17:28
"...and the man in the mirror has sad eyes."
Back to Top
Sean Trane View Drop Down
Special Collaborator
Special Collaborator

Prog Folk

Joined: April 29 2004
Status: Offline
Points: 13569
Post Options Post Options   Thanks (0) Thanks(0)   Quote Sean Trane Quote  Post ReplyReply Direct Link To This Post Posted: October 30 2017 at 03:19
great jobClap...

Kind of surprised Finland doesn't rate higherConfused


BTW, on your world map, you may want to colour the Spitzberg Islands (east of Greenland) the same colour as Norway, since they're part of it.

Do you think your stats could be expandeed to the top 1000?
Back to Top
mechanicalflattery View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: August 08 2016
Location: Seattle
Status: Offline
Points: 1056
Post Options Post Options   Thanks (0) Thanks(0)   Quote mechanicalflattery Quote  Post ReplyReply Direct Link To This Post Posted: October 30 2017 at 10:25
I'm interested to see what graphing individual PA members' favorite album lists would look like. Maybe even do a regression analysis to determine the signficance of average rating and # of ratings on list placement, although obviously it'd differ with each individual. Besides, it'd be interesting to see the distribution of years and nationalities on a given list, including my own. I'd probably use RateYourMusic over Progarchives though.

And they said prog fans couldn't get any more self-absorbedly nerdy. We're literally graphing our favorite music now...
Back to Top
omphaloskepsis View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: October 19 2011
Location: Texas
Status: Offline
Points: 1310
Post Options Post Options   Thanks (0) Thanks(0)   Quote omphaloskepsis Quote  Post ReplyReply Direct Link To This Post Posted: October 30 2017 at 14:29
  "Tales of Graphical Devotion" 
Masterpiece of information Dude!  Bravo!
Back to Top
omphaloskepsis View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: October 19 2011
Location: Texas
Status: Offline
Points: 1310
Post Options Post Options   Thanks (0) Thanks(0)   Quote omphaloskepsis Quote  Post ReplyReply Direct Link To This Post Posted: October 30 2017 at 14:39
I found the ratings per year chart disturbing as it pertains to Prog Archives.   Downhill from 2013, even though the last few years have been banner years for prog albums.

Could it be that the " ratings per year" chart suggests Prog Archives is dying?   


Before anyone assumes I jumped to an insane conclusion ,  I did say "suggests".  
Back to Top
siLLy puPPy View Drop Down
Collaborator
Collaborator
Avatar
PSIKE, JR/F/Canterbury & Eclectic Teams

Joined: October 05 2013
Location: SFcaUsA
Status: Offline
Points: 4855
Post Options Post Options   Thanks (0) Thanks(0)   Quote siLLy puPPy Quote  Post ReplyReply Direct Link To This Post Posted: November 22 2017 at 19:12
Posting this here since i can't find a better place to post this AND everyone should appreciate Zugzwang's awesome graphs here anyways :)

WHY I'M POSTING THIS!

I'm wondering WHY Wobbler's "From Silence To Somewhere" can make it to #19 on the Top 250 list with only 135 ratings just ahead of Anglagard's "Hybris" with 1546 ratings and Rush's "Hemispheres" with 2152 ratings.

I know collabs and admins are given magical rating powers that amplify their vote casting but this has to be the most incredible chart positioning for a brand new album with only 135 ratings that i've ever noticed.

BTW, i just got this album and agree that it is a 5 star album and a masterpiece of the 21st century. I'm just perplexed by how the charting system works.

Any takers on this matter?

https://rateyourmusic.com/~siLLy_puPPy
Back to Top
Zugzwang View Drop Down
Forum Groupie
Forum Groupie
Avatar

Joined: March 01 2017
Location: United States
Status: Offline
Points: 79
Post Options Post Options   Thanks (0) Thanks(0)   Quote Zugzwang Quote  Post ReplyReply Direct Link To This Post Posted: November 22 2017 at 20:37
I don't know the exact formula used to calculate QWR, but it looks like the effect of rating count starts to drop off after a few hundred ratings. I'd guess this is mostly due to the bonus given to collaborator reviews dropping off as the percentage of reviews that are from collaborators decreases.

Some examples:
From Silence to Somewhere: 135 ratings, 4.62 average, 4.36 QWR
Cardiacs' Sing to God: 263 ratings, 4.30 average, 4.19 QWR (This is the lowest # of ratings in the top 100)
Hybris: 1546 ratings, 4.38 average, 4.35 QWR

There's always an initial spike when an album is first released, because the people who rate it first are usually people who like the album. Then, as the album becomes more known, the rating balances out as the "I'll listen to this because it has good reviews" crowd comes in. I think this is just a case where the album is really good, and the band is well known enough to have a bunch of early reviews but not well known enough to have many people reviewing it just because it's a significant band. So it hit a bit of a high rating sweet spot.

Unrelated: Thank you all for the kind comments!
Back to Top
miamiscot View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: April 23 2014
Location: Ohio
Status: Offline
Points: 996
Post Options Post Options   Thanks (0) Thanks(0)   Quote miamiscot Quote  Post ReplyReply Direct Link To This Post Posted: November 27 2017 at 09:27
Nice work. Very interesting!
Back to Top
moshkito View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: January 04 2007
Location: Grok City
Status: Offline
Points: 7532
Post Options Post Options   Thanks (0) Thanks(0)   Quote moshkito Quote  Post ReplyReply Direct Link To This Post Posted: December 24 2017 at 08:27
Hi,

This is very tough, because PA is the best site out there for anything progressive, but these graphics while meaningful and important, in many ways, do not represent the state of the music, and neither do they represent the tastes in people's tongues for the music they like.

It also creates an impression that for several years nobody liked music, and that is because the only albums used were the "known" ones, that everyone voted for, and since the Internet was not up and about live like it IS NOW, there are almost no new bands or music mentioned that made any difference or had an interest that would have been similar to any of the great names that everyone had already loved for 15 years ... thus, it appears that no new music had shown up, when in reality the setup of the graphs and the numbers, excludes the new material and does not have any special weights that measure the differences ... for example ... an album that was done in 1972, can get its 5 rating ... but that album can NOT get the 5 rating in 1987, because it is NOT an album that came out in that year ... which would give better weight to the albums that were new, and to new artists ... instead, the numbers you work with only show a CORPORATE STYLE of numbers, which is, that their top 5 is what this whole thing is about ... and that's a very false impression of what the "music" that we call progressive is all about ... and something that hurts over all the ability of folks to hear something new.

It's the same with classical music ... you have heard Beethoven, Mozart, Bach ... but you can hardly mention anyone from today ... specially since you do not consider any of the folks in your charts "composers" that measure up, which some bands deserve to, and the modern model is a "band" and not a single composer. (Ex: Check Rachel Flowers doing piano on Tarkus ... imagine Keith trying to show his mates the piece!)

Adjusting the numbers in your charts, would probably give a better idea of the realities in each year ... and how some of the music was REALLY enjoyed/addressed, that your numbers do not reflect. I find it hard to believe that a Peter Hammill or Van der Graaf Generator album got a lesser rating than Yes, or KC album that was released at least 10 years before ... that does not tell you anything about PH or VdGG, and the difference shows the commercial potential of the known material opposite the lesser known material.

I would like to see the adjustments made, however I am not sure it can be done, since I am not sure that there is a "date" for when the values were set ... but something that came out in 1972, should only be able to get a 5 on that year ... and not the same 5 in other years ... would make a lot more sense ... but you're telling me that Beethoven's 9th is still better than YES and KC, because it's still better known than both of those, and studied in school a lot more! And I would immediately say ... you are behind the times, and your study is imcomplete and not paying attention to its particular/specific date when it should apply, instead of a general ideal, that supposedly is what we all are ... and this is the problem with Sociology 101 and Psychology 101 at ANY university out there ... the generalizations are ... ridiculous and we accept them as truth.

Are we that smart?
... none of the hits, none of the time ... now you know what the art is all about!
www.pedrosena.com
Back to Top
Lewian View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: August 09 2015
Location: London
Status: Offline
Points: 1329
Post Options Post Options   Thanks (0) Thanks(0)   Quote Lewian Quote  Post ReplyReply Direct Link To This Post Posted: December 24 2017 at 17:24
Beethoven's 9th is bloody good though. Wink
Back to Top
King of Loss View Drop Down
Prog Reviewer
Prog Reviewer


Joined: April 21 2005
Location: Watching You
Status: Offline
Points: 14594
Post Options Post Options   Thanks (0) Thanks(0)   Quote King of Loss Quote  Post ReplyReply Direct Link To This Post Posted: December 26 2017 at 01:51
Very cool.Thumbs Up

Zizou 1988-2006
Back to Top
Frenetic Zetetic View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 09 2017
Location: New Hampshire
Status: Offline
Points: 2078
Post Options Post Options   Thanks (0) Thanks(0)   Quote Frenetic Zetetic Quote  Post ReplyReply Direct Link To This Post Posted: December 27 2017 at 11:04
This is REALLY cool to see!

Pythagoras with the looking glass reflects the full moon...
In blood, he's writing the lyrics of a brand-new tune...
Back to Top
moshkito View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: January 04 2007
Location: Grok City
Status: Offline
Points: 7532
Post Options Post Options   Thanks (0) Thanks(0)   Quote moshkito Quote  Post ReplyReply Direct Link To This Post Posted: January 02 2018 at 21:47
Hi,

There was a book, a few years back, done by some BBC folks, and it was a VERY incendiary bunch of ideas and thoughts and it kinda went through European History backwards ... until it reached a conclusion that ruined a lot of our childish stories and childish beliefs.

Essentially, a book was written some 300 years later by folks that "were not there" and were relying on whatever they could find, including stories and more stories, and a few "books" that were available, HALF OF WHICH were ignored and even buried and destroyed because they did not help the Disney story any more ... and as one other writer mentioned, it's hard to believe, that we believe some news reporting 300 years late, and how untimely it was, and how well the bottle made it with the message in one piece.

This graph, to me, is similar ... not the same thing ... but very similar. It is slanted to views that are from a different time, and by people that did not exactly listen, or hear, a whole lot of other things at that time, except their "favorites" ... and that becomes, again, a story that becomes fabricated, and badly translated ... into another of those Disney like stories that we seem to love so much ... even when they are taken/stolen from many other European writers ... that no one reads or cares to learn about!

Keep on Crinting ... fantasy is good for your soul, I guess! But calling it "progressive"?


Edited by moshkito - January 02 2018 at 21:48
... none of the hits, none of the time ... now you know what the art is all about!
www.pedrosena.com
Back to Top
Formentera Lady View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: August 20 2010
Location: Germany
Status: Offline
Points: 1662
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formentera Lady Quote  Post ReplyReply Direct Link To This Post Posted: January 14 2018 at 11:03
Great work! Clap
Really no surprises here, but it is nice to see in graphics what I always thought:
A peak in the 70's, the 'dark ages of prog' in the 80's (with the birth of neo-prog and Marillion etc.), and a rebirth of prog (I call it third wave) since the 90's with a continuing constant few but high-quality output until today.

Back to Top
Formentera Lady View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: August 20 2010
Location: Germany
Status: Offline
Points: 1662
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formentera Lady Quote  Post ReplyReply Direct Link To This Post Posted: January 14 2018 at 11:14
Originally posted by omphaloskepsis omphaloskepsis wrote:

I found the ratings per year chart disturbing as it pertains to Prog Archives.   Downhill from 2013, even though the last few years have been banner years for prog albums.
Could it be that the " ratings per year" chart suggests Prog Archives is dying?   
Before anyone assumes I jumped to an insane conclusion ,  I did say "suggests".  


This is indeed a disturbing observation, now that I look closely at the end of the graphs. There is also this 'I am not a robot' thing that probably reduces participation...

Back to Top
moshkito View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: January 04 2007
Location: Grok City
Status: Offline
Points: 7532
Post Options Post Options   Thanks (0) Thanks(0)   Quote moshkito Quote  Post ReplyReply Direct Link To This Post Posted: January 15 2018 at 10:41
Originally posted by Formentera Lady Formentera Lady wrote:

...
This is indeed a disturbing observation, now that I look closely at the end of the graphs. There is also this 'I am not a robot' thing that probably reduces participation...


If the charts were adjusted for the time and place, these observations would normalize a lot more ... again, all they are saying is that more people are here and that some of them are not kissing the top 3 as much as they are enjoying their own listens, and that is ... AS SHOULD BE.

Think of it this way ... Beethoven and Mozart are gone, and Stravinsky just came up ... so you gonna say that Strav is crap because you like the gods from before?

This is the reason why I suggest that you can only vote a top number for one album for the year that is was come up ... but if you first showed up in 2005 and immediately voted "Close to the Edge" number one, your vote would not have had as much weight as it would if you had voted in 1972 ... and this would have given the locals, and more recent listeners a LOUDER voice, with newer material.

As it is right now, there is a rather odd ball thing ... basically no one "today" should be able to vote a 5 on any of Genesis/Yes or ELP work ... and if the numbers (quantity) still show that the band stands up to the newer folks of the day, then I would say that they are much TRUE'r than right now ... but that the previous big names and favorites have lost a little appreciation, should be accepted as par for the course ... but we need to learn to appreciate the newer materials a LOT better than we do at this time.

The internet has been great for all newer material, but it is creating a fantasy about the old stuff that is getting rather pathetic and boring. PF was pretty good, but not the best and certainly not the only one! There were many, many, many, many others out there, and you never gave Guru Guru, Amon Duul 2, Ange, Banco, PFM and many others a good enough listen to understand and appreciate what they did ... for example, in my book, Guru Guru in the first 4 albums was everything that you wanted to hear Jimi do that you never got a chance to hear. Ax's guitar work is more Jimi than Jimi in so many ways ... and the folks that own Jimi's music, are still hiding and not listening to the experimental and crazy stuff that Jimi did, that many Europeans took to heart ... but American Corporate Radio made a point of killing ... perfect example was Grateful Dead, known for their long cuts and extended everything ... not a single one of these on albums!

And folks here, still have no idea where so much of that "progressive" and "experimental" edge came from ... and the reason why so many bands in Germany were thanking the Californians for their spirit! It wasn't just ... 8 Miles High or the 20 minute version by Golden Earring!
... none of the hits, none of the time ... now you know what the art is all about!
www.pedrosena.com
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 11.01
Copyright ©2001-2014 Web Wiz Ltd.

This page was generated in 0.767 seconds.