A general survey about P.A. and RYM |
Post Reply | Page <1234 8> |
Author | ||||||||||||||||||||||||||||
Dean
Special Collaborator Retired Admin and Amateur Layabout Joined: May 13 2007 Location: Europe Status: Offline Points: 37575 |
Posted: January 30 2015 at 08:10 | |||||||||||||||||||||||||||
...all that aside and ...
...an interesting article and quite in depth considering the authors' analysis was conducted from outside the system as it were. We can never eradicate fan-biased attempts at album ranking manipulation from a system where anyone can submit a rating and/or review, but from my lengthy and extensive experience in dealing with manipulation and alleged cases of manipulation I don't believe that it is as harmful as many people think it is given the measures we have to contain and regulate it. During my time as an Admin I was as much concerned by the over-reaction to it as I was by the deliberate manipulation itself. Unfortunately, dealing with the perception that fan-bias is detrimental to an album is a lot harder.
|
||||||||||||||||||||||||||||
What?
|
||||||||||||||||||||||||||||
Guldbamsen
Special Collaborator Retired Admin Joined: January 22 2009 Location: Magic Theatre Status: Offline Points: 23098 |
Posted: January 30 2015 at 08:14 | |||||||||||||||||||||||||||
I agree a 100% with that last bit Dean
|
||||||||||||||||||||||||||||
“The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.”
- Douglas Adams |
||||||||||||||||||||||||||||
HolyMoly
Special Collaborator Retired Admin Joined: April 01 2009 Location: Atlanta Status: Offline Points: 26133 |
Posted: January 30 2015 at 08:23 | |||||||||||||||||||||||||||
I only skimmed the article, but it appears to be a well reasoned analysis, touching on important features that can fairly assess the value of a site. I didn't see anything at first that I really disagreed with. I definitely empathized with the constructive criticism that PA can be a bit confusing to the first time visitor - because I remember how long it took me to "get" this site. Its design is not optimal and finding stuff can be difficult. I'm no web designer so I don't know how I would improve it, but as a user I can relate. Because of this, I've always tried to give newbies the benefit of the doubt when they post things in the wrong forum or post questionable reviews (for example, when I first started writing reviews, I didn't know that anyone would actually read it unless they were specifically searching for reviews by that artist... I didn't know about the "front page").
PA does have all its bases covered: detailed writeups of policies, guidelines, genre definitions, etc, and all the links work and so on... but it's not the most user-friendly site I've come across, and it does take a while to learn the ropes. Not sure how RYM sizes up in this regard because I don't use that site. Edited by HolyMoly - January 30 2015 at 08:28 |
||||||||||||||||||||||||||||
My other avatar is a Porsche
It is easier for a camel to pass through the eye of a needle if it is lightly greased. -Kehlog Albran |
||||||||||||||||||||||||||||
Rednight
Forum Senior Member Joined: January 18 2014 Location: Mar Vista, CA Status: Offline Points: 4807 |
Posted: January 30 2015 at 10:35 | |||||||||||||||||||||||||||
"Zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz ... "
Edited by Rednight - January 30 2015 at 10:39 |
||||||||||||||||||||||||||||
earlyprog
Collaborator Neo / PSIKE / Heavy Teams Joined: March 05 2006 Location: . Status: Offline Points: 2086 |
Posted: January 30 2015 at 11:59 | |||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
Dean
Special Collaborator Retired Admin and Amateur Layabout Joined: May 13 2007 Location: Europe Status: Offline Points: 37575 |
Posted: January 30 2015 at 12:12 | |||||||||||||||||||||||||||
Why this instead of the more informative post you subsequently deleted? It is relevant for many reasons, least of all is the pedestrian speed in which things happen around here. 2008 is like yesterday in PA years, and frankly, aside from a minor change in the collaborator weighting nothing much else has changed in rating and reviewing system, or the manner in which fan-bias manipulation occurs, or the look and feel of the site in general since 2008.
|
||||||||||||||||||||||||||||
What?
|
||||||||||||||||||||||||||||
Angelo
Special Collaborator Honorary Collaborator / Retired Admin Joined: May 07 2006 Location: Italy Status: Offline Points: 13239 |
Posted: January 30 2015 at 12:15 | |||||||||||||||||||||||||||
Oh yes... only this afternoon I came across 3 screens full of calculations in a single post by you. The math behind the rating/review balance was your thing in this case.
|
||||||||||||||||||||||||||||
ISKC Rock Radio
I stopped blogging and reviewing - so won't be handling requests. Promo's for ariplay can be sent to [email protected] |
||||||||||||||||||||||||||||
Angelo
Special Collaborator Honorary Collaborator / Retired Admin Joined: May 07 2006 Location: Italy Status: Offline Points: 13239 |
Posted: January 30 2015 at 12:16 | |||||||||||||||||||||||||||
Takes getting used to as well - as far as user-friendliness is concerned there's a lot to gain on music sites.
|
||||||||||||||||||||||||||||
ISKC Rock Radio
I stopped blogging and reviewing - so won't be handling requests. Promo's for ariplay can be sent to [email protected] |
||||||||||||||||||||||||||||
SteveG
Forum Senior Member Joined: April 11 2014 Location: Kyiv In Spirit Status: Offline Points: 20497 |
Posted: January 30 2015 at 12:17 | |||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
This message was brought to you by a proud supporter of the Deep State.
|
||||||||||||||||||||||||||||
Nogbad_The_Bad
Forum & Site Admin Group RIO/Avant/Zeuhl & Eclectic Team Joined: March 16 2007 Location: Boston Status: Online Points: 20199 |
Posted: January 30 2015 at 12:33 | |||||||||||||||||||||||||||
The line between Krautrock & RPI runs through the the Alps while Canterbury has a 30 mile circle around it in Kent.
|
||||||||||||||||||||||||||||
Ian
Host of the Post-Avant Jazzcore Happy Hour on Progrock.com https://podcasts.progrock.com/post-avant-jazzcore-happy-hour/ |
||||||||||||||||||||||||||||
earlyprog
Collaborator Neo / PSIKE / Heavy Teams Joined: March 05 2006 Location: . Status: Offline Points: 2086 |
Posted: January 31 2015 at 06:40 | |||||||||||||||||||||||||||
I deleted all except what I felt was the core of my post because it was unsubstantiated and would have required more effort on my part to pass on my thoughts in order to contribute in a constructive and valuable manner to this - yes - interesting topic.
|
||||||||||||||||||||||||||||
earlyprog
Collaborator Neo / PSIKE / Heavy Teams Joined: March 05 2006 Location: . Status: Offline Points: 2086 |
Posted: January 31 2015 at 07:35 | |||||||||||||||||||||||||||
A minor change im the weighting factors of various reviewer groups can have a profound effect on the overall album ranking. Over the years I have seen sudden, abrupt changes in top 100 which can only be attributed to changes in the weights. I assume and accept that the PA team has favorable weightings in the ranking process after all this is one way to thank them for their efforts. Fans and haters are found in tail regions of an album's rating distribution and can be (are?) weighted disfavorably. The rating and reviewing system of PA is not transparent on this, at least not to me, but then I haven't searched this site in an attempt to find the weighting factors. Edit: one can argue that fan boys and haters cancel out in the statistics but often the former tend to be a larger group than the latter. In a more recent case, dare I say Corvus Stone II, the ranking appeared to be suspiciously high in the beginning (in my view*) but then, as always, levelled out, as the collaborators' reviews started to clock in. Here there's always the risk that the 'higher weighted reviewers' abuse their weighting to rate an album a few notches lower (or higher) in order to 'restore order' in their (PA) universe. *Perhaps I just didn't 'get' the music, but repeated listening convinced me that this emperor in fact did not wear any clothes (someone just had to say so before it was realised by all).
Edited by earlyprog - January 31 2015 at 12:55 |
||||||||||||||||||||||||||||
Dean
Special Collaborator Retired Admin and Amateur Layabout Joined: May 13 2007 Location: Europe Status: Offline Points: 37575 |
Posted: January 31 2015 at 09:32 | |||||||||||||||||||||||||||
As the article stated, early votes and reviews for an album tend to come from people who like the album. This is perfectly reasonable since if an album doesn't move you then you will be less motivated to write anything about it. From my analysis of the rating trend with time of several different albums I have concluded that you cannot make any valid judgement from an album average until there are at least 30 ratings and it can take up to 100 ratings before the trend starts to "level-out". Personally I would not include an album with less than 50 ratings in any chart. As I have said many times, the best indicator of a bands true rating is not the average, but shape of the rating distribution (the article also used this as an indicator), which does not take into account any reviewer/collab weighting:
In the above example you can see that the distribution is neither a normal distribution nor a power-law distribution but a combination of the two. This is because the album has been rated by two distinct groups of people - band-fans and (ambivalent) Prog-fans, but (interestingly) not by haters. So from this you can conclude that if you are a fan of the band then you'll love the album (5-stars), but if you an average Prog fan you think it is good, but not great (3-stars).
Edited by Dean - January 31 2015 at 09:38 |
||||||||||||||||||||||||||||
What?
|
||||||||||||||||||||||||||||
SteveG
Forum Senior Member Joined: April 11 2014 Location: Kyiv In Spirit Status: Offline Points: 20497 |
Posted: January 31 2015 at 09:38 | |||||||||||||||||||||||||||
^I think the problem with this ratio, Dean, is that there is a fine line between an album being "good but not essential." I cannot come up with a more suitable wording for this 3 star rating. Perhaps someone else can.
|
||||||||||||||||||||||||||||
This message was brought to you by a proud supporter of the Deep State.
|
||||||||||||||||||||||||||||
Dean
Special Collaborator Retired Admin and Amateur Layabout Joined: May 13 2007 Location: Europe Status: Offline Points: 37575 |
Posted: January 31 2015 at 09:47 | |||||||||||||||||||||||||||
The wording of all five rating options has been a bone of contention since the year dot. The problem arises from 3-star rating being the median, which most non-mathematicians would also regard as being the average (i.e., mean) value. In a biased system the median will not be the mean value, which is why the QWR (Query Weighted Rating) uses a value for the site-average that is higher than "3". Simply put, a 3-star album is a less than average album.
|
||||||||||||||||||||||||||||
What?
|
||||||||||||||||||||||||||||
SteveG
Forum Senior Member Joined: April 11 2014 Location: Kyiv In Spirit Status: Offline Points: 20497 |
Posted: January 31 2015 at 10:04 | |||||||||||||||||||||||||||
^Agreed, but perhaps classifying a 3 star album as 'decent' instead of 'good' would help. Perhaps not, as all subjective opinions are subject to wide interpretation.
|
||||||||||||||||||||||||||||
This message was brought to you by a proud supporter of the Deep State.
|
||||||||||||||||||||||||||||
Angelo
Special Collaborator Honorary Collaborator / Retired Admin Joined: May 07 2006 Location: Italy Status: Offline Points: 13239 |
Posted: January 31 2015 at 10:24 | |||||||||||||||||||||||||||
Hmmmm... so the mathematical definition is completely different from what people see based on the names of the stars. I've read the phrase 'a 3-star album is not a bad album' a couple of times this week, which is not what you say here. Glad you're on the math of this, Dean, I'd go crazy straight away.
|
||||||||||||||||||||||||||||
ISKC Rock Radio
I stopped blogging and reviewing - so won't be handling requests. Promo's for ariplay can be sent to [email protected] |
||||||||||||||||||||||||||||
SteveG
Forum Senior Member Joined: April 11 2014 Location: Kyiv In Spirit Status: Offline Points: 20497 |
Posted: January 31 2015 at 10:36 | |||||||||||||||||||||||||||
^Come on, Angelo. It's only math rock.
|
||||||||||||||||||||||||||||
This message was brought to you by a proud supporter of the Deep State.
|
||||||||||||||||||||||||||||
Dean
Special Collaborator Retired Admin and Amateur Layabout Joined: May 13 2007 Location: Europe Status: Offline Points: 37575 |
Posted: January 31 2015 at 11:05 | |||||||||||||||||||||||||||
Actually, the phrases "less than average" and "not a bad album" are not mutually exclusive since "average" is not a good/bad threshold. As Steve says, it's subjective, and with that it is also relative. A 5-star album is better than a 4-star album, and both are better than a 3-star album. All of three of them can be good albums: it is simply that the 3-star album isn't as good as the 4-star album; just as the 4-star album isn't as good as the 5-star album. When we take all the ratings cast by all the members of the site and calculate the average rating that we give to all the albums on the site it comes out at somewhere around 3.75. Basically, even though we have a 5-star system we concentrate most of our votes on 3, 4, and 5-star ratings. As the article says, our ratings follow a power-law distribution. So the 3-star median point is less than the 3.75-star mean point and any album that scores less than 3.75 average is a "less than average" album by our standards, and that can also be called "not a bad album". |
||||||||||||||||||||||||||||
What?
|
||||||||||||||||||||||||||||
Gerinski
Prog Reviewer Joined: February 10 2010 Location: Barcelona Spain Status: Offline Points: 5093 |
Posted: January 31 2015 at 11:40 | |||||||||||||||||||||||||||
^ that's also because, happily, 1-star stuff has less chance of being released and purchased by somebody than 5-star stuff, and 2-star stuff less chance than 4-star stuff, so it's only normal that the average of all released and rated stuff is higher than the median value 3. Otherwise it would mean that there's a lot of crap being released and purchased by people who can then rate it.
|
||||||||||||||||||||||||||||
Post Reply | Page <1234 8> |
Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |