Forum Home Forum Home > Site News, Newbies, Help and Improvements > Help us improve the site
  New Posts New Posts RSS Feed - This sort of thing annoy me
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

This sort of thing annoy me

 Post Reply Post Reply
Author
Message
Saperlipopette! View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 20 2010
Location: Tomorrowland
Status: Offline
Points: 10052
Post Options Post Options   Thanks (0) Thanks(0)   Quote Saperlipopette! Quote  Post ReplyReply Direct Link To This Post Topic: This sort of thing annoy me
    Posted: April 18 2023 at 07:28
Can it be fixed?

No one in their right mind can think this seems about right. No matter how you look at it the "logic" to the algorithm/QWR PA uses, is clearly flawed.


Back to Top
chopper View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: July 13 2005
Location: Essex, UK
Status: Offline
Points: 19945
Post Options Post Options   Thanks (1) Thanks(1)   Quote chopper Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 07:41
I don't pretend to know how that is calculated but it does seem a bit bizarre. The Framus 5 album has no reviews. I think there may be a higher weighting to collab reviews/ratings but it only has 1 collab rating and that's a 2 star one.
The Framus 5 album has 45% 2 star ratings and Tull have 50% 3 star. Nope, I don't get it but I'm not going to lose any sleep over it.
Back to Top
JD View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: February 07 2009
Location: Canada
Status: Offline
Points: 18373
Post Options Post Options   Thanks (0) Thanks(0)   Quote JD Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 08:01
What the h3ll is QWR ?
Thank you for supporting independently produced music
Back to Top
Cristi View Drop Down
Special Collaborator
Special Collaborator
Avatar
Crossover / Prog Metal Teams

Joined: July 27 2006
Location: wonderland
Status: Offline
Points: 41360
Post Options Post Options   Thanks (0) Thanks(0)   Quote Cristi Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 08:02
Can't say this bothers me, yeah, something went wrong there, but it's just a list after all... 
Back to Top
Saperlipopette! View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 20 2010
Location: Tomorrowland
Status: Offline
Points: 10052
Post Options Post Options   Thanks (0) Thanks(0)   Quote Saperlipopette! Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 08:04
Originally posted by chopper chopper wrote:

I don't pretend to know how that is calculated but it does seem a bit bizarre. The Framus 5 album has no reviews. I think there may be a higher weighting to collab reviews/ratings but it only has 1 collab rating and that's a 2 star one.
The Framus 5 album has 45% 2 star ratings and Tull have 50% 3 star. Nope, I don't get it but I'm not going to lose any sleep over it.
I don't lose any sleep over it myself. But I don't like things that are... wrong. When I stumble over extremes such as this example (which appears every time I look at a specific year, genre etc...) it just irritates me. Ca. on the level of a headline in the newspaper with spelling mistakes.
Back to Top
Saperlipopette! View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 20 2010
Location: Tomorrowland
Status: Offline
Points: 10052
Post Options Post Options   Thanks (0) Thanks(0)   Quote Saperlipopette! Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 12:26
Originally posted by Cristi Cristi wrote:

Can't say this bothers me, yeah, something went wrong there, but it's just a list after all... 
Nothing went wrong there. The algorithm works this way. Similar things occur everywhere and every time, if one uses the lists like I do - to have a closer look in a specific year or a genre.
Back to Top
Cristi View Drop Down
Special Collaborator
Special Collaborator
Avatar
Crossover / Prog Metal Teams

Joined: July 27 2006
Location: wonderland
Status: Offline
Points: 41360
Post Options Post Options   Thanks (0) Thanks(0)   Quote Cristi Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 12:29
Originally posted by Saperlipopette! Saperlipopette! wrote:

Originally posted by Cristi Cristi wrote:

Can't say this bothers me, yeah, something went wrong there, but it's just a list after all... 
Nothing went wrong there. The algorithm works this way. Similar things occur everywhere and every time, if one uses the lists like I do - to have a closer look in a specific year or a genre.

Oh, i wasn't blaming you, but whatever algorithm used there. 
Back to Top
Saperlipopette! View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 20 2010
Location: Tomorrowland
Status: Offline
Points: 10052
Post Options Post Options   Thanks (0) Thanks(0)   Quote Saperlipopette! Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 12:32
Originally posted by Cristi Cristi wrote:

Originally posted by Saperlipopette! Saperlipopette! wrote:

Originally posted by Cristi Cristi wrote:

Can't say this bothers me, yeah, something went wrong there, but it's just a list after all... 
Nothing went wrong there. The algorithm works this way. Similar things occur everywhere and every time, if one uses the lists like I do - to have a closer look in a specific year or a genre.

Oh, i wasn't blaming you, but whatever algorithm used there. 
No, I didn't think you were blaming me. It's just that according to the algorithm used, this is correct. So nothing went wrong.
Back to Top
Cristi View Drop Down
Special Collaborator
Special Collaborator
Avatar
Crossover / Prog Metal Teams

Joined: July 27 2006
Location: wonderland
Status: Offline
Points: 41360
Post Options Post Options   Thanks (0) Thanks(0)   Quote Cristi Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 12:33
Originally posted by Saperlipopette! Saperlipopette! wrote:

Originally posted by Cristi Cristi wrote:

Originally posted by Saperlipopette! Saperlipopette! wrote:

Originally posted by Cristi Cristi wrote:

Can't say this bothers me, yeah, something went wrong there, but it's just a list after all... 
Nothing went wrong there. The algorithm works this way. Similar things occur everywhere and every time, if one uses the lists like I do - to have a closer look in a specific year or a genre.

Oh, i wasn't blaming you, but whatever algorithm used there. 
No, I didn't think you were blaming me. It's just that according to the algorithm used, this is correct. So nothing went wrong.

Thumbs Up
Back to Top
Lewian View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: August 09 2015
Location: Italy
Status: Offline
Points: 14110
Post Options Post Options   Thanks (0) Thanks(0)   Quote Lewian Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 14:54
The formula is explained here:

The implication is that all albums are scored relative to the query average. If an album has many ratings, those ratings will strongly determine its QWR. If an album has few ratings, the QWR will be close to the query average. This is based on the somewhat strange but not totally bonkers idea that everybody who didn't rate the album (but would've rated an album with average number of ratings) would have rated it at query average. In statistics this is a "quick and dirty" method for missing values imputation; many people know that this can be bad and is better avoided, but sometimes they don't take the time and make the effort to come up with something better.

In particular, for albums above the query average, it is good to have many ratings because only in this way they can rise clearly above the query average. However, for albums below the query average, it is better to have fewer ratings, because this means that their QWR will basically be equal to the query average, whereas a large number of ratings cements their lower average status.

In the posted list by Saperlipopette! the query average seems to be about 3.36. The Framus 5 album has QWR very close to this because it has only 11 ratings, which can't drag it down much. The Jethro Tull album has the largest number of ratings, which gives its actual average rating a very high weight, so its QWR is close to it's actual average of 3.31.   

In my view this idea is somewhat reasonable above the average but makes very little sense below it. Chances are the people who came up with it thought that anybody will mainly be interested in what goes on in the higher ranks, i.e., above the average, anyway.

I should also say that I'm not quite sure how the query average is determined (which is said to depend on the actual query), as there is a default (next to last filter) to only include albums with rating above query average, which would actually require to compute the query average before applying this filter (and maybe others!?), but naively one should think that the query is defined by the filters. (Running some small experiments it seems that the last four numerical filters are ignored, and everything that fulfills the other filters defines the query average. This is probably reasonable.)


Edited by Lewian - April 19 2023 at 03:25
Back to Top
Saperlipopette! View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 20 2010
Location: Tomorrowland
Status: Offline
Points: 10052
Post Options Post Options   Thanks (0) Thanks(0)   Quote Saperlipopette! Quote  Post ReplyReply Direct Link To This Post Posted: April 18 2023 at 23:30
^Thanks and I'm sorry you had to spend time writing that post explaining how it works. I sort of know the basics and "logic" behind it. I suppose it does the job for the all time top 100 and maybe even top 250. But as someone more likely to use the charts to locate albums in 1967 or 1968... or in Indo/Raga etc... that I'm unaware of (and therefore often exposed to below average, lower half of lists), it comes up with absurdities like this all the time.
Originally posted by Lewian Lewian wrote:

Chances are the people who came up with it thought that anybody will mainly be interested in what goes on in the higher ranks, i.e., above the average, anyway.

Yes, they are geeks, but not music geeks. One thing is cheering for the underdog, but there is something not right with an algorithm that places an album that six people rated and "nobody liked", ten spots above a relatively treasured semiclassic enjoyed by most - such as This Was.


 
Back to Top
Saperlipopette! View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 20 2010
Location: Tomorrowland
Status: Offline
Points: 10052
Post Options Post Options   Thanks (0) Thanks(0)   Quote Saperlipopette! Quote  Post ReplyReply Direct Link To This Post Posted: April 19 2023 at 01:34
^Maybe semiclassic is a bit of a stretch, but a very promising start.

Edit: Long, totally off topic digression (which I found interesting)

Btw: I actually gave Don Ellis - Shock Treatment a listen and it's a weirdly interesting album. A mixerd bag, not bad, but nothing there for someone looking for any kind of prog - or rock.

...Columbia Records created confusion surrounding Shock Treatment by releasing multiple versions of the recording. Ellis attempted to clarify the situation by sending a letter to the “Chords and Discords” forum of Down Beat magazine immediately following the magazine’s review of the album:

“Regarding the review of my record Shock Treatment by Harvey Pekar (DB, Sept. 19), I would like to set the record straight on some little known facts in connection with this album. The copy that was reviewed was one about which I am embarrassed and not proud. The story behind this is as follows:

Upon completion of the album, I did the mixing and editing here in California and then sent the finished product to New York. It wasn’t until the album was already released that I heard a pressing. Much to my horror, I found that without consulting me the whole album had been changed around—rejected masters and unapproved takes were used (not the ones which I had selected and edited), the wrong tunes were on the album, unauthorized splices were made which disturbed the musical flow of some of the compositions (beats were even missing from bars), whole sections were cut out, some of these being the high points of the album. Therefore the liner notes, which were done to the original album, do not agree with what is actually on the album, calling attention to solos and high spots which are not there. I’m surprised that this wasn’t mentioned in the review! Also, the wrong personnel is listed on the jacket. When I discovered what had happened I was, naturally, disturbed and asked Columbia to redo the album. They graciously consented and I was able to change the album back to its original form except that I left Mercy Maybe Mercy, which my producer particularly liked, in place of Zim, which I hope will appear in a future album. Unfortunately, they were not able to call back all the thousands of albums which had already been released. However, they did send a note to the reviewers telling them that the copy which they had received was defective, and to please not review it until they received the corrected copy. It looks as if Down Beat didn’t get that letter. In conclusion, let me state that I have no quarrel with Harvey’s review, but I do wish that he or someone else would review the correct album.”..

-The version I listened to (on Spotify) was a brassy (not my bag), groovin' and spiritual jazz (my bag) with a full choir on half the album kind of thing. The latter aren't mentioned in any of the versions I could find. So who knows what people have been listening to.


Edited by Saperlipopette! - April 19 2023 at 21:46
Back to Top
Lewian View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: August 09 2015
Location: Italy
Status: Offline
Points: 14110
Post Options Post Options   Thanks (0) Thanks(0)   Quote Lewian Quote  Post ReplyReply Direct Link To This Post Posted: April 19 2023 at 03:12
Originally posted by Saperlipopette! Saperlipopette! wrote:

^Thanks and I'm sorry you had to spend time writing that post explaining how it works. 

Don't mind, I wouldn't put in the time were I not interested myself.
I also tried to set myself the task to come up with a filter choice that can put This Was above Framus 5, but I was sabotaged by the system as it produces errors when I combine too complex combinations (like multiple choices in several categories). The secret is to put together filters in such a way that the query average is 3.31 or lower and still it includes both of these albums. If anyone finds a possibility, that'd be really prizeworthy.


Edited by Lewian - April 19 2023 at 03:15
Back to Top
octopus-4 View Drop Down
Special Collaborator
Special Collaborator
Avatar
RIO/Avant/Zeuhl,Neo & Post/Math Teams

Joined: October 31 2006
Location: Italy
Status: Offline
Points: 13389
Post Options Post Options   Thanks (0) Thanks(0)   Quote octopus-4 Quote  Post ReplyReply Direct Link To This Post Posted: April 19 2023 at 08:10

Is Equipe 84 on PA? Confused
Curiosity killed a cat, Schroedinger only half.
My poor home recorded stuff at https://yellingxoanon.bandcamp.com
Back to Top
Cristi View Drop Down
Special Collaborator
Special Collaborator
Avatar
Crossover / Prog Metal Teams

Joined: July 27 2006
Location: wonderland
Status: Offline
Points: 41360
Post Options Post Options   Thanks (0) Thanks(0)   Quote Cristi Quote  Post ReplyReply Direct Link To This Post Posted: April 19 2023 at 08:11
Originally posted by octopus-4 octopus-4 wrote:


Is Equipe 84 on PA? Confused

I don't know this band, but if they are on a PA list like that one, then yes, they are on PA. Geek
Back to Top
chopper View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: July 13 2005
Location: Essex, UK
Status: Offline
Points: 19945
Post Options Post Options   Thanks (0) Thanks(0)   Quote chopper Quote  Post ReplyReply Direct Link To This Post Posted: April 19 2023 at 09:05
Originally posted by octopus-4 octopus-4 wrote:


Is Equipe 84 on PA? Confused

Back to Top
Saperlipopette! View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: December 20 2010
Location: Tomorrowland
Status: Offline
Points: 10052
Post Options Post Options   Thanks (0) Thanks(0)   Quote Saperlipopette! Quote  Post ReplyReply Direct Link To This Post Posted: April 23 2023 at 23:15
Originally posted by Lewian Lewian wrote:

Originally posted by Saperlipopette! Saperlipopette! wrote:

^Thanks and I'm sorry you had to spend time writing that post explaining how it works. 

Don't mind, I wouldn't put in the time were I not interested myself.
I also tried to set myself the task to come up with a filter choice that can put This Was above Framus 5, but I was sabotaged by the system as it produces errors when I combine too complex combinations (like multiple choices in several categories). The secret is to put together filters in such a way that the query average is 3.31 or lower and still it includes both of these albums. If anyone finds a possibility, that'd be really prizeworthy.
I have no idea. I think this whole algorithm should be scrapped and replaced with one that produces fewer errors, is overall more logic, truthful or "realistic"- in both ends of the charts. 
Back to Top
Lewian View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: August 09 2015
Location: Italy
Status: Offline
Points: 14110
Post Options Post Options   Thanks (0) Thanks(0)   Quote Lewian Quote  Post ReplyReply Direct Link To This Post Posted: April 24 2023 at 04:50
I won't disagree, but chances are this is hopeless as I suspect nobody active can change this.

In any case, it isn't trivial to come up with a better formula. The way to do it would be to first collect some "formal axioms", i.e., specify what a good formula should fulfill. Based on this one can design one that does it. Can you define what's wrong? Is the only thing you'd like to have fixed that an album with better average and more ratings should always be better than an album with fewer ratings and lower average? This obviously currently doesn't happen below the query average. Fixing just this would probably be relatively simple, even though it may run counter to the basic concept of the current formula. Of course one needs to take care to not introduce other issues if this one is fixed. (I also wonder whether anybody who is interested would argue in favour of the current system telling you that what actually happens is in some sense better than giving more ratings always a positive influence. I wonder whether there is consensus that you are right - maybe not? Obviously the problem isn't me...)
Back to Top
wiz_d_kidd View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: January 13 2018
Location: EllicottCityMD
Status: Offline
Points: 1360
Post Options Post Options   Thanks (0) Thanks(0)   Quote wiz_d_kidd Quote  Post ReplyReply Direct Link To This Post Posted: April 24 2023 at 07:42
I agree with Lewian that it is not trivial to come up with a better formula. I've done some extensive analysis of the existing one which you can find here. It basically says that "all albums are ranked as average unless they obtain a sufficient number of votes to pull them up or down, away from the average". It's a reasonable criteria. However, I think the sticking point is that it can take a whopping number of votes to influence the ranking, if there are other albums in the mix that already have a huge number of votes. But there's nothing we can do about any of this because it's unlikely anyone will come up with a better formula, and even more unlikely that M@X will overhaul the site to implement it.
“I don’t like country music, but I don’t mean to denigrate those who do. And for those who like country music, denigrate means to ‘put down.'” – Bob Newhart
Back to Top
Logan View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Site Admin

Joined: April 05 2006
Location: @ wicker man
Status: Offline
Points: 32710
Post Options Post Options   Thanks (0) Thanks(0)   Quote Logan Quote  Post ReplyReply Direct Link To This Post Posted: April 24 2023 at 23:40
It certainly is well above my page grade, which remains a s steady nought, looks weird but isn't wrong. As said, coming up with a better formula is not easy, and a change would be up to M@x to decide on and implement. It has come up various times, and Dean too went into great detail on it and why, as I recall, he thought what we use is acceptable (he tried many formulae).

I wish we could use multiple algorithms depending on preference, but that would be quite the change and is not something I would expect. I'm always dreaming about what might be; some of my wishes are much grander and totally preposterous than others.
Just a fanboy passin' through.
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.133 seconds.
Donate monthly and keep PA fast-loading and ad-free forever.