Author |
Topic Search Topic Options
|
Easy Livin
Special Collaborator
Honorary Collaborator / Retired Admin
Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
|
Topic: Ratings for Reviews - A Cure for Abuse? Posted: March 25 2006 at 05:10 |
Hi Joolz,
Of course you're not being presumptuous! You're views are considered, valid, and very welcome!
Welcome to the site.
|
 |
Joolz
Special Collaborator
Honorary Collaborator
Joined: March 24 2006
Location: United Kingdom
Status: Offline
Points: 1377
|
Posted: March 24 2006 at 17:37 |
Hi
I am a new member, and will get around to introducing myself properly shortly. I have been a prog fan since the late sixties and have often had a look at this site. It contains a lot of fascinating stuff and is obviously a real haven.
While I love the fact that anyone can post reviews, and look forward to writing a few of my own, I have grave concerns about the rating system. It soon struck me that people were often rating based purely on whether they happen to like something or not. Your definition of 5-stars is "Essential: a masterpiece of progressive music" - to my mind this can only apply to 'a few'. So I am pleased to find this forum discussing this very problem.
You guys have obviously lived with this for a while, and have it given longer thought than I have, but for what it is worth, I do find some of the proposals somewhat complicated and am not keen on the idea of rating the review or reviewer.
How about a system whereby it is acknowledged by all that an album in the top 100 (or whatever) MUST by definition be up there somewhere near the 5-star bracket, and therefore any album not in that list cannot have 5-stars - if anyone should wish to allocate 5-stars to anything else, then a special forum should be created for that album, and members debate the issue, either until a consensus is reached, or a time limit or something, at the end of which a vote would be taken (of those participating in the debate). Maybe, either: the writer of the review would take part in the debate in order to defend his/her postion; or, the reviewer is excluded so that the discussion is more impartial. Depending on the outcome of the vote, the rating either stays or is withdrawn. Of course, this could also be applied to 1-star reviews - ANY 1-star review would automatically have this treatment.
By the way, I agree with the guy who suggests that albums which have more reviews should require a new review to be longer to force people to say something more constructive.
I also would agree that no-one should be able to provide a rating without a review.
Hope you do not think I am being presumptious, but I feel this issue needs sorting before I can take things too seriously.
Regards Joolz
|
 |
Wilcey
Forum Senior Member
VIP Member
Joined: August 11 2005
Location: United Kingdom
Status: Offline
Points: 2696
|
Posted: March 17 2006 at 16:20 |
i have just deiscovered this thread and found it very
interesting, occisionally funny, and at times narrow
sighted.
But that it just my opinion, and how I see it. Which is
surely what a review should be?
If I write a review, (I havn't written many) it will be
about something I feel passionate about, (personally
I tend to keep my negative passions away from a
public arena, so I don't waste either my or your time
telling you when I think an album sucks!) So In my
passion, I will tell you what I think of an album, how I
feel about it, and why I think it deserves your
attention.
Surely, if that is the basis a review is written on, then
all reviews are valid? Surely it's just the obvious
spam-monsters that need attention? Opinion is
SUCH a subjective thing, for example if I were to love
an album, it would be for me a personal thing, and if
the rest of you thought it sucked big time, that does
not make a bad review, (flawed opinion maybe!?) my
thoughts would still be valid?
This is where prog-snobbery comes in. Being
popular (and maybe therefore commercially a
success) does NOT make something bad, as a
group of fans of a particular genre we should
celebrate success surely? If we weedle out the
successfull on this basis we will be left with
talenless drivel, and we will all be left admiring the
Emperors new clothes!
My other point here, is how amusing I find it, when
some one who may have spent anywhere from 10
mins to a handful of hours writing a review, and then
feeling bad if someone says they disagree or that it
is "rubbish"............ imagine that feeling, then imagine
having spent, oooh lets say TWO YEARS of your life
writing and recording an album, you know in your
heart that it says what you want it to say, how you
want it said, it is full of passion and truth. Then you
log on to the good ol' WWW to discover someone
has written it off with an hours worth of typed
words..... yes I should imagine it hurts.
It does not mean this album IS rubbish. It is opinion.
thats all.
So I think we should all try to take reviews with a
pinch of salt.
They can be informative, they can be funny, that can
be ridiculous, but unless they are offensive or
illegible in language then they should stay.
If for no other reason, than for entertainment!
I enjoy the reviews here, the good (star rating) and
the bad.
We all have what other folk would consider flawed
taste on things we are passionate about. Thats what
makes us so interesting and so much fun!
Prog-Chick x
|
 |
sleeper
Prog Reviewer
Joined: October 09 2005
Location: Entropia
Status: Offline
Points: 16449
|
Posted: March 07 2006 at 14:51 |
As regardings to any possible inclusion on PA of this system can I sugest that only the collabs are allowed to give "Karma" rateings to reviews. I am famillier with a lot of the style of writeing of many of the collabs here and though I dont allways agree, I can respect the views of these people because they have given a well thought out review that explains their position clearly. Its this that should be reviewed and as I trust many of the collabs to rate this and not wether they agree with the persons view, I sugest we only allow them to make the rateings.
|
Spending more than I should on Prog since 2005
|
 |
Ivan_Melgar_M
Special Collaborator
Honorary Collaborator
Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557
|
Posted: March 06 2006 at 17:34 |
Sean Trane wrote:
That's exactly my case Sean, if you remember I started making reviews under my real and complete name (Iván Melgar Morey), recently my nick has been added because of the site's rules but I asked not do delete my name, because I love what I do.
I also made reviews for GEPR and Rick Wakeman's Communication center plus other places like Magenta Web site, and my real email address can be found in some of those places.
Thanks to this reviews (before Prog Archives) I got connected with Magenta and a group of Israel bands that had the kindness to send me exclusive material for clinics to my real home address.
Until today I mostly recieved positive mails (A lot of Spam) and one or two Phil Collins fans insulting me (One called me stinking negro latin who dared to talk against a white icon of Brithish Rock , something not too exact because my Mom is Scottish/Italian ands my dad is Spanish/English/Native Peruvian but I'm proud of my Latin inheritance, probably 12.5% of native Peruvian Indian from my father's father who was the most intelligent man I ever knew, MD, who learned 5 languages including ancient Greek and Latin by his own)
So if I recieve negarive messages or ratings, I give a damn, most of my reviews are very large (Over 800 words except the first 5 or 6 first ones) because I take the job seriously, so if anybody wants to send me negative ratings, I honestly don't care and won't stop writting.
But there are reviewers that could be discouraged, and we can't afford to loose good and hard working members because a couple of trolls and flamers believe we manipulate the top 100 (As if none of us had a life!!!!) just because they don't see their beloved band oin the list.
I believe the system I explained could make even harder to manipulate ratings, because a good number of content administrators will be ready to delete reviews made with a lot of words but no sense, plus our members who keep sending flawed reviews to be deleted in the abuse section.
Iván
Edited by ivan_2068
|
|
 |
glass house
Forum Senior Member
Joined: June 16 2005
Location: Netherlands
Status: Offline
Points: 4986
|
Posted: March 06 2006 at 06:45 |
HEAR, HEAR ,
|
 |
Sean Trane
Special Collaborator
Prog Folk
Joined: April 29 2004
Location: Heart of Europe
Status: Offline
Points: 20642
|
Posted: March 06 2006 at 06:06 |
[QUOTE=Snow Dog]For what its worth, I think its quite a good idea. There are a lot of reviewers who deliberately overrate and underrate albums by either their favourite bands or baNDS THEY HATE.[/QUOTE]
Agreed with you, this is one of the vicissitude of our system
I with my some 1500 reviews am not really concerned about someone challenging my views. I have been attacked/badmouthed (for even putting a three stars review) many times and always managed to defend succesfully what I have written if I am aware someone is attacking me but this means you have to check back on your previous reviews.
If you post some 50 reviews, one can actually monitor the reviews attacking your reviews but once you get to spme 300 you lack the time to do so or even worse in my case the will or wish to do so. I must say that if there was not Bob (now with Atkingi) monitoring the reviews, I would have stopped a while ago to review, simply because you do not want your name being plastered over a toilet wall of hate simply because you do not think Snow Goose is not a 5* and someone is hating you for it.
I take great pride to review under my name, which is not a majority of members who "hide" (not meant to be cowardly but rather prudent , because of abusers) under a nickname . Writing under your real name (how to make sure it is your real name is a different matter altoghether ) is proof that you do so with a real honesty (who knows , one day Latimer will one day tell me:" oh you are this guy who wrote that not-so-good review about Snow Goose, care to tell me more?" and then he knives up in the belly if I ever get to meet him and have a chance to tell me my name.) and I even spend money to rent the albums, so I can write reviews. I love to give you an idea what Interference Sardine sounds like. look'em up if you do not believe me
But having haters , fanboys and trolls already abusing the system , why give them another opportunity to trash reviewers (for the haters), skew the ratings (for the fanboys) and spread unrest (for the trolls). If this guy hates my guts for having given Snow Goose three stars, he might just be tempted to give me a bad profile just to discredit me and you will not trust my reviews on Interference Sardine or Art Bears 
Not that I care so much about my name being trashed (I am fairly thick skinned ) , but this rating ther reviews may stop some people from discovering new bands
And unlike Amazon (which is about selling records and therefore will retrieve any reviews not favorable as Ivan stated, since it will not sell the record - and probably retrieve any bad rating of positive reviews), we are not in the selling business here. We want newly found fans of prog to be directed to the better bands (not to the third division of also-ran) whether they are known or not. Amazon will never propose you to sell a record it cannot easilty get or is widely available , therefore AMAZON has no interest in objectivity (which is what a worthy reviewer tries to reach beyond his particular tastes)
Amazong, uh?
|
let's just stay above the moral melee prefer the sink to the gutter keep our sand-castle virtues content to be a doer as well as a thinker, prefer lifting our pen rather than un-sheath our sword
|
 |
glass house
Forum Senior Member
Joined: June 16 2005
Location: Netherlands
Status: Offline
Points: 4986
|
Posted: March 06 2006 at 02:33 |
Good idea Ivan, isn't the part - The system requires - a bit like mine idea ? Especially the second part. See what the admins think about it !
|
 |
stonebeard
Forum Senior Member
Joined: May 27 2005
Location: NE Indiana
Status: Offline
Points: 28057
|
Posted: March 06 2006 at 00:17 |
That's not a bad idea, Ivan.
|
|
 |
Ivan_Melgar_M
Special Collaborator
Honorary Collaborator
Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557
|
Posted: March 05 2006 at 19:36 |
MikeEnRegalia wrote:
Ok guys, here's what I will do:
For each album each user will be able to rate the "controversiality" of an album (as a percentage).
From that the system can calculate the average controversiality of each album. From the ratings the system also knows the average general rating for the album, and the average user rating for the album (the average compiled from the ratings of the user for the tracks of the album).
Now the system can determine the "karma" of each user review on the basis of this user's deviation from the average rating combined with the average controversiality of the album:
deviation = abs(avg_general - avg_user)
karma = 5 - abs (deviation / max(0.2, avg_controversiality_pct)*14)
The deviation ranges from 0 to 14. A high deviation will decrease the karma value, but a high controversiality percentage will lower that effect. For the maximum deviation of 14 (avg: 15 points, user rating: 1 point) the karma will range from 0 (20% controversiality and below) to 4 (100% controversiality). For the minimum deviation (0) the karma is 5.
If we do this for every album review, we can compute the user karma as the average of all the reviews of the user.
So now we have a karma number from 0 to 5 for each user. We can now use that number as a weight for each track rating of the user. But is it fair to apply karma to each track rating?
|
I find your system interesting except the rating track by track, this a waste of time, I believe albums are an entity not a mathematical formula, so I agree patially, because if I want to rate aln album with 5 stars, who is the majority to decide my rating is unfair??? If I rate an album with 1 or 5 stars i want them to be counted as that.
Still believe is too complex and doesn't cover all aspects of what we shoud try:
- We know nobody can manipulate top 20 albums, because of the large number of reviews.
- People review too many top 10 or 20 albums and ignore the rest.
- 99% of complains come from trolls and/or flamers claiming that our top 20 list is crap, almost nobody cares for the rest or at least I dion't see too many complains.
So the solution is easier to implement.
- Albums with 19 or less reviews require a minimum of 200 characters for the review and the rating to be counted.
- Albums with 20 to 49 reviews require a minimum of 400 characters for the review and the rating to be counted.
- Albums with 50 to 99 reviews require a minimum of 800 characters for the review and the rating to be counted.
- Albums with 100 or more reviews require a minimum of 1, 600 characters for the review and the rating to be counted.
- Definitely and absolutely banned ratings without reviews, they don't count or better don't exist.
This system will give two advantages.
- Any fanboy or troll who wants to manipulate ratings will have to do a hell of a job, and would make almost impossible any attempt
- We will encourage people to review lesser known albums with shorter reviews. 1,600 characters is a lot of work, and I doubt many people would want to rate SEBTP or Close to the Edge.
This system requires.
- A larger number of review content adms, who will read at least 10 reviews daily.
- If two of this adms agree, the review is deleted without further questioning (To avoid claims of hate from one person against a determined Adm).
Iván
Edited by ivan_2068
|
|
 |
MikeEnRegalia
Special Collaborator
Honorary Collaborator
Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21817
|
Posted: March 05 2006 at 15:38 |
I now implemented a really simple solution - it occurred to me while I was preparing the algorithm that I explained above. While I may still implement that later, I think that the following solution might suffice:
- For each track compute the average rating.
- For each rating, determine a weight using this formula: 1/(1+d), where d=distance between the individual rating and the average rating.
- For each track compute a weighted average rating using the weights computed in the previous step.
This is really simple, and it minimizes the effect of odd votes. And it's perfectly fair - the algorithm makes no preferences or anything.
What all this means is that the more a rating deviates from the average, the less it's weight will be. So if someone comes along and rates an established 5 star album 1 star, the effect on the average will be negligible. But if more people follow that example and at some point there is a substantial amount of 1 star reviews, the weight will adjust itself.
To put it even simpler: The more stable an average is, the less is the impact of new ratings which differ much from that average.

Edited by MikeEnRegalia
|
|
 |
erik neuteboom
Prog Reviewer
Joined: July 27 2005
Location: Netherlands
Status: Offline
Points: 7659
|
Posted: March 05 2006 at 15:24 |
Well, I feel free to nail Tales From Topografic Oceans by Yes, to me it sounds as an overambitious and too fragmentic bunch of songs. And And Then There Were Three by Genesis is 50% boring polished pop-prog. Is this too subjective? Am I allowed to write such negative reviews as a progrock specialist? Can progheads still take me serious while writing those negative reviews? I think so, as long as I deliver good and appreciated reviews on this site is my opinion. For me it's that simple.
|
 |
Ivan_Melgar_M
Special Collaborator
Honorary Collaborator
Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557
|
Posted: March 05 2006 at 12:05 |
Mike, Slipp3ry I rest my case with this statement:
Progger wrote:
Hurray,
Does this mean that the top ten will no longer be clogged up with *** star Genesis albums!!!
|
Guys like this one will only vote to destroy what has been achieved for all of us since 2004.
We have grown a very bushy tree, healthy and big, it's ok to shake it to let the rotten leaves and fruits fall, but with guys like this that declare they will shake it to destroy the roots, and this is not acceptable.
Iván
Edited by ivan_2068
|
|
 |
eugene
Forum Senior Member
Joined: May 30 2005
Location: Ukraine
Status: Offline
Points: 2703
|
Posted: March 05 2006 at 11:39 |
glass house wrote:
To Eugene : Your explanation of my words are not at all what I mean. You just twisted them around.
You also said,
You guys sound like you must have been brought up in totalitarian system, and now, being too scared about too much freedom of speach, want your "good old days" back.
Please, if you don't like the idea just say so without the rubbish.
By the way, Thanks Snowdog.
|
What you said in your own words means "censorship". There is nothing to explain further or to twist around.
I do not like this idea, and I said so and explained why.
Moreover, I do not like what YOU particularly said, and I explained why, giving you my impression of how it sounded to me.
You might not like what I said and you do not have to like it, but to call it rubbish is a bit unpolite from you (to say the least), and might cause unnecessary aggression. So please watch your mouth.
Hope everything is clear for you this time.
|
carefulwiththataxe
|
 |
glass house
Forum Senior Member
Joined: June 16 2005
Location: Netherlands
Status: Offline
Points: 4986
|
Posted: March 05 2006 at 10:54 |
To Eugene : Your explanation of my words are not at all what I mean. You just twisted them around.
You also said,
You guys sound like you must have been brought up in totalitarian system, and now, being too scared about too much freedom of speach, want your "good old days" back.
Please, if you don't like the idea just say so without the rubbish.
By the way, Thanks Snowdog.
Edited by glass house
|
 |
Snow Dog
Special Collaborator
Honorary Collaborator
Joined: March 23 2005
Location: Caerdydd
Status: Offline
Points: 32995
|
Posted: March 05 2006 at 10:02 |
For what its worth, I think its quite a good idea. Ther are a lot of reviewers who deliberately overrate and underrate albums by either theur favourite bands or baNDS THEY HATE.
|
|
 |
MikeEnRegalia
Special Collaborator
Honorary Collaborator
Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21817
|
Posted: March 05 2006 at 09:30 |
eugene wrote:
MikeEnRegalia wrote:
glass house wrote:
Mike : I don't know if your idea will work, we will see. I hope her on Pa the collaborators will rate a review before it is posted. If they think it is allright then post it. There are many skillful people here that can do that and who I trust. |
That's another topic ... I like the idea, but it sounds a bit like cencorship.
|
No, it sounds like a terrible bloody censorship in full force!!!
You guys sound like you must have been brought up in totalitarian system, and now, being too scared about too much freedom of speach, want your "good old days" back.
I am aware that there are always certain individuals who are ready to smoothly provide their censorship, and there always will be crowds "trusting" their beloved leaders (quite skillful by the way) and allowing them to act as they want, no matter what.
Funny and quite unbelievable that this is being discussed nowadays on international site devoted to progressive music.
Ah, never mind, go ahead, it's interesting to see what happens next...
|
I would like that idea if (and only if) the cencorship is limited to abuse. From time to time the front page suffers from a few members who post offensive reviews - and this situation could be helped if new reviews would have to be approved of by a collab (or admin) before they become visible.
But of course this would seem like cencorship even if we would approve of 100% of all the reviews, because visitors - knowing how the system works - could never be sure of what's going on behind the scenes.
|
|
 |
eugene
Forum Senior Member
Joined: May 30 2005
Location: Ukraine
Status: Offline
Points: 2703
|
Posted: March 05 2006 at 09:00 |
MikeEnRegalia wrote:
glass house wrote:
Mike : I don't know if your idea will work, we will see. I hope her on Pa the collaborators will rate a review before it is posted. If they think it is allright then post it. There are many skillful people here that can do that and who I trust. |
That's another topic ... I like the idea, but it sounds a bit like cencorship.
|
No, it sounds like a terrible bloody censorship in full force!!!
You guys sound like you must have been brought up in totalitarian system, and now, being too scared about too much freedom of speach, want your "good old days" back.
I am aware that there are always certain individuals who are ready to smoothly provide their censorship, and there always will be crowds "trusting" their beloved leaders (quite skillful by the way) and allowing them to act as they want, no matter what.
Funny and quite unbelievable that this is being discussed nowadays on international site devoted to progressive music.
Ah, never mind, go ahead, it's interesting to see what happens next...
|
carefulwiththataxe
|
 |
MikeEnRegalia
Special Collaborator
Honorary Collaborator
Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21817
|
Posted: March 05 2006 at 07:59 |
glass house wrote:
Mike : I don't know if your idea will work, we will see. I hope her on Pa the collaborators will rate a review before it is posted. If they think it is allright then post it. There are many skillful people here that can do that and who I trust. |
That's another topic ... I like the idea, but it sounds a bit like cencorship.
|
|
 |
glass house
Forum Senior Member
Joined: June 16 2005
Location: Netherlands
Status: Offline
Points: 4986
|
Posted: March 05 2006 at 05:43 |
Mike : I don't know if your idea will work, we will see. I hope her on Pa the collaborators will rate a review before it is posted. If they think it is allright then post it. There are many skillful people here that can do that and who I trust.
|
 |