Forum Home Forum Home > Site News, Newbies, Help and Improvements > Report abuse here
  New Posts New Posts RSS Feed - Inappropriate Ratings
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Inappropriate Ratings

 Post Reply Post Reply Page  123 23>
Author
Message Reverse Sort Order
Bonnek View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal Team

Joined: September 01 2009
Location: Belgium
Status: Offline
Points: 4137
Post Options Post Options   Thanks (0) Thanks(0)   Quote Bonnek Quote  Post ReplyReply Direct Link To This Post Topic: Inappropriate Ratings
    Posted: October 22 2014 at 14:37
Originally posted by aapatsos aapatsos wrote:

Originally posted by Dean Dean wrote:

Originally posted by Bonnek Bonnek wrote:



Why not the RYM system?
Your ratings don't count till your account is 'somehow' proven not to be a troll, rating manipulater or belonging to a mentally 3year old.

I admit there's a grey zone as to which criteria they apply to establish that but I expect just like here mods apply common sense rules to make such decisions.


I agree Karl.
During this time I analysed the effects of ratings abuse and the effectiveness of rating manipulation and came to the counter-intuitive conclusion that it doesn't make a great deal of difference. The current review and collab weighting out weighs the rating-only score and that is more effective than deleting 1000s of bogus ratings. The problem is that the perception of this abuse is that deliberate manipulation makes a big difference - you can look at any well respected album and see lots of 1 and 5 star ratings and that looks bad. Perception is everything since the number of votes we get for any album is too small to have any statistical value or meaning.



Great points Karl and Dean. Before I went through this, I thought "just how much of a problem is it really?" and your post gives the answer. This needs to be SPELT OUT somewhere on the site for artists/members/abusers to understand along with a note that checks are performed etc.

A previous idea of making the rating's value even smaller than it is now compared to reviews helps towards this direction. Perhaps not the ultimate solution, but an improvement.


Yes I remember that analysis of how the trolls basically sabotage other trolls, so it's futile, but it's a nuisance and indeed it looks bad on us.

- Another advantage RYM has is that they have significantly more ratings
- And compared to PA it looks like the number of ratings has a greater weight then the actual rating (it's an impression, i have not checked the actual calculation).
- Also, they have half stars which avoids somewhat the overabuse of the 5 stars

Anyhow, not much we can do about the number of ratings, but it all balances out their ratings much better.



Edited by Bonnek - October 22 2014 at 14:39
Back to Top
aapatsos View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal and Heavy Prog Teams

Joined: November 11 2005
Location: Rotherham, UK
Status: Offline
Points: 7079
Post Options Post Options   Thanks (0) Thanks(0)   Quote aapatsos Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 11:35
Originally posted by Dean Dean wrote:

Originally posted by Bonnek Bonnek wrote:



Why not the RYM system?
Your ratings don't count till your account is 'somehow' proven not to be a troll, rating manipulater or belonging to a mentally 3year old.

I admit there's a grey zone as to which criteria they apply to establish that but I expect just like here mods apply common sense rules to make such decisions.


I agree Karl.
During this time I analysed the effects of ratings abuse and the effectiveness of rating manipulation and came to the counter-intuitive conclusion that it doesn't make a great deal of difference. The current review and collab weighting out weighs the rating-only score and that is more effective than deleting 1000s of bogus ratings. The problem is that the perception of this abuse is that deliberate manipulation makes a big difference - you can look at any well respected album and see lots of 1 and 5 star ratings and that looks bad. Perception is everything since the number of votes we get for any album is too small to have any statistical value or meaning.



Great points Karl and Dean. Before I went through this, I thought "just how much of a problem is it really?" and your post gives the answer. This needs to be SPELT OUT somewhere on the site for artists/members/abusers to understand along with a note that checks are performed etc.

A previous idea of making the rating's value even smaller than it is now compared to reviews helps towards this direction. Perhaps not the ultimate solution, but an improvement.
"Prog Heavy Petting" show - every Monday 7-9pm UK time on http://www.justincaseradio.com, the first progressive radio in Greece
Back to Top
SteveG View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: April 11 2014
Location: usa
Status: Offline
Points: 1616
Post Options Post Options   Thanks (0) Thanks(0)   Quote SteveG Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 10:54
I have an interesting question. Since I rarely review albums that I don't like, except for the sake of completing some review sites where many albums are unrated such as later day Roy Harper material, do my 4 and 5 star ratings look amiss? I generally check my reviews against those of a collaborator when possible and we are usually on the same page. But again, I don't relish reviewing inferior material unless I absolutely have to.


Edited by SteveG - October 22 2014 at 10:57
"Even a broken watch is right twice a day" -Man too lazy to wind a watch.
Back to Top
Gotrek1966 View Drop Down
Forum Newbie
Forum Newbie
Avatar

Joined: October 07 2014
Location: Australia
Status: Offline
Points: 7
Post Options Post Options   Thanks (0) Thanks(0)   Quote Gotrek1966 Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 03:48
Originally posted by Dean Dean wrote:

This is appalling and inexcusable. You cannot review an album you have never heard. That is a preposterous idea and your review must be deleted immediately.

My apologies Dean if what I have done was incorrect, it was not intended to be that way by my explanation. However if the Moderators feel my review should be deleted then I accept their decision.  
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Albion
Status: Offline
Points: 33166
Post Options Post Options   Thanks (0) Thanks(0)   Quote Dean Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 03:33
Originally posted by Bonnek Bonnek wrote:



Why not the RYM system?
Your ratings don't count till your account is 'somehow' proven not to be a troll, rating manipulater or belonging to a mentally 3year old.

I admit there's a grey zone as to which criteria they apply to establish that but I expect just like here mods apply common sense rules to make such decisions.


I agree Karl.

The 'somehow' isn't that difficult but would be prone to the occasional mistake and the dedicated rating-abusers would soon find ways of fooling the system. Of course the harder we make it for people to abuse the rating system the harder it is to detect them. The problem with rating-abuse counter measures is they encourage manipulators to abuse more, for example the review-weighting encourages manipulators to create multiple bogus accounts and to submit plagiarised reviews. To be effective the system needs to be foolproof and that is an unattainable goal. However, I don't believe it needs to be effective, it just needs to be visible.

The problem of rating-abuse is not that it distorts the average rating for an album, and thus affects it position in the yearly Top-100 chart but that reflects badly on the image of the PA as a credible album-rating site. Any 'somehow' system that visibly addresses the perception of ratings-abuse would not affect the dedicated rating-manipulators but it would improve the perception of "fairness" as seen by the other members, including the artists themselves. 

We cannot stop rating manipulation, we have made it ineffective now we need to make it so it is perceived to be ineffective.

We already use most of the counter-measures that RYM (and IMDB) use to combat rating abuse, the only one we really lack is the part of the algorithm that ignores the ratings from 'unapproved' members. At present we deal with this after-the-event and that's a really poor way of doing it.

We can only detect a trollish manipulator after they have rated albums and since their ratings count towards the album average and chart position this abuse is perceived to be very visible. There is an algorithm in the Admin page that identifies possible ratings manipulators but this is crude and requires a human to further investigate each account in the list individually and that is time consuming. It could be (and should be) refined and even automated of course but only M@X can do that, but it would still need those abusers to rate albums first so the analysis could be performed.

In the past I have spent a lot of time and effort tackling ratings abuse and have deleted 1000s of bogus ratings, deleted/disabled 100s of bogus accounts and blocked 100s of IP addresses for very little positive benefit. During this time I analysed the effects of ratings abuse and the effectiveness of rating manipulation and came to the counter-intuitive conclusion that it doesn't make a great deal of difference. The current review and collab weighting out weighs the rating-only score and that is more effective than deleting 1000s of bogus ratings. The problem is that the perception of this abuse is that deliberate manipulation makes a big difference - you can look at any well respected album and see lots of 1 and 5 star ratings and that looks bad. Perception is everything since the number of votes we get for any album is too small to have any statistical value or meaning.

[I have seen artists complain of rating-abuse for their latest album only to see it drop down a place or two the yearly Top-100 chart after I had deleted a few 100 suspicious ratings. The reason for this is two-fold, the first has already been identified in this thread - after deleting ratings the new average for each album is only re-calculated when the next person rates that album; and the second is that all albums in the chart get abused so if we delete a bunch of bogus ratings then the average score for each album is affected and, on the whole, this tends to average out.]



Edited by Dean - October 22 2014 at 03:35


If you cannot be wise, pretend to be someone who is wise and then just behave like they would - Neil Gaiman
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Albion
Status: Offline
Points: 33166
Post Options Post Options   Thanks (0) Thanks(0)   Quote Dean Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 03:29
Originally posted by Gotrek1966 Gotrek1966 wrote:

Originally posted by Argonaught Argonaught wrote:

A lot of would-be 1- and 5-star reviewers may feel uncomfortable about having to justify their extreme ratings by writing a review; for one reason, the rating would no longer be anonymous. If the justification is clearly phony or OTT exaggerated, the reviewer's reputation would be put at risk. 

I'm kind of agreeing on this one. I gave a review of Gandalf's Fists new album and a 5 star rating before the album was out and have no issue with justifying why. Having listened to previous songs, new album samples and listening to full songs (new/old albums) on youtube, my review and rating is based purely on what "I" think about the album. I have listened to a few bands of late with mixed feelings and if I did a review then it's an honest review with no bells or whistles added.

If somebody gives a review that is in my mind a lesser review, then that's up to them with no issues from me. So long as it's an honest review and rating based on what that person feels I see no problem. Obviously if it was all claims of "The best album in the world, all other bands are nothing compared to this band" then yes I would be raising an eyebrow and asking questions.

That's just me though, you ladies and gents have been listening to prog music a lot longer than me and I respect your thoughts on this.

Cheers
Gotrek
This is appalling and inexcusable. You cannot review an album you have never heard. That is a preposterous idea and your review must be deleted immediately.


If you cannot be wise, pretend to be someone who is wise and then just behave like they would - Neil Gaiman
Back to Top
Gotrek1966 View Drop Down
Forum Newbie
Forum Newbie
Avatar

Joined: October 07 2014
Location: Australia
Status: Offline
Points: 7
Post Options Post Options   Thanks (0) Thanks(0)   Quote Gotrek1966 Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 03:08
Originally posted by Argonaught Argonaught wrote:

A lot of would-be 1- and 5-star reviewers may feel uncomfortable about having to justify their extreme ratings by writing a review; for one reason, the rating would no longer be anonymous. If the justification is clearly phony or OTT exaggerated, the reviewer's reputation would be put at risk. 

I'm kind of agreeing on this one. I gave a review of Gandalf's Fists new album and a 5 star rating before the album was out and have no issue with justifying why. Having listened to previous songs, new album samples and listening to full songs (new/old albums) on youtube, my review and rating is based purely on what "I" think about the album. I have listened to a few bands of late with mixed feelings and if I did a review then it's an honest review with no bells or whistles added.

If somebody gives a review that is in my mind a lesser review, then that's up to them with no issues from me. So long as it's an honest review and rating based on what that person feels I see no problem. Obviously if it was all claims of "The best album in the world, all other bands are nothing compared to this band" then yes I would be raising an eyebrow and asking questions.

That's just me though, you ladies and gents have been listening to prog music a lot longer than me and I respect your thoughts on this.

Cheers
Gotrek
Back to Top
Bonnek View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal Team

Joined: September 01 2009
Location: Belgium
Status: Offline
Points: 4137
Post Options Post Options   Thanks (0) Thanks(0)   Quote Bonnek Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 02:07


Why not the RYM system?
Your ratings don't count till your account is 'somehow' proven not to be a troll, rating manipulater or belonging to a mentally 3year old.

I admit there's a grey zone as to which criteria they apply to establish that but I expect just like here mods apply common sense rules to make such decisions.


Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Albion
Status: Offline
Points: 33166
Post Options Post Options   Thanks (0) Thanks(0)   Quote Dean Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 00:26
There is only one way to stop ratings abuse and that is to scrap ratings completely.


If you cannot be wise, pretend to be someone who is wise and then just behave like they would - Neil Gaiman
Back to Top
Argonaught View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: June 04 2012
Location: Virginia
Status: Offline
Points: 523
Post Options Post Options   Thanks (0) Thanks(0)   Quote Argonaught Quote  Post ReplyReply Direct Link To This Post Posted: October 21 2014 at 19:48
Originally posted by Second Life Syndrome Second Life Syndrome wrote:

So, basically, 1 and 5 would become 2 and 4; not a real solution.

In real life, there is no real solution to anything. People make improvements by finding partial/compromise solutions and working on bettering them. Would preventing 50% of frivolous and malignant ratings by ad hoc "reviewers" be a step in the right direction? What would you propose? 

A lot of would-be 1- and 5-star reviewers may feel uncomfortable about having to justify their extreme ratings by writing a review; for one reason, the rating would no longer be anonymous. If the justification is clearly phony or OTT exaggerated, the reviewer's reputation would be put at risk. 

As for leaving 2- instead of 1-star, well, that wouldn't be nearly as gratifying. If you hate someone so much that you want to punch him, but instead all you can do is tapping him with a rolled-up monthly electricity bill, you may feel rather disappointed Disapprove 


Edited by Argonaught - October 21 2014 at 19:49
Back to Top
SteveG View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: April 11 2014
Location: usa
Status: Offline
Points: 1616
Post Options Post Options   Thanks (0) Thanks(0)   Quote SteveG Quote  Post ReplyReply Direct Link To This Post Posted: October 21 2014 at 16:13
^LOLLOLLOL
"Even a broken watch is right twice a day" -Man too lazy to wind a watch.
Back to Top
HolyMoly View Drop Down
Special Collaborator
Special Collaborator
Avatar
RIO/Avant/Zeuhl and Canterbury Teams

Joined: April 01 2009
Location: Atlanta
Status: Offline
Points: 20596
Post Options Post Options   Thanks (0) Thanks(0)   Quote HolyMoly Quote  Post ReplyReply Direct Link To This Post Posted: October 21 2014 at 15:24
Originally posted by Second Life Syndrome Second Life Syndrome wrote:

Originally posted by Argonaught Argonaught wrote:

And once again I will propose that access to extreme ratings be regulated and rationed. 

No 1- or 5-star ratings should be accepted from new members (which would help weed out those who sign up under a bunch of different names and stir the pot). 

Applicable to all members: 1- and 5-star ratings to be automatically up- and downgraded to 2- and 4-stars respectively, if not supported by a meaningful review. 

Introduce a reasonable quota for extreme ratings, for each reviewer: 10% max should be enough. 

So, basically, 1 and 5 would become 2 and 4; not a real solution.

So we can allow only 3's.  Sounds good to me.
My other avatar is a Porsche / RARE GOAT bandcamp page

It is easier for a camel to pass through the eye of a needle if it is lightly greased.
-Kehlog Albran
Back to Top
Second Life Syndrome View Drop Down
Collaborator
Collaborator
Avatar
Post/Math Rock and Crossover Teams

Joined: August 20 2012
Location: United States
Status: Offline
Points: 330
Post Options Post Options   Thanks (0) Thanks(0)   Quote Second Life Syndrome Quote  Post ReplyReply Direct Link To This Post Posted: October 21 2014 at 15:11
Originally posted by Argonaught Argonaught wrote:

And once again I will propose that access to extreme ratings be regulated and rationed. 

No 1- or 5-star ratings should be accepted from new members (which would help weed out those who sign up under a bunch of different names and stir the pot). 

Applicable to all members: 1- and 5-star ratings to be automatically up- and downgraded to 2- and 4-stars respectively, if not supported by a meaningful review. 

Introduce a reasonable quota for extreme ratings, for each reviewer: 10% max should be enough. 

So, basically, 1 and 5 would become 2 and 4; not a real solution.
The Prog Mind:

*For those who love meaningful, artistic music.*

https://www.facebook.com/TheProgMind
Back to Top
Guldbamsen View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Site and Forum Admin

Joined: January 22 2009
Location: Dolly Parton
Status: Online
Points: 13328
Post Options Post Options   Thanks (0) Thanks(0)   Quote Guldbamsen Quote  Post ReplyReply Direct Link To This Post Posted: October 21 2014 at 06:43
Originally posted by antonyus antonyus wrote:

what the hell is going on here :)

http://www.progarchives.com/Collaborators.asp?id=53526


Ratings are swiped now and I've sent him a mail.
“The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.”

- Douglas Adams
Back to Top
Argonaught View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: June 04 2012
Location: Virginia
Status: Offline
Points: 523
Post Options Post Options   Thanks (0) Thanks(0)   Quote Argonaught Quote  Post ReplyReply Direct Link To This Post Posted: October 20 2014 at 22:52
And once again I will propose that access to extreme ratings be regulated and rationed. 

No 1- or 5-star ratings should be accepted from new members (which would help weed out those who sign up under a bunch of different names and stir the pot). 

Applicable to all members: 1- and 5-star ratings to be automatically up- and downgraded to 2- and 4-stars respectively, if not supported by a meaningful review. 

Introduce a reasonable quota for extreme ratings, for each reviewer: 10% max should be enough. 


Edited by Argonaught - October 20 2014 at 22:54
Back to Top
odinalcatraz View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: July 12 2010
Location: London
Status: Offline
Points: 206
Post Options Post Options   Thanks (0) Thanks(0)   Quote odinalcatraz Quote  Post ReplyReply Direct Link To This Post Posted: October 20 2014 at 08:23
I wish it would stop.
No idea how to stop it! Sickening really and childish but it happens every day it seems.
Back to Top
antonyus View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: January 30 2006
Location: Amsterdam
Status: Offline
Points: 488
Post Options Post Options   Thanks (0) Thanks(0)   Quote antonyus Quote  Post ReplyReply Direct Link To This Post Posted: October 20 2014 at 05:47
what the hell is going on here :)

http://www.progarchives.com/Collaborators.asp?id=53526
Back to Top
odinalcatraz View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: July 12 2010
Location: London
Status: Offline
Points: 206
Post Options Post Options   Thanks (0) Thanks(0)   Quote odinalcatraz Quote  Post ReplyReply Direct Link To This Post Posted: October 13 2014 at 13:00
Thumbs Down
I saw this WTF! http://www.progarchives.com/Collaborators.asp?id=53479&listreviews=rate&showall=true#reviews

I see that someone has signed up downed a huge number of albums (I'm used to having that done to us) but this time, 3 albums that I am on were given 5s. This is another idiot up to no good. Nobody I know would do that. i hope it can be fixed.
Damn those people!
Back to Top
Guldbamsen View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Site and Forum Admin

Joined: January 22 2009
Location: Dolly Parton
Status: Online
Points: 13328
Post Options Post Options   Thanks (0) Thanks(0)   Quote Guldbamsen Quote  Post ReplyReply Direct Link To This Post Posted: April 29 2014 at 02:49
Keishiro took care of this
“The Guide says there is an art to flying or rather a knack. The knack lies in learning how to throw yourself at the ground and miss.”

- Douglas Adams
Back to Top
ScorchedFirth View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: May 16 2012
Status: Offline
Points: 229
Post Options Post Options   Thanks (0) Thanks(0)   Quote ScorchedFirth Quote  Post ReplyReply Direct Link To This Post Posted: April 28 2014 at 12:55
Suddenly about 100 new 5-star ratings for the whirlwind from day old accounts with randomly generated user names/names/locations. All of them giving 5-stars to the whirlwind and no other rating. Happens pretty much the same time the user HelloYes, another 1-day account that rates only the whirlwind, reviews the album.

Why go to all this bother just to push up one transatlantic album by about a 20th of a star?

Edit: This was my favourite fake account: 5 stars Birdemic (Alan Bagh) LOL

Edited by ScorchedFirth - April 28 2014 at 12:57
breathing, eating, defecating, screwing, drinking, spewing, sleeping...

Back to Top
 Post Reply Post Reply Page  123 23>
  Share Topic   

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 11.01
Copyright ©2001-2014 Web Wiz Ltd.

This page was generated in 0.113 seconds.