Forum Home Forum Home > Site News, Newbies, Help and Improvements > Report abuse here
  New Posts New Posts RSS Feed - Inappropriate Ratings
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Inappropriate Ratings

 Post Reply Post Reply Page  123 24>
Author
Message Reverse Sort Order
aapatsos View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal and Heavy Prog Teams

Joined: November 11 2005
Location: Rotherham, UK
Status: Offline
Points: 7212
Post Options Post Options   Thanks (0) Thanks(0)   Quote aapatsos Quote  Post ReplyReply Direct Link To This Post Topic: Inappropriate Ratings
    Posted: 15 hours 32 minutes ago at 15:46
This user seems to outright assign 1-star to every S Wilson and P Tree release there is out there...http://www.progarchives.com/Collaborators.asp?id=7677
"Prog Heavy Petting" show - every Monday 7-9pm UK time on http://www.justincaseradio.com, the first progressive radio in Greece
Back to Top
Argonaught View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: June 04 2012
Location: Virginia
Status: Offline
Points: 739
Post Options Post Options   Thanks (0) Thanks(0)   Quote Argonaught Quote  Post ReplyReply Direct Link To This Post Posted: November 01 2014 at 20:42
Originally posted by floflo79 floflo79 wrote:

http://www.progarchives.com/album.asp?id=47220

An user named Matt T rate The Endless River with 5 stars. Even if there's only 2 songs who came out.

Why even enable the rating/review feature before the album has been released? 
Back to Top
aapatsos View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal and Heavy Prog Teams

Joined: November 11 2005
Location: Rotherham, UK
Status: Offline
Points: 7212
Post Options Post Options   Thanks (0) Thanks(0)   Quote aapatsos Quote  Post ReplyReply Direct Link To This Post Posted: November 01 2014 at 11:31
Remains to be seen if abuse on this (yet another new) entry is about to begin - hope not
"Prog Heavy Petting" show - every Monday 7-9pm UK time on http://www.justincaseradio.com, the first progressive radio in Greece
Back to Top
floflo79 View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: March 21 2013
Location: France
Status: Offline
Points: 118
Post Options Post Options   Thanks (0) Thanks(0)   Quote floflo79 Quote  Post ReplyReply Direct Link To This Post Posted: November 01 2014 at 11:15
http://www.progarchives.com/album.asp?id=47220

An user named Matt T rate The Endless River with 5 stars. Even if there's only 2 songs who came out.
Back to Top
Nogbad_The_Bad View Drop Down
Collaborator
Collaborator
Avatar
RIO/Avant/Zeuhl Team

Joined: March 16 2007
Status: Offline
Points: 3638
Post Options Post Options   Thanks (0) Thanks(0)   Quote Nogbad_The_Bad Quote  Post ReplyReply Direct Link To This Post Posted: November 01 2014 at 09:06
Originally posted by Xonty Xonty wrote:

This guys reviews are pretty extreme - about 80% 5-star and 10% 1-star.

http://www.progarchives.com/Collaborators.asp?id=52858


Haha

Pavlovs Dog get one 5 star album, and 4 one start album. Apparently Dream Theater have done 33 5 star albums, singles and dvds!!


Edited by Nogbad_The_Bad - November 01 2014 at 09:07
Ian

Anyone who thinks Kansas is Prog get out of the room - Adolf Hitler

Back to Top
Xonty View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: June 23 2013
Status: Offline
Points: 1424
Post Options Post Options   Thanks (0) Thanks(0)   Quote Xonty Quote  Post ReplyReply Direct Link To This Post Posted: November 01 2014 at 08:41
This guys reviews are pretty extreme - about 80% 5-star and 10% 1-star.

http://www.progarchives.com/Collaborators.asp?id=52858
Back to Top
Kati View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 10 2010
Location: Mozambique
Status: Offline
Points: 2337
Post Options Post Options   Thanks (1) Thanks(1)   Quote Kati Quote  Post ReplyReply Direct Link To This Post Posted: October 31 2014 at 08:42
Positive feedback in terms of this new software update, the new thanks button love that! Also I seem not to have any problem whatsoever typing taking my time post and not having the text I typed disappear! Ha! I like it! I like this! A massive hug Hug
When playing strip poker, make sure you are wearing a lot of clothes! or google will ban you ;)
Back to Top
Argonaught View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: June 04 2012
Location: Virginia
Status: Offline
Points: 739
Post Options Post Options   Thanks (0) Thanks(0)   Quote Argonaught Quote  Post ReplyReply Direct Link To This Post Posted: October 31 2014 at 05:41
Originally posted by Bonnek Bonnek wrote:

Thanks for this!
Sound like they'll need another review to get this refreshed



I can't remember if this was in this thread or somewhere else: I "accidentally" discovered that, after a few fake reviews have been removed, the system doesn't automatically recalculate the average rating. I went to my own review, changed the rating (e.g. from 4 to 3 stars), saved it, then changed the rating back to what it was (from 3 to 4 stars) and saved again. This way you don't need to wait for a new reviewer to trigger the recalculation. 
Back to Top
Bonnek View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal Team

Joined: September 01 2009
Location: Belgium
Status: Offline
Points: 4144
Post Options Post Options   Thanks (0) Thanks(0)   Quote Bonnek Quote  Post ReplyReply Direct Link To This Post Posted: October 31 2014 at 04:06
Thanks for this!
Sound like they'll need another review to get this refreshed




Edited by Bonnek - October 31 2014 at 04:09
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Albion
Status: Offline
Points: 33272
Post Options Post Options   Thanks (1) Thanks(1)   Quote Dean Quote  Post ReplyReply Direct Link To This Post Posted: October 31 2014 at 02:49
Originally posted by Bonnek Bonnek wrote:

Originally posted by Angelo Angelo wrote:

I reported a few more tonight. Not sure if anything can be done though - we devised the rating calculation to be such that ratings only count with a far lesser weight than reviews, and these are outweighed by collaborator reviews. Usually, that works, and it may be less work than removing the trolls. However, after 8 years I also tend to feel its about time to send out a bit of a stronger signal here...


I'm not so sure how this whole weighted calculation works.
Take this album for instance as an example. It hasn't got too much reviews/ratings so calculation shouldn't be too difficult to simulate?

http://www.progarchives.com/album-reviews.asp?id=44375

This album has got 3 reviews
1 x 5 star rating by member
1 x 5 star rating by collaborator
1 x 4 star rating by collaborator

On top 52 ratings.

This gives an overal average is only 3.81, which is very low for PA standards and especially so if you look at the appreciation in the reviews.
The weightings are simple: 
rating only = x1
review = x10
collab review = x20

working out the average would be:
Quote
Warthur = 20 :: 20 x 5 = 100
UMUR  = 20 :: 20 x 4 = 80
alderbran well  = 10 :: 10 x 5 = 50
total 5 ratings = 13 :: 13 x 5 = 65
total 4 ratings =  6 ::   6 x 4 = 24
total 3 ratings = 12 ::  12 x 3 = 36
total 2 ratings = 10 ::  10 x 2 = 20
total 1 ratings =  6 ::   6 x 1 = 6
------ -------
totals  = 97 :: = 381
------ -------
therefore average = 381 / 97 = 3.93 (50 ratings)
There is a discrepancy between that and the published average of 3.81 (55 ratings) because the total hasn't been recalculated since 5 bogus ratings were deleted. (I suspect those were 3x1-star and 2x2-star)

To see whether the ratings are "fair" you need to look at the distribution of votes, this little bar-chart on each album page is far more informative and meaningful than any calculated average:

Quote
5-star 31% ************
4-star 15% ******
3-star 25% **********
2-star 17% ******
1-star 12% ****
The first thing this tells you is that the 3-star average is not wrong.

You would expect this to follow a "Normal" distribution but as you can see the number of 4-star ratings is lower than expected, which implies the 5-stars are higher than expected. This is not unusual or "wrong" because the sample size of 50 votes is too low to be statistically meaningful so is skewed by the number of "legitimate" fan votes (all albums attract a disproportionate number of 5-star ratings to begin with).



If you cannot be wise, pretend to be someone who is wise and then just behave like they would - Neil Gaiman
Back to Top
Kati View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 10 2010
Location: Mozambique
Status: Offline
Points: 2337
Post Options Post Options   Thanks (0) Thanks(0)   Quote Kati Quote  Post ReplyReply Direct Link To This Post Posted: October 30 2014 at 23:01
Originally posted by Bonnek Bonnek wrote:

Originally posted by Angelo Angelo wrote:

I reported a few more tonight. Not sure if anything can be done though - we devised the rating calculation to be such that ratings only count with a far lesser weight than reviews, and these are outweighed by collaborator reviews. Usually, that works, and it may be less work than removing the trolls. However, after 8 years I also tend to feel its about time to send out a bit of a stronger signal here...


I'm not so sure how this whole weighted calculation works.
Take this album for instance as an example. It hasn't got too much reviews/ratings so calculation shouldn't be too difficult to simulate?

http://www.progarchives.com/album-reviews.asp?id=44375

This album has got 3 reviews
1 x 5 star rating by member
1 x 5 star rating by collaborator
1 x 4 star rating by collaborator

On top 52 ratings.

This gives an overal average is only 3.81, which is very low for PA standards and especially so if you look at the appreciation in the reviews.



You peaked my interest Bonnek :) looked them up and so far only managed to hear bits on CD Universe, however this was enough for me to know I will not be able to judge them or their music. They are more death metal compared to my music preference Wink hugs Hug 
When playing strip poker, make sure you are wearing a lot of clothes! or google will ban you ;)
Back to Top
Bonnek View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal Team

Joined: September 01 2009
Location: Belgium
Status: Offline
Points: 4144
Post Options Post Options   Thanks (0) Thanks(0)   Quote Bonnek Quote  Post ReplyReply Direct Link To This Post Posted: October 29 2014 at 13:02
Originally posted by Angelo Angelo wrote:

I reported a few more tonight. Not sure if anything can be done though - we devised the rating calculation to be such that ratings only count with a far lesser weight than reviews, and these are outweighed by collaborator reviews. Usually, that works, and it may be less work than removing the trolls. However, after 8 years I also tend to feel its about time to send out a bit of a stronger signal here...


I'm not so sure how this whole weighted calculation works.
Take this album for instance as an example. It hasn't got too much reviews/ratings so calculation shouldn't be too difficult to simulate?

http://www.progarchives.com/album-reviews.asp?id=44375

This album has got 3 reviews
1 x 5 star rating by member
1 x 5 star rating by collaborator
1 x 4 star rating by collaborator

On top 52 ratings.

This gives an overal average is only 3.81, which is very low for PA standards and especially so if you look at the appreciation in the reviews.



Back to Top
aapatsos View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal and Heavy Prog Teams

Joined: November 11 2005
Location: Rotherham, UK
Status: Offline
Points: 7212
Post Options Post Options   Thanks (1) Thanks(1)   Quote aapatsos Quote  Post ReplyReply Direct Link To This Post Posted: October 28 2014 at 14:29
Important not to set too many barriers for young members joining as said by Sonia. Progressively Wink, the site has tried to dis-incentivise the use of rating-only and I think we are getting there. Maybe some fine tuning is still required.
"Prog Heavy Petting" show - every Monday 7-9pm UK time on http://www.justincaseradio.com, the first progressive radio in Greece
Back to Top
odinalcatraz View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: July 12 2010
Location: London
Status: Online
Points: 236
Post Options Post Options   Thanks (1) Thanks(1)   Quote odinalcatraz Quote  Post ReplyReply Direct Link To This Post Posted: October 28 2014 at 08:04
Originally posted by Argonaught Argonaught wrote:

Originally posted by Kati Kati wrote:

The main fact here, we should take in consideration that this site needs a constant flow of activity and interest by newcomers to survive in order to stay at the top. Unfortunately the bogus ratings manipulation equally discredit the site.
However, I think it's important that we not make it even more difficult for newcomers to join, this is detrimental to the wellbeing of the site, in fact maybe we should facilitate this even more for newcomers to join and give more incentives for them to participate, however in terms of ratings I too think this method is being abused, maybe as a suggestion newcomers should only be able to rate albums after they have written at least 2 reviews?
There is no simple solution, we know this unfortunately, nor do I know how to fix this problem :( bah 
Hugs Hug

^I completely agree, and I am glad that you said this (because no-one here will listens to me) LOL I really want to see rating abuse stamped out .. preferably through prevention rather than playing whack-a-mole with someone who signs up under 15 different names. 

Anyone interested should be able to join, and to begin making good faith contributions to the discussions. Everyone knows that forum posts are personal opinions only, and takes them as such. But the PA average ratings are different; they either represent-, or appear to represent the prevalent opinion of the PA community as a whole. Thus, when people post their reviews and ratings, they de-facto speak on behalf of, well, everyone here. It's not unfair to ask them to wait until they are qualified to do so. No-one's gonna die. If they get upset and quit because they can't slap an album with a bunch of one-star ratings under different names, good riddance.  


Ha ha!!
I gave up getting involved in all this but I LOVE the "Whack-a-mole" comment. Brilliant!
Some one told me he didn't even need an email address to sign up. Isn't that a bit odd? No site has that weakness these days I thought.
Back to Top
Angelo View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: May 07 2006
Location: The Netherlands
Status: Offline
Points: 11704
Post Options Post Options   Thanks (1) Thanks(1)   Quote Angelo Quote  Post ReplyReply Direct Link To This Post Posted: October 27 2014 at 19:11
I reported a few more tonight. Not sure if anything can be done though - we devised the rating calculation to be such that ratings only count with a far lesser weight than reviews, and these are outweighed by collaborator reviews. Usually, that works, and it may be less work than removing the trolls. However, after 8 years I also tend to feel its about time to send out a bit of a stronger signal here...
Even prog is rooted in the blues, at some point...
Back to Top
Argonaught View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: June 04 2012
Location: Virginia
Status: Offline
Points: 739
Post Options Post Options   Thanks (1) Thanks(1)   Quote Argonaught Quote  Post ReplyReply Direct Link To This Post Posted: October 25 2014 at 16:53
Originally posted by Kati Kati wrote:

The main fact here, we should take in consideration that this site needs a constant flow of activity and interest by newcomers to survive in order to stay at the top. Unfortunately the bogus ratings manipulation equally discredit the site.
However, I think it's important that we not make it even more difficult for newcomers to join, this is detrimental to the wellbeing of the site, in fact maybe we should facilitate this even more for newcomers to join and give more incentives for them to participate, however in terms of ratings I too think this method is being abused, maybe as a suggestion newcomers should only be able to rate albums after they have written at least 2 reviews?
There is no simple solution, we know this unfortunately, nor do I know how to fix this problem :( bah 
Hugs Hug

^I completely agree, and I am glad that you said this (because no-one here will listens to me) LOL I really want to see rating abuse stamped out .. preferably through prevention rather than playing whack-a-mole with someone who signs up under 15 different names. 

Anyone interested should be able to join, and to begin making good faith contributions to the discussions. Everyone knows that forum posts are personal opinions only, and takes them as such. But the PA average ratings are different; they either represent-, or appear to represent the prevalent opinion of the PA community as a whole. Thus, when people post their reviews and ratings, they de-facto speak on behalf of, well, everyone here. It's not unfair to ask them to wait until they are qualified to do so. No-one's gonna die. If they get upset and quit because they can't slap an album with a bunch of one-star ratings under different names, good riddance.  


Edited by Argonaught - October 25 2014 at 16:54
Back to Top
Kati View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 10 2010
Location: Mozambique
Status: Offline
Points: 2337
Post Options Post Options   Thanks (0) Thanks(0)   Quote Kati Quote  Post ReplyReply Direct Link To This Post Posted: October 25 2014 at 15:00
The main fact here, we should take in consideration that this site needs a constant flow of activity and interest by newcomers to survive in order to stay at the top. Unfortunately the bogus ratings manipulation equally discredit the site.
However, I think it's important that we not make it even more difficult for newcomers to join, this is detrimental to the wellbeing of the site, in fact maybe we should facilitate this even more for newcomers to join and give more incentives for them to participate, however in terms of ratings I too think this method is being abused, maybe as a suggestion newcomers should only be able to rate albums after they have written at least 2 reviews?
There is no simple solution, we know this unfortunately, nor do I know how to fix this problem :( bah 
Hugs Hug
When playing strip poker, make sure you are wearing a lot of clothes! or google will ban you ;)
Back to Top
Bonnek View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal Team

Joined: September 01 2009
Location: Belgium
Status: Offline
Points: 4144
Post Options Post Options   Thanks (0) Thanks(0)   Quote Bonnek Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 14:37
Originally posted by aapatsos aapatsos wrote:

Originally posted by Dean Dean wrote:

Originally posted by Bonnek Bonnek wrote:



Why not the RYM system?
Your ratings don't count till your account is 'somehow' proven not to be a troll, rating manipulater or belonging to a mentally 3year old.

I admit there's a grey zone as to which criteria they apply to establish that but I expect just like here mods apply common sense rules to make such decisions.


I agree Karl.
During this time I analysed the effects of ratings abuse and the effectiveness of rating manipulation and came to the counter-intuitive conclusion that it doesn't make a great deal of difference. The current review and collab weighting out weighs the rating-only score and that is more effective than deleting 1000s of bogus ratings. The problem is that the perception of this abuse is that deliberate manipulation makes a big difference - you can look at any well respected album and see lots of 1 and 5 star ratings and that looks bad. Perception is everything since the number of votes we get for any album is too small to have any statistical value or meaning.



Great points Karl and Dean. Before I went through this, I thought "just how much of a problem is it really?" and your post gives the answer. This needs to be SPELT OUT somewhere on the site for artists/members/abusers to understand along with a note that checks are performed etc.

A previous idea of making the rating's value even smaller than it is now compared to reviews helps towards this direction. Perhaps not the ultimate solution, but an improvement.


Yes I remember that analysis of how the trolls basically sabotage other trolls, so it's futile, but it's a nuisance and indeed it looks bad on us.

- Another advantage RYM has is that they have significantly more ratings
- And compared to PA it looks like the number of ratings has a greater weight then the actual rating (it's an impression, i have not checked the actual calculation).
- Also, they have half stars which avoids somewhat the overabuse of the 5 stars

Anyhow, not much we can do about the number of ratings, but it all balances out their ratings much better.



Edited by Bonnek - October 22 2014 at 14:39
Back to Top
aapatsos View Drop Down
Special Collaborator
Special Collaborator
Avatar
Prog Metal and Heavy Prog Teams

Joined: November 11 2005
Location: Rotherham, UK
Status: Offline
Points: 7212
Post Options Post Options   Thanks (0) Thanks(0)   Quote aapatsos Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 11:35
Originally posted by Dean Dean wrote:

Originally posted by Bonnek Bonnek wrote:



Why not the RYM system?
Your ratings don't count till your account is 'somehow' proven not to be a troll, rating manipulater or belonging to a mentally 3year old.

I admit there's a grey zone as to which criteria they apply to establish that but I expect just like here mods apply common sense rules to make such decisions.


I agree Karl.
During this time I analysed the effects of ratings abuse and the effectiveness of rating manipulation and came to the counter-intuitive conclusion that it doesn't make a great deal of difference. The current review and collab weighting out weighs the rating-only score and that is more effective than deleting 1000s of bogus ratings. The problem is that the perception of this abuse is that deliberate manipulation makes a big difference - you can look at any well respected album and see lots of 1 and 5 star ratings and that looks bad. Perception is everything since the number of votes we get for any album is too small to have any statistical value or meaning.



Great points Karl and Dean. Before I went through this, I thought "just how much of a problem is it really?" and your post gives the answer. This needs to be SPELT OUT somewhere on the site for artists/members/abusers to understand along with a note that checks are performed etc.

A previous idea of making the rating's value even smaller than it is now compared to reviews helps towards this direction. Perhaps not the ultimate solution, but an improvement.
"Prog Heavy Petting" show - every Monday 7-9pm UK time on http://www.justincaseradio.com, the first progressive radio in Greece
Back to Top
SteveG View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: April 11 2014
Location: usa
Status: Offline
Points: 2031
Post Options Post Options   Thanks (0) Thanks(0)   Quote SteveG Quote  Post ReplyReply Direct Link To This Post Posted: October 22 2014 at 10:54
I have an interesting question. Since I rarely review albums that I don't like, except for the sake of completing some review sites where many albums are unrated such as later day Roy Harper material, do my 4 and 5 star ratings look amiss? I generally check my reviews against those of a collaborator when possible and we are usually on the same page. But again, I don't relish reviewing inferior material unless I absolutely have to.


Edited by SteveG - October 22 2014 at 10:57
"If you don't know who the 13th Floor Elevators are, then you don't know music"-Jerry Moss.
Back to Top
 Post Reply Post Reply Page  123 24>
  Share Topic   

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 11.01
Copyright ©2001-2014 Web Wiz Ltd.

This page was generated in 0.094 seconds.