Forum Home Forum Home > Site News, Newbies, Help and Improvements > Help us improve the site
  New Posts New Posts RSS Feed - Tendency to 5 star rate....?
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Topic ClosedTendency to 5 star rate....?

 Post Reply Post Reply Page  <12
Author
Message
jimmy_row View Drop Down
Forum Senior Member
Forum Senior Member


Joined: July 11 2007
Location: Hibernation
Status: Offline
Points: 2601
Direct Link To This Post Posted: May 28 2008 at 21:45
my $0.25 (adjusted for inflation)
 
 
I would favor the "normal curve" approach, and maybe for those who want it, some statistical analysis of personal ratings...simply because I know that when I start putting up more reviews, it will be almost a competition to be stingy.  In other words, to have a lower average rating than most reviewers.  So I agree with Finnforest.  With the normal curve, 5-star and 1-star ratings are quite rare, and 3-star reviews are the most common which makes sense once you've heard a lot of albums.  I understand perfectly the tendency to rate/review albums you LIKE, naturally...but after some time it should level off, and some criteria should be established to level off the ratings.  If you find yourself giving 5 stars to half of your reviews, then I think you need to become more tight-fisted and analytical.
 
 
 
edit:  I'm also a big fan of the half-star rating (I believe Ivan was shot down repeatedly...?).  That 4.5 option could relieve a lot of pressure from the full 5 and encourage people to give more thought to their rating as Jim said.
 
 


Edited by jimmy_row - May 28 2008 at 21:49
Signature Writers Guild on strike
Back to Top
micky View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: October 02 2005
Location: .
Status: Offline
Points: 46843
Direct Link To This Post Posted: May 28 2008 at 21:49
that is really the way to do it...  of course my .25 cents is worth half that since I do my ratings differently than many.. and if I don't want to have too many 5 star reviews.. I review the albums that are not essential in my opinion.. and review stuff that people might find interesting rather than the same old sh*t that has been reviewed to death.. as most true 5 star albums usually are.
The Pedro and Micky Experience - When one no longer requires psychotropics to trip
Back to Top
jimmy_row View Drop Down
Forum Senior Member
Forum Senior Member


Joined: July 11 2007
Location: Hibernation
Status: Offline
Points: 2601
Direct Link To This Post Posted: May 28 2008 at 21:52
^ I like that approach too, and I can see myself doing that just looking at my "project list" of reviews... a lot of stuff with under 15-20 ratings that will get 3 well deserved stars from me.
Signature Writers Guild on strike
Back to Top
russellk View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: February 28 2005
Location: New Zealand
Status: Offline
Points: 782
Direct Link To This Post Posted: May 28 2008 at 22:00
Originally posted by Finnforest Finnforest wrote:

Originally posted by russellk russellk wrote:

That was a swift capitulation, Finnforest! But you are an excellent case in point: I barely notice what you award an album as I'm always taken by what you have to say.

Some reviewers only review what they love, and they will issue a preponderance of five-star ratings. Others will review a wide range and their ratings will be more balanced. I think we're a long way from Amazon.



Russ, this topic just makes me so g'damn angry which is why I need to stay out of threads about it.  I disagree so profoundly with this advancing disregard of the ratings.  If they truly mean nothing to the majority of folks, then lets have Admin disable it and be done with it.  Just have reviews without ratings. 
Maybe that would really be the way to go in the future.  All reviews, no more ratings.

As mentioned, the star ratings on Amazon are useless because of 5 star syndrome.  They will be here too if we don't have common parameters,and continuing the drum beat of "stars don't matter, only the review does" gets us there quicker.  Do you guys really mean this honestly?




No, I don't mean to say stars don't matter, just that they are less important than the meat of the review. This is precisely because of the differences in approach even between the prog reviewers - one scatters five-star reviews liberally about the place, another is stingy. Also, the overall rating of an album reflects a great deal more than its merit. Some albums are denigrated because the're 'not prog' ("I'd give this album four stars on a metal site but only two stars on this site" is a common example) or are slapped with one-star ratings to 'counter the fanboys' (DT's 'Train of Thought' for example). In the end you have to read the review and take the overall rating of many albums with a grain of salt.

That said, it's great when people think carefully about how to rate an album. I try hard to come up with a meaningful rating for each review I do.
Back to Top
Slartibartfast View Drop Down
Collaborator
Collaborator
Avatar
Honorary Collaborator / In Memoriam

Joined: April 29 2006
Location: Atlantais
Status: Offline
Points: 29630
Direct Link To This Post Posted: May 28 2008 at 22:10
I actually may be giving too many 4 starts these days.  But I absolutely don't waste my reviewing something I'd consider 1 star.  I have seen some falling into a trap of writing a little too much about their rating rather than more about the music.  Something I've been guilty of, too.  But then there's my pet peeve, people who give something half a star and round down.  .5 rounds up mathematically, dagnabbit!

Edited by Slartibartfast - May 28 2008 at 22:14
Released date are often when it it impacted you but recorded dates are when it really happened...

Back to Top
tuxon View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 21 2004
Location: plugged-in
Status: Offline
Points: 5502
Direct Link To This Post Posted: May 28 2008 at 22:14
I always wanted to just rate whatever you know to create a difference between the good and the bad, but unfortunatly the importance of written text and therefor reviews is regarded more important than opinions on what's there.
 
so I stopped carring about reviews and ratings as it's only a small fraction of albums that are known get reviewed and rating only is not appreciated.
 
I would love to just rate what I know, but if I do I get accussed of being lazy, while in fact I can rate 20 albums a day and can only review 10 albums a month, and I know 3k albums, so I can't review them all in this lifetime, but I can rate them all within a year.
 
choices.
I'm always almost unlucky _ _ _ _ _ _ _ _ Id5ZcnjXSZaSMFMC Id5LM2q2jfqz3YxT
Back to Top
Raff View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: July 29 2005
Location: None
Status: Offline
Points: 24439
Direct Link To This Post Posted: May 30 2008 at 02:46
I have been 'guilty' of giving a lot of 5-star ratings to the albums I have reviewed, though the majority my ratings are actually 4-star ones. However, in defense of my choice, I could say that in giving the top rating to an album I don't necessarily take my own taste into account (though of course it factors in, and very relevantly too), but also the album's importance in the grand scheme of prog things, or (in the case of PP/PR albums) its influentiality and/or likelihood to appeal to a prog fan. This is why, for instance, I gave 5 stars to albums I don't consider perfect such as Selling England by the Pound or Pawn Hearts.
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21790
Direct Link To This Post Posted: May 30 2008 at 02:58
Originally posted by tuxon tuxon wrote:

I always wanted to just rate whatever you know to create a difference between the good and the bad, but unfortunatly the importance of written text and therefor reviews is regarded more important than opinions on what's there.
 
so I stopped carring about reviews and ratings as it's only a small fraction of albums that are known get reviewed and rating only is not appreciated.
 
I would love to just rate what I know, but if I do I get accussed of being lazy, while in fact I can rate 20 albums a day and can only review 10 albums a month, and I know 3k albums, so I can't review them all in this lifetime, but I can rate them all within a year.
 
choices.


You're very welcome to rate 3k albums on my website ... there the ratings and tags are equally important as the reviews.Big%20smile
Release Polls

Listened to:
Back to Top
rushfan4 View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: May 22 2007
Location: Michigan, U.S.
Status: Offline
Points: 66755
Direct Link To This Post Posted: May 30 2008 at 10:03

To second Mike's post, I have about 500 ratings on his site, but they are mostly just ratings, with some with quick comments, versus full scale reviews.  Granted most of my ratings are good or better, I do throw in a mediocre or worse on occasion. 

While on PA, I have only done 20+ reviews, because I try to do these as full scale reviews, which takes more time and effort, and more "intimacy" with an album.  The one thing that I have tried to be fairly consistent with on my reviews on PA is to base my star ratings on this site's guidelines and I generally explain that in my conclusion paragraph.  I suppose this probably shouldn't be necessary since the star rating should say exactly that, but I find in reading the reviews that sometimes the ratings seem to conform more to a 1 to 5 scale with one being the worst and 5 being the best, rather than what the stars are designated to mean on this site.  Based on my interpretation of the site guidelines, a prog related album should never have a 5 star rating because how can a non prog album be a prog rock masterpiece.  On Mike's site, I can give Led Zeppelin albums 10 ratings, but on PA, the best I can give would be a 4 star.  This doesn't mean I don't think it is a masterpiece, but it is not a prog rock masterpiece.  If it were a prog rock masterpiece, then the band should not be included in the prog related category.  On the flipside, the same thing applies to 1 and 2 star ratings.  2 star ratings are for collectors and fans only.  With my reviews, depending on the words this could be a good thing or a bad thing.  For example, I gave Rush's Feedback 2 stars here, but a 9.0 on Mike's site.  Yes, I am a Rush fanboy, and I love that album.  It is a fun album.  But as good as I think it is, it still is only something that I feel would appeal to fans of Rush and collectors of Rush.  I suppose I could have given it 3 stars for good, but not essential but it just didn't feel like it applied.  On the other hand, I have given both Opeth's Blackwater Park and Meshuggah's Catch 33, 2 stars.  I really can't stand either of them, because of the growls, etc., but I can't say that they are poor and for completionists only, but to me I think that they are definitely for fans only.  (Which again is kind of a catchall for most acts, since obviously if you are a fan than yes, their albums will probably appeal to you.)  My point being that both what I consider a good album and what I consider bad albums both received 2 star designations, but the meaning of these 2 star ratings was differentiated by what I had to say within my reviews.  On the other hand, the Fantomas album just got 1 star because I found it to be the most awful rubbish that I ever laid ears on, but I suppose technically I am contradicting myself because based on what I just said, it does appear to appeal to fans of that band, so I should have gave it 2 stars.  I'm not going to change it though, because it was just that horrible for me. 
Back to Top
Gamemako View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: March 31 2006
Location: United States
Status: Offline
Points: 1184
Direct Link To This Post Posted: May 30 2008 at 20:45
I was actually about to make a new thread about this problem when I saw this one.

I think we should have an average rating and a normalized rating so we can really compare (because the average rating overall is probably about 4).
Hail Eris!
Back to Top
debrewguy View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 30 2007
Location: Canada
Status: Offline
Points: 3596
Direct Link To This Post Posted: May 31 2008 at 01:17
Yeah, the easy answer - don't rate albums you love at 5 stars. Think of what others might think, where they might situate it, what your friends rate it as ; then write th review that others would.
Hey, it's a personal thing. Who am I to tell you, for whatever reason, that you are wrong ?
I make it a game to insert inane comments as to how Klaatu & Ange are underrated. Do I complain/whine/execrate menacing posts because most of PAers don't agree (yet). Not yet, any way.
The true beauty in a review, or true truth (what a whatever you call something that uses the same sounds , blah blah) is in the words, the experience, the enthusiasim, or lack of, in the writing.
Heck, read a fanboy's review, and do you go  " I gotta get that Sepultura's album" automatically  ? NO, you (hopefully) have more self control (and self-esteem) than that.
So don't worry about the grades, man. Could it be that most here have enough of a smidgen of intelligence to read between the lines ????

"Here I am talking to some of the smartest people in the world and I didn't even notice,” Lieutenant Columbo, episode The Bye-Bye Sky-High I.Q. Murder Case.
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21790
Direct Link To This Post Posted: May 31 2008 at 05:30
Originally posted by Gamemako Gamemako wrote:

I was actually about to make a new thread about this problem when I saw this one.

I think we should have an average rating and a normalized rating so we can really compare (because the average rating overall is probably about 4).


What exactly do you mean by "normalized" - a rating which does not regard the prog status and/or genre?


Edited by MikeEnRegalia - May 31 2008 at 05:30
Release Polls

Listened to:
Back to Top
Gamemako View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: March 31 2006
Location: United States
Status: Offline
Points: 1184
Direct Link To This Post Posted: May 31 2008 at 08:29
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

Originally posted by Gamemako Gamemako wrote:

I was actually about to make a new thread about this problem when I saw this one.

I think we should have an average rating and a normalized rating so we can really compare (because the average rating overall is probably about 4).


What exactly do you mean by "normalized" - a rating which does not regard the prog status and/or genre?


As in, normalized such that the average rating is 3.0  and it is all moved along the average. Thus, the highest-rated album will become a 5.0 and the lowest a 1.0 and the vast majority of albums will be between 2 and 4.

To me, ratings as a whole should fall in a bell curve -- 70% between 2 and 4. Only the notably good or notably poor get above 4 or below 2, and only the rare exceptions achieve 5 (See: Perfect Element Part 1) and the utter sh*t obtain the coveted 1s (see: Beyond Sanctorum).

//EDIT: Out of curiosity, what is the absolute average of ratings at PA now?


Edited by Gamemako - May 31 2008 at 08:29
Hail Eris!
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21790
Direct Link To This Post Posted: May 31 2008 at 08:32
^ I agree on the bell curve - *if* all users would rate all albums there are. In reality users tend to rate the albums they like best, so the bell curve will gravitate towards 4/5. It's unavoidable ... after all, we can only rate albums we know well, and we don't spend a lot of time listening to stuff we don't like.

Still don't understand what you mean by normalized though ... maybe what is done currently at PA with the weighted average calculation?
Release Polls

Listened to:
Back to Top
Peter View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: January 31 2004
Location: Canada
Status: Offline
Points: 9669
Direct Link To This Post Posted: May 31 2008 at 08:45
This is an old topic here, so I'll briefly make my old point:
 
Remember, our reviewers are music fans, not professionals, so we are reviewing albums from our own collections. Since most folks like most of the CDs they own (we tend to buy albums we already know we like, or can expect to like), highly favourable reviews will naturally predominate.
 
Plus, as noted in the first post, there is the understandable human tendency to want to share that which gives us great pleasure.
 
it will always be this way. When you really like an album, the only real choice in rating it is four or five stars.
 
Of course, i have long wanted PA to drop the numeric ratings altogether, but that's not going to happen (the vast majority find ratings useful), and it's really another issue in any case.
"And, has thou slain the Jabberwock?
Come to my arms, my beamish boy!
O frabjous day! Callooh! Callay!'
He chortled in his joy.
Back to Top
Finnforest View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: February 03 2007
Location: The Heartland
Status: Offline
Points: 17506
Direct Link To This Post Posted: May 31 2008 at 09:53
Originally posted by Peter Peter wrote:

This is an old topic here, so I'll briefly make my old point:
 
Remember, our reviewers are music fans, not professionals, so we are reviewing albums from our own collections. [not true for many of us] Since most folks like most of the CDs they own (we tend to buy albums we already know we like, or can expect to like), highly favourable reviews will naturally predominate.
 
Plus, as noted in the first post, there is the understandable human tendency to want to share that which gives us great pleasure. [or to provide accurate information for fellow music buyers]
 
it will always be this way. When you really like an album, the only real choice in rating it is four or five stars. [this is so wrong.  i really like many albums i rated 3 stars, even some 2 star ones, but they are those ratings for a reason...because I follow the sites ratings descriptions....a good album can be not essential or even "for fans."]
 
Of course, i have long wanted PA to drop the numeric ratings altogether [agreed now that I know how people feel], but that's not going to happen (the vast majority find ratings useful)[they're only useful if they are attempting to follow the Admin suggestions in good faith] , and it's really another issue in any case.




To someone else's point that people can read between the lines and spot faulty rating....good point.  I hope its true the majority of time but I think many people have been tricked by inflated ratings.  I know I have...so I guess I'm stupid.  LOL
If you love cats, please adopt an older cat. They've been in a cage far too long, and they long for a home.
Back to Top
Peter View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: January 31 2004
Location: Canada
Status: Offline
Points: 9669
Direct Link To This Post Posted: May 31 2008 at 10:20
Originally posted by <strong><FONT color=#ff0000>Finnforest</FONT> Finnforest wrote:

Originally posted by <FONT color=#009900><strong><FONT color=#0000ff>Pe</FONT>ter</strong></FONT> Peter wrote:

This is an old topic here, so I'll briefly make my old point:
 
Remember, our reviewers are music fans, not professionals, so we are reviewing albums from our own collections. [not true for many of us] (I was generalizing, obviously, but i still believe it's true for the majority of PA users who review. And how do you know that about so many others?Confused Are you illegally exchanging MP3s?Wink) Since most folks like most of the CDs they own (we tend to buy albums we already know we like, or can expect to like), highly favourable reviews will naturally predominate.
 
Plus, as noted in the first post, there is the understandable human tendency to want to share that which gives us great pleasure. [or to provide accurate information for fellow music buyers] (Well yeah, obviously, but that 'accurate" is problematic, due to the personal, subjective nature of our response to art, and taste.)
 
it will always be this way. When you really like an album, the only real choice in rating it is four or five stars. [this is so wrong.  i really like many albums i rated 3 stars, even some 2 star ones, but they are those ratings for a reason...because I follow the sites ratings descriptions....a good album can be not essential or even "for fans."] ("So wrong?" You are aware, of course, that when i wrote "really" I did not mean "genuinely"  or "in reality" but the secondary meaning of the word, as in "very much; to a high degree.' I like some 3-star CDs too, but they are not the ones I really like.)
 
Of course, i have long wanted PA to drop the numeric ratings altogether [agreed now that I know how people feel], but that's not going to happen (the vast majority find ratings useful)[they're only useful if they are attempting to follow the Admin suggestions in good faith] (yes, as I always did) , and it's really another issue in any case.




To someone else's point that people can read between the lines and spot faulty rating....good point.  I hope its true the majority of time but I think many people have been tricked by inflated ratings.  I know I have...so I guess I'm stupid.  LOL
 
Wow -- I really didn't think my post was so far off the mark, or so contentious! ShockedConfused
 
Wink
 
BTW, Finny, I have NEVER liked those "one size -- or five -- fits all" textual descriptors which accompany the ratings. They are just too limiting, and too open to interpretation. Effectively, they force me to use someone else's description in MY review. I have always been more inclined to view the ratings as percentages, or A - F "grades" (though I still tried to work within them, in most cases -- unless I specified otherwise, in my review).Smile


Edited by Peter - May 31 2008 at 10:59
"And, has thou slain the Jabberwock?
Come to my arms, my beamish boy!
O frabjous day! Callooh! Callay!'
He chortled in his joy.
Back to Top
Finnforest View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: February 03 2007
Location: The Heartland
Status: Offline
Points: 17506
Direct Link To This Post Posted: May 31 2008 at 10:49
Pete, no need for the blackeye smiley, I wasn't rude to you.  Just countering/debating where needed.  As long as someone is trying to follow the descriptions in good faith and recognizes the importance, which you say you do, I have no problem. 
If you love cats, please adopt an older cat. They've been in a cage far too long, and they long for a home.
Back to Top
Peter View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: January 31 2004
Location: Canada
Status: Offline
Points: 9669
Direct Link To This Post Posted: May 31 2008 at 10:59
^ No sweat, FF -- consider it replaced. Thumbs%20Up
 
I thought, though, that you were a trifle -over-analytical about some of my points, which still seem to me to be broadly applicable, and pretty much common sense. Fan site = lots of high ratings. Predictable, & unavoidable.
 
If I worked for a music magazine, I'd be sent lots of crap albums for free (and PAID to listen to them, & review them), and of course I'd rate them accordingly.
 
I simply don't own very many CDs I don't like. My music purchases are almost always informed ones.
 
And again, i have always had problems with trying to work within those ratings descriptors -- I find them to be much too prog-specific, and too limiting. I really dislike them (though I know why they are there).
 
Take care, pal.Smile


Edited by Peter - May 31 2008 at 11:05
"And, has thou slain the Jabberwock?
Come to my arms, my beamish boy!
O frabjous day! Callooh! Callay!'
He chortled in his joy.
Back to Top
Finnforest View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: February 03 2007
Location: The Heartland
Status: Offline
Points: 17506
Direct Link To This Post Posted: May 31 2008 at 11:24
I think you hit on an important point there.  I do take this more seriously than perhaps I should, or than others do.  I'm not here to simply rate my favorites that sit on my shelf.  I buy, borrow, and steal (kidding) lots of other sh*t to review that I will never own, for the purpose of being a widely versed reviewer and music afficianado.  I am certainly not interested in reviewing only titles that I like and I find that premise very stagnant and counter-productive.  While Italy is a clear favorite, one look at my pie chart will show I make some attempt to be diverse. I may not be professional in the sense of being paid, but I'd like to think I provide professional quality content all the same (as do many of our fine Collabs). 

So when people are flippant or disregarding of the site's core importance (which to me rating recklessly certainly is), I take it more personally than I should.  If we don't maintain high quality reviewing and rating, why would we be here, and why would others read us with so many other choices out there.  Many of you say we're in no danger of that happening, but it  starts by the casual diminishing of the importance of standards. 

I still maintain that "loose" reviewing and rating lowers our quality and relevance, so I'll have to push back on this as long as I remain here.  If this makes me an elitist ass to some, I'll have to live with that tagWink
My intention is only a better site. 
If you love cats, please adopt an older cat. They've been in a cage far too long, and they long for a home.
Back to Top
 Post Reply Post Reply Page  <12

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.141 seconds.
Donate monthly and keep PA fast-loading and ad-free forever.