Comments on: Pitchfork // A Statistical Look at Their Ratings

By: Tom

Tom — Fri, 11 Nov 2011 04:39:03 +0000

Thanks for the interesting article! That must have been some hard work. Very interesting!

By: tony

tony — Tue, 14 Dec 2010 13:13:47 +0000

Pretty ridiculous assigning grades and numbers to music in the first place. Maybe read the review to get an idea of what the music offers. Other than “oh look, this album offers me 7.6 utils of satisfaction” and then refusing to read the actual review.

By: Ricky

Ricky — Wed, 20 Oct 2010 07:17:53 +0000

I was really happy to come across this article because it articulated for me a few things that I sensed by regularly reading pitchfork but couldn’t prove. Good work!

On an unrelated note…I think it’s great that pitchfork hands out their BNM tags based on feel, on instinct, rather than scores. It is true that well rated albums will be left out but it’s their ranking system to begin with. They aren’t saying ‘this album scored highest’, but they are saying that they know their audience, their core demographic and they think that the average pitchfork reader will like this. I’d be opposed to this system if they didn’t recommend me fantastic music on a regular basis. I disagree with them probably 25% of the time but they get me moreso than any other music magazine out there.

It is for people like me that p4k chose to leave certain metal albums off the BNM list and while I can appreciate that is unfair, I thank them for it anyways.

By: AB

AB — Wed, 04 Aug 2010 00:09:31 +0000

I don’t have any qualms with albums getting a rating and then appearing in a place on their year-end list that doesn’t quite correspond with that initial rating. Feelings change. If you rate an album released in February within its first week after release, there’s still 9-10 months in the year to re-play that album. It may sound silly, but something like a change in season can really sway the perception of an album.

I think people read too much into ratings and reviews as a whole. In the case of P4K, people are especially harsh. They have their reasons, I suppose. But the way I view these things is this: I don’t need any entity to “tell me something is cool”. But I’m also a busy dude. I appreciate having different outlets — P4K being just one of them — that monitor this stuff and slap a number on it. These sites are like a musical GPS for me. Sometimes I get to my destination and agree that its great, other times I get there and wish I hadn’t bothered. But I appreciate having things brought to my attention that I might have otherwise missed.

By: Sputnikmusic - Pitchfork and the “hive-mind” effect « Staff Blog

Fri, 30 Jul 2010 02:07:46 +0000

[...] begets predictability, though, and then things like this happen. That’s fine when Pitchfork reviews underground/unknown music (which is still its [...]

By: Cam

Cam — Wed, 05 May 2010 13:52:44 +0000

Always thought this review was strange:

http://pitchfork.com/reviews/albums/12454-the-59-sound/

It’s a pretty good album and was well-hyped in other circles, but I was certainly surprised to see it buried as a 2nd or 3rd review on the page AND receive an 8.6 rating. That’s got to be one of the highest without scoring a BNM. I believe back when this was reviewed Pitchfork was also using that strange “recommended” tag as well… in any case, interesting reading.

By: Sanjay

Sanjay — Tue, 16 Mar 2010 03:41:53 +0000

I always find it funny when people knock p4k. All they are are a bunch of people with opinions on music. It just so happens, if you don’t like p4k, that your opinions and tastes differ than those of p4k. I just treat p4k like a friend who’s into music and has more knowledge than I do. They offer suggestions, I listen and then agree or disagree. It just so happens I tend to agree with them more often than not. We just have similar tastes. This became readily apparent when I could easily guess their scores of a CD based off my own feelings for the same CD and usually come within a few tenths or them. So I trust p4k but they aren’t the end-all to music and they have never tried to be.

By: Curt

Curt — Thu, 04 Mar 2010 15:01:50 +0000

Chris / Cale: I remember vividly what happened with the Sufjan record. There was never a bad review that was retracted. What you see on the site now is the original review with the original score. What happened, which supports one of the major points in this analysis, is that the record was reviewed positively and about two weeks later, when the editor heard it, it was given the “Best New Music” tag. The review with the added tag was sitting there front and center on a Monday morning.

By: P4K FAN

P4K FAN — Thu, 04 Mar 2010 09:29:16 +0000

Thanks, I really love statistics and pitchfork, so this was perfect!

By: TAL

TAL — Wed, 03 Mar 2010 18:07:26 +0000

Thanks for the insight and taking the time to code all of this up. As someone who does data analysis this raises a lot of questions. Interesting to me is, as you point out, the propensity to rate a record as a whole number. It is odd that, despite Pitchfork’s desire to use a quasi-continuum for their rating (e.g. 7.1,7.2,etc.) their reviewers tend to subconsciously return to intervals, which reminds me of some political science work on “feeling thermometers” where people tend to place themselves on the 10s. At any rate, this screws up the variance if we want to do some modelling. Lots of hypotheses to test here after you convince someone to collect more data. Again, thanks.