Like the other judges in the thread said, it's not science. It IS very tough to get right. If YOU want to wake up early on a Saturday morning, drive an hour to get there before 9am, and have sampled a dozen or so Stouts and Porters before noon, then please, take the test and help us out. God knows we could use you.
My best advice to those receiving score sheets is to NOT dwell on the absolute number, but really its the RANGE. If I give it a 32 and the other judge gives it a 34, then we are in agreement that it's a decent beer, probably one or two minor flaws, and/or lacking in some aspects that would make it a standout. Examples of those factors might include things such as balance out of whack or a bit too thin or a bit too estery or wrong hopping levels, or something like that. In the 40+ levels, it's a DAMN good beer, definitely a contender to win BOS. In the teens or low 20's, its got some problems. 25 to about 30 are the pretty typical average home brews with a couple apparent faults. 31-35 - decent, slight flaws.
The little guide that gives the ranges is how you need to consider the scores. That's what I (and all the other judges I've ever judged with) go for, then the numbers may be tweaked a little just to ensure that the right final placement happens within the flight. Honestly, I think the upper-middle range beers right around 35 give or take are the toughest to score. They are fine, well made, very solid, enjoyable beers. Not a whole lot wrong with them. Usually it's the subtle intangibles that tend to differentiate those from the "holy shit that's really fucking phenomenal" ones. They simply aren't the VERY BEST examples of the style.
Split flights that go to mini BOS's also cause TONS of confusion. A really solid beer that scores a 36 in one half of the flight goes to mini-BOS and comes in second behind another one that scored a 35 in the other half. Sorry folks. Both are really good, solid beers, need some little bit of a tweak to push them up to awesome, but the lower scoring beer had something about it that the mini-BOS judges deemed better. They don't know whose beers they are, so they aren't out to screw anybody. One just comes across as better than the other, and that's how they end up being placed relative to each other, contrary to the numerical scores.
The other thing is that I try to do my damndest to write down all that I perceive, have the score match appropriately, and then try to offer guidance in the form of things to concentrate on, such as "work on reducing ester levels, possibly by controlling fermentation temps or maybe using a different yeast strain". I try my best NOT to assume what the brewer used or the process he followed, so you'll most likely find lots of question marks and "possibly's" in my feedback. I do this hoping that it jogs the brewer's thought process into finding the solution more than it serves as THE solution. Everybody knows not to boil with the cover on, so telling them "don't boil with the cover on" doesn't help. Instead, I'll write something like "Work on reducing your DMS (vegetal/cooked corn) levels. Maybe a longer boil or faster wort chilling might help?"
I also always add my email address (usually using the BJCP stickers - much easier and more legible than writing them all out) so that the brewers can contact me later if he wants to. So far, not a single person has. I sincerely hope that's a good thing and not a bad thing. PLEASE send me an email if you think I totally fucked something up or if you can't read my shitty handwriting. I try to do a good job, but I freely admit that I'm very far from perfect. Send the judge an email. That's why he puts it on the score sheet, or at least that's why I include it.
Well, I hope this demystifies it a LITTLE bit for somebody. At least, you might be able to understand this is how I frame myself when I judge.
I'm flattered. Glad you get something out of 'em
Sergeant, BN Army
