The Anonymous Brewers Speak: Rating The Raters

anonbrew2aFrom Alan: Recently I was contacted by a brewer who wondered if he, too, could write for A Good Beer Blog. Sure, no problem I thought. If Knut and Travis can, why not a craft brewer? But the brewer wanted to do it under the cloak of anonymity. I wavered. I wondered. I let it go for a while. Brewers usually stay silent like the one to the right. Then, quite a while later, unbeknownst to the first, I got a message from another brewer a world away asking for exactly the same thing. I knew then that there was a venue needed. A way for brewers to share what they really felt. So, from time to time they, too, can post here and share their thoughts. This is the first, a message from someone I will call Brewer A.¹ Please feel free to comment as you would in response to any post.

Well, how to get started? Sites like R(H)atebeer.com are a thorn in the side for many brewers. They are dominated by a handful of posters that don’t reflect the opinion of the general public. As with most critics they go off half cocked and I think often fully pickled. They pretend to know grain and hop varieties that they feel were used in a certain beer. I have seen the same poster rate the same beer twice in the same day and give it very different reviews. Hiding behind the mask of anonymity (like I am now) instills false bravery into these fellas (mostly boys but not all.) I have witnessed raters backing up a certain opinion to follow later in the same paragraph with “but I have not tried it yet.”

These raters looking to increase their numbers will will gather at fests to collect single mouthfuls of a new beer in the same way they once collected mint condition action figures. No need to engage the brewer or enjoy the beer for the sake of it – just get “Han Solo in the original packaging” and never open it up.

This involves further discussion. Maybe nine RateBeer guys and I could split a six pack and talk.

¹Stan’s point is excellently made: it’s Secret Brewer XJ17 from now on.

One thought on “The Anonymous Brewers Speak: Rating The Raters”

  1. [Original comments…]

    beerinator – April 12, 2008 10:15 PM
    http://beermapping.com
    Uh oh.

    anon (from Denton, Texas) – April 12, 2008 10:30 PM
    pissing in your bottles and serving them hot would probably net a better rating.

    agreed (from Lewisville, Texas) – April 12, 2008 10:46 PM
    I’m so sick of beer geek raters that rate a beer based on IBUs or alcohol percentage. (the higher the better in their mind) I’m guessing they need such a strong beer because they don’t have a developed enough palette to appreciate a good ordinary bitter or the like. As a home brewer with more than 100 brews under my belt I respect the brewer for putting themselves out there. Any twit can drink a beer and comment, only a handful can make a beer and get it to market. If you are in the former category, keep your pie whole shut and quit assuming the world gives a damn as to what your feeble intellect has to say about what your underdeveloped palette detects.

    Disagreed (from Mississauga, Ontario) – April 12, 2008 11:24 PM
    “As a home brewer with more than 100 brews under my belt I respect the brewer for putting themselves out there. Any twit can drink a beer and comment, only a handful can make a beer and get it to market. If you are in the former category, keep your pie whole shut and quit assuming the world gives a damn as to what your feeble intellect has to say about what your underdeveloped palette detects.”

    As a homebrewer with less than 20 brews under my belt I have to cringe when some twit with more money than skill gets to bottle and sell a creation that is less tasty than something that even I can produce with my limited skill. I’ll ignore the irony of you feeling that the world gives a damn what your feeble intellect has to say about the issues. Brewing is tough, so is running a business, but even harder is admitting that maybe your product is less amazing than you think it is.

    Buttle (from Pittsford, New York) – April 12, 2008 11:38 PM
    They are dominated by a handful of posters that don’t reflect the opinion of the general public.

    Unless you brew for A-B, this could be very easily thrown back in your face.

    I’m not unsympathetic to the criticisms of ratebeer. Just remember, though: if you’ve ever reviewed a book, CD or movie at Amazon or a similar site, then on those topics, you’re the equivalent of the ratebeer amateur.

    Alan – April 12, 2008 11:45 PM
    [I am just adding locations from the IP addresses for something to do. Not really making a point. Maybe it’s the NASCAR on the TV making me do it.]

    Stephen (from Mt. Laurel, New Jersey) – April 13, 2008 5:19 AM
    Ratebeer’s sample size is large and minimally biased. Against the few illegitimate or incompetent reviews stand scores of helpful, often insightful reviews by experienced drinkers and even brewers themselves. The wealth of constructive criticism offered by good reviewers would be considered an asset by anyone with a mind to improve his product – are you really that sensitive about a string of negative ratings? Did another boy push you off the see-saw?

    Stéphane A (from Meudon, France) – April 13, 2008 5:56 AM
    http://www.ratebeer.com/View-User-14765.htm
    There’s nothing new here. The same has been debated ad nauseam about wine, books, films, art and essentially any creative field. It is an accomplishment for brewers to be recognized as part of this community. The sooner they can accept it, the sooner we’ll all realize there’s no point in pursuing the argument. Accept it, many people will continue reviewing from a single sip. I’m not endorsing these practices (see my RB profile), all I can suggest is to be happy that people are so passionate about beer.

    anoym (from Denmark) – April 13, 2008 8:16 AM
    The only problem with having fair and balanced ratings on these rating sites is that most the people who rate there are favoring the artisan type brewer which tends to appeal more to their tastes. Where as a regular type beer like a pilsener is shunned as a mass marketed type.Even if its a well made example if it is too boring they asociate that with low quality, where as the opposite is true for the high rated beers many are full of defects and just made by a brewer with too big of an ego and looking for attention by going over the top extreme.
    There really is no reflextion of how well a beer rates to how well it sells.
    Its all about the ego, brewers and just about every artist out there has a big one and wants it to be stroked by others…Better to have a thick skin and ignore the bad reviews and be happy fo the good ones.

    Alan – April 13, 2008 9:39 AM
    Being one of three people who know who this is, you can be assured there is no point imagining this is a bad brewer or an egomaniac. Why would I give any bit of my bandwidth to the unworthy?

    The lack of self-reflection in response so far has been remarkable, too. My interest in hosting such posts, in addition to giving a venue to those unable to speak freely, would be to encourage reflection. As you know, I still have my Howard the Duck comics (aka “my precioussses“) in the plastic bags I put them into 32 years ago. I see that connection in my life – which is why I started the “Your Stash Is Not A Vault” campaign way back when.

    Stan Hieronymus – April 13, 2008 10:16 AM
    http://appellationbeer.com/blog
    Alan,

    I thought Andrew Mason at Flossmoor Station had trademarked Brewer A . . .

    OK, seriously. A couple of wine blog posts I read in the last couple of days, from Dr. Vino and Eric Asimov got me thinking how lucky, in balance, people seeking flavorful beers are that Rate Beer and Beer Advocate exist.

    I’m reminded by Lew Bryson’s recent call for some other term for “beer geeks” and his suggestion “beer fans” would be good. I think the analogy works. Fans don’t always have 20-20 vision, many are know-it-alls, they get drunk (sometimes with beer, others with emotion) in the course of action, the yell stupid things, etc.

    They are passionate. And, in this day of million-dollar salaries, sometimes it feels like they care more than the players.

    I wouldn’t argue that much of what Brewer A (can we change the name to B and leave A to Andrew?) writes is incorrect, although it is overstated. He points right off to things that bother me (such as evaluating beer on a quick sip, although I do that myself when it’s not for print).

    But I’m really bothered by calling it HateBeer.com. It’s just the opposite.

    Alan – April 13, 2008 10:41 AM
    I was thinking about the 007 numerical system but “A” and “B” and however many others there may be will have to do. “B” is already taken, by the way – that brewer just waiting for the proper moment I guess. But, on reflection, maybe I need to make it Secret Brewer XJ17 for greater clarity. I had forgotten about Andrew, a sad thing given that they have been prize offering supporters around here. “B” might be Secret Brewer LQ93.

    I’ve heard “hatebeer” before from other brewers so I would not want to stifle that choice of language – it is, however, a broader comment than just one forum but “Beer Schmadvocate” does not have the same resonance. Remember the concluding words of that old Canadian parable: I cannot control the power of rhyme anymore than I can tell the birds not to shit on my car.

    Like you, I also do not share that depth of unhappiness – but I am not this brewer. That may be the point. While it is true I was the one who asked “Do We Love The Beer Or Brewer?, I like and admire the craft brewer not as artist or celebrity so much as the struggling successful small business person. Where are the Bread Advocate or RateCheese forums that include absurdly misallocated and thoughtless trashings of those products, putting risk to livelihood? Maybe the point is to measure one’s words before we hit “post”, to think of Robbie Burn’s poem “To a Louse”:

    O wad some Power the giftie gie us
    To see oursels as ithers see us!
    It wad frae mony a blunder free us,
    An’ foolish notion:
    What airs in dress an’ gait wad lea’e us,
    An’ ev’n devotion!

    For at the end of the day, there is nothing in this hobby so odd as the place of unhappiness – not to mention judgment (in the theological sense.) Focusing on the pleasures is so much more interesting.

    John (not from Yonkers but from Connecticut) – April 13, 2008 12:34 PM
    I think it is just as big a problem that some brewers get offended by a handful of reviews and forget that ratebeer is a hobbyist site like any other. It’s not easy to make something and find out that some people think it is bad and even harder when someone finds it bad just because it is not some uber rare barrel aged top 50 beer but the same thing happens in any hobby.

    This is me. No anonymity here:

    http://www.ratebeer.com/View-User-17111.htm

    I’m a typical user of Ratebeer.com and can assure you that I enjoy a good pilsner just as much if not more than an imperial stout just as many others do.

    Chewbacca (from Toms River, New Jersey) – April 13, 2008 1:26 PM
    Brewers don’t get offended by ratings on Ratebeer and BeerAdvocate. The majority realize that it is for the most part a highly biased un-professional rating scheme. Look at the top “rated” beers on either site, they’re all “extreme” beers and beers that are hard to find. Of all the styles in the world, the best is the one you can’t get. That is sad.

    The major problem with Ratebeer and BeerAdvocate is that bar owners and retailers have started using them as a “source” to find good beer. If you start choosing your beer selection at your pub based on what Ratebeer tells you as opposed to what you think, that’s a dangerous game.

    John (from Connecticut) – April 13, 2008 2:15 PM
    “The major problem with Ratebeer and BeerAdvocate is that bar owners and retailers have started using them as a “source” to find good beer. If you start choosing your beer selection at your pub based on what Ratebeer tells you as opposed to what you think, that’s a dangerous game.”

    That’s largely due to the fact that there is an increasing demand for craft beers but no one out there to help educate these people on what a good craft beer is. It’s no coincidence that in my area (lower New England), that the vast majority of bars that once only stocked Budweiser, Coors Light, Heineken, and Amstel Light are seeking out 2-3 craft beer taps to add to this. In most instances, they are turning to Sam Adams and Brooklyn Brewery as both breweries have been proactive in educating these people. Many others are looking to the good regionals in the area as they also spend the time to talk to the bars and educate both the bar owner and consumer.

    It’s no secret that the other source for information: the distributors are not knowledgeable. One of my good friends who owns a bar had a distributor tell him that Blue Moon is an excellent example of a Belgian tripel and is best served with a lemon. With statements like that being the norm, it’s no wonder bars are turning to sites like ratebeer and beeradvocate for information.

    Boak (from London, England) – April 13, 2008 2:49 PM
    http://www.boakandbailey.com
    Interesting post, and interesting set of responses. My first feeling is – what’s all the fuss about? I never use Ratebeer or Beer Advocate to make decisions as to what beers to buy. I may be under the influence of other prejudices – what the label looks like being the classic one, or previous experiences from that brewer.

    But perhaps I’m unusual in this? Have there been cases of microbrewers stymied at the first hurdle because of bad reviews? This thought has put us off publishing negative reviews of beers from small producers – if we don’t like it, we move on. But I can see this applying much more to pub and restaurant review sites than beer sites – after all, most beer geeks are sufficiently completist to want to try the beer for themselves, no matter how much they’ve heard it sucks.

    Finally, Bailey adds that – “I would never base my opinion of a beer on what was said on one of these sites – but all the beers in the top ten are bloody good.” If enough people get stuck in, the balance works in the end. Look at Wikipedia.

    Knut Albert Solem (from Norway) – April 13, 2008 3:05 PM
    http://knutalbert.wordpress.com
    “The major problem with Ratebeer and BeerAdvocate is that bar owners and retailers have started using them as a “source” to find good beer. If you start choosing your beer selection at your pub based on what Ratebeer tells you as opposed to what you think, that’s a dangerous game.”

    I don’t know about that. Do you have good alternatives? What other databases do you have that are truly global? What about beer styles you don’t particularly like yourself?

    I would like excamples of high quality beers that are scoring low on ratebeer. Sure, there are beers that gets ratings all over the scale, but in my experience thay usually have problems with their quality system and have problems turning out a product of consistent quality.

    The ratebeer people I have met around Europe are wildly enthusiastic about beer, and spend a lot of time and money seeking out new brews. This enthusiasm drives demand, too, opening up the market in countries like Denmark and Sweden where you can get beers from hundreds of breweries – imports and domestic – where a decade ago you’d be lucky to find a dozen.

    I use ratebeer a lot when travelling, and there is simply no alternative to help me find pubs, breweries and beers. I give my own input, and am happy to contribute both opinion and knowledge.

    Are there biased, bad and sloppy reviews? Sure, but the positive sides are, IMHO, much stronger than the negative ones.

    But, of course, there is a tough market out there. And the extree beers get more attention. So if you want to impress the world with a well crafted lager with subtle flavour, you have an uphill fight, and you’ll have to convince lost of people why they should buy your brews…

    Stan Hieronymus – April 13, 2008 3:24 PM
    http://appellationbeer.com/blog
    To Knut’s question about if there are any alternatives.

    The disclaimer here is that Ray Daniels is a friend of mine, but I think his Cicerone program is a great start. If you can find a bar, restaurant or retail store where the establishment a) goes to the trouble to learn was is good, b) takes good care of it and c) thus makes good recommendations you are set.

    John (from Connecticut) – April 13, 2008 3:30 PM
    Err, why am I showing up as from Yonkers, NY? That’s a good hour from where I live in Connecticut.

    [Al the Ed.: Thanks! for this one I am addresses based on IP. I can’t actually drill down to your house but I will change all Yonkers to Connecticut.]

    Alan – April 13, 2008 3:58 PM
    For my two cents, I think there are a few problems with any aggregating ratings system for any group or topic:

    • ranking: by definition, any rating system seems to have to rank. Yet it is not clear to me that this is required or even informative. Is a beer that is rated in the 20% percentile really four times worse than one in the 80%? The resulting relative positioning is a false assurance imposed through the numerical hierarchy compounded by averaging.
    • weighting: not all reviewers rate in the same way. You see this in purchasing committee work. People who rate different proponents have personal scales. Some will never award over 80% and others never below 50%. This challenges my understanding of the utility of averaging.
    • subjectivity and context: science has proven a lighter lager is better on a hot day. I will prefer it then to the weight of the Baltic porter that is perfect in December after tobaggoning. No subjective experience can be stripped of one’s context during the experience. But there is no way of including that and it compounds the difficulty of obtaining understanding – especially when the further layer of averaging is applied.
    • reduced focus on pleasure: beer is good. It is a social lubricant, a comfort and a release. Taste is intertwined with memory. These and other factors are not capturable by numerical statistics. The are also individual, making averaging a deviation from experience rather than a harmonizer.

    These are different points from those of Secret Brewer XJ17 but I they speak to my concerns. That is one reason I was happy to see the BAers move to the letter system but I would prefer to see a radical shift to an icon system of rating which had different sliding scale category ranges from “simple” to “complex”, “share with friends” to “hide for self”, “good fun” to “ponder for hours”, “safe dark beer for newbie” to “no one will understand you” – the sort of icon charts you see for holiday resorts, taking a photo or automobile characteristics. Throw in icons for what foods they go with, too.

    I would also like to see ratings factor in price as, for me, this obvious consumer consideration relative to other purchases is seemingly a taboo for rating craft beer. For me, an overly priced beer relative to quality should rank lower compared to a lower price one which exceeds expectations.

    John (o’ CT) – April 13, 2008 4:27 PM
    Why should price factor in to one’s enjoyment of a beer (or anything for that matter)? If the brewer uses expensive grains and adjuncts, the price will follow suit in all likelihood. I don’t know too many brewers who would make a beer and be willing to take a loss on it just so they can compete with a much larger scale brewer who benefits from bulk buying and smaller overhead.

    As far as the letter system or a star system or a creative limerick system or whatever other system, it is just a different way of showing the same thing. Personally, I think the letter system was about the worst thing the Alstroms could do to their site as it makes them look like a Wine Spectator clone rather than a site dedicated to beer. They had a good numerical rating system, then went ahead and tossed it in favor of a system many in the wine world consider to be majorly flawed.

    On the numbers, if Secret Brewer XJ17 is basing his comments on whatever the numerical values assigned to his beers are, I challenge him/her to take some time, actually read the reviews and see what people are actually saying in them. There is a reason why a user must make comments in order to log a review and if there are 150 reviews and 130 of them are saying the same thing, maybe it’s time to revisit the recipe of whatever that beer is and make a few tweaks to it. If this particular brewer is a small local brewer who sees about a dozen reviews of even his/her flagship beer, then why even consider them at all? A small scale brewer in all likelihood knows his/her clientele already as they frequent their establishment and caters to them. Not a dozen beer geeks.

    This would be a lot easier to respond to if said brewer came forward and replied directly, even if they want to remain anonymous. It’s hard to address whatever points said brewer is making without having a clear picture of what they are.

    Alan – April 13, 2008 4:41 PM
    Of course, price is relevant. Otherwise you are accepting the taboo. Note this post about one traditional kriek and the observations on a 40% price swing between three almost identical brews.

    And I expect XJ17 is brewing or enjoying the day. Another point that is implicit in his comments is the lack of value in these sorts of debates compared to the pleasures and obligations of life.

    John (CT-centric) – April 13, 2008 5:04 PM
    “And I expect XJ17 is brewing or enjoying the day.”

    As am I. Right now, I have my kolsch boiling away and kegging my maibock. 😉

    Alan – April 13, 2008 5:13 PM
    Sweet. I do not brew as much as I should but I bottled my ESB last night and took well deserved naps today.

    brewer a – April 13, 2008 6:18 PM
    http://flossmoorstation.blogspot.com
    It’s easy to say, (as a brewer), that you take the ratings with a grain of salt but it’s a lot harder to practice it. Even though at Flossmoor we get very positive reviews I still get frustrated with the push towards extreme beers getting highest ratings. We made a pretty damn good zwickel-bier and it’s score was a 2.9/5. I thought it was pretty dead on, and about as authentic as we could make it without dispensing it from a cask. Darron Welch from Pelican told us it was dead on, but the RB ratings said,

    “I don’t think I’ve had this type before but it was like any other boring, yellow fizzy lager. Hazy, dark yellow pour with a foamy, long-lasting white head, canned corn and grass in the nose, bland and uninteresting flavor, dry finish. Yawn.”

    “The educational mantra behind deciding to make this batch 500 is a neat idea, but I’m not sure this is a style that deserves much hype.”

    Other comments were that it was an easy drinking sessionable beer or okay but nothing special. I didn’t exactly cry myself to sleep but the point remains that certain styles are pretty much never going to get the love on ratings sites.

    I think it’s kind of the difference between the way that Michael Jackson would write about beers vs. the way the ratings sites work. MJ would talk about how a certain beer was the perfect example of a style and rate on it’s merits instead of picking out the faults. Sometimes you could read one of his reviews and not even realize he didn’t think the beer was any good. Different strokes for different folks I guess.

    John (still of CT) – April 13, 2008 7:59 PM
    I think you outgeeked the beer geeks on that one. 😀

    Looking at the rating history, there were several knowledgeable reviews of that beer and all of them reviewed the beer favorably. One guy rated at a festival, so that can be taken with a grain of salt. The rest of them probably haven’t quite gotten there yet. When it comes to lesser known styles that a lot of these guys have no real knowledge of (and it is difficult to have a knowledge of what makes a good zwickel without having been to Germany), they tend to be dismissive. With further sampling, many of those guys will probably come to appreciate beers like the one you refer to. Don’t take it the wrong way.

    “I think it’s kind of the difference between the way that Michael Jackson would write about beers vs. the way the ratings sites work. MJ would talk about how a certain beer was the perfect example of a style and rate on it’s merits instead of picking out the faults. Sometimes you could read one of his reviews and not even realize he didn’t think the beer was any good. Different strokes for different folks I guess.”

    Yes, it would definitely be nice if more people borrowed from MJ’s philosophy and were more diplomatic in their reviews. Unfortunately, a lot of people don’t consider the fact that the brewers do sometimes read reviews on sites like Ratebeer and/or may be on the young side (I’ve noticed at gatherings and festivals that the majority attending tend to be in their early-mid 20s).

    FWIW, a lot of people in the Ratebeer community are fed up with the way the top 50 is calculated. It’s not only brewers who get annoyed when they look at a top 50 dominated by extreme beers.

    JerC (from Toronto, Ontario) – April 14, 2008 12:04 AM
    Technical difficulties kill my big long post, oops.
    Summary:
    1) Not everyone on a rating site is an expert, we post what we think, just like your customers think what they think about your product regardless of their level of experience.
    2) Brewers, you need to realize that people were saying the exact same things about your products before they started posting it on the internet, you can just see it now.
    3) If you spend more time on sites like ratebeer its not hard to find out what they think the best examples of the more subtle styles are. Or who the people that love them are.

    This post, a recent blog about how we are “too judgmental”, the post on ProBrewer (‘Rate the raters!’) make me ponder the nature of criticism. Also, see Point #1 above – I can critique your product simply by having money and tastebuds and talk about it with others. Posting online is only the next level.

    Flossmoor: The ratings you quoted aren’t exactly stellar examples, but its a rare style (i personally have had only 2 out of over 2800 beers rated), that is fairly subtle by nature. Personally, I would often rather have a tasty flawed pale ale, than a textbook perfect pilsener because I personally don’t care for the flavour profile of the latter. That said, I would rather read a review that leaves me with an impression of what they person thought of it rather than an ambiguous technical analysis. Especially since without knowing the personals level of expertise, asking them to rate it as it fits the style will only invite the kinds of mistakes noted in Alan’s blog post above. I can’t tell you if your zwickel fits the style, I *can* tell you if I like it, and what I think about it…

    Nathan (from Sydney, New South Wales) – April 15, 2008 9:58 PM
    http://www.beerguide.com.au
    I found your link through Brew Poll (www.brewpoll.com) and it is an interesting topic. I probably will not have a lot to offer, but I’ll throw in my experiences and position on sites like BA, RB and the rating/reviewing of beer online.

    As a ‘beer geek’ as many people above have called us I reguarly rate beers which are available on the Australian market on our own website in the vein of the RB or BA sites mentioned above. Do I profess to be the most educated person on beer – far from the truth I am but a humble beer drinker who knows what I like and do not like given my years and home brewing experience.

    I personally am very enthusiastic about beer and its a field I’m constantly researching and surrounding myself with (eg: home brew, beer festivals etc..). Unfortunately websites like our’s (RB, BA etc..) will always have bias unhelpful results by some reviewers and we’ll probably always step on the toes of a brewer or upset a retailer here and there. Remember how the saying goes though, if you can’t handle the heat get out of the kitchen. If you think every beer you brew will be perfect and loved by everyone, than you’ve either mis-understood your potential market or you’ve just not performed enough pre-sales testing to make sure your product and/or business is market ready. Beer is subjective and like everything in life everyone will always have a differing opinion and some people will definately not like a particular brand or style for their own personal reason/s – to me I think this is the best thing about beer!

    Whilst our site is very young and our market limited, our goal is not to acknowledge winners or losers in the field but to let fellow Aussies learn what is on the market that they may be missing out on. We do this through a weighted numerical scoring system accompanied with comments by the reviewer. A common comment I hear is “I’ve never heard of this beer before” after which they typically go looking for it to have a taste themselves.

    Are we doing the right thing by numerically scoring a beer that is exceptionally subjective (ie: weather, personal taste, price)? Maybe not, maybe there is a better way? But one thing is for sure the number of emails we receive daily indicates that the average Australian drinker after having visited our website is no longer just a beer drinker, but a drinker who is willing to have a shot at some of the lesser known products brewed by the craft-brewers. Beers which they may not have in the past due to fear of wasting their hard earned money. No one wants to pay for a beer or a case of beer which had they done a little reasearch prior to purchase would have found that 9/10 drinkers with a similar palate to their own reviewed this particular beer poorly. Equally true is the opposite, people are prepared to purchase a beer if 9/10 people with a similar palate all recommend a certain beer.

    The result of our site, others in Australia and the constant media coverage on the craft brewing field is that the Australian market is now being expanded with more and more options popping up all the time – which like our form of website or not, I believe is a winning situation for us in Australia given our infancy in craft beer.

    Lastly like yourself Alan (McLeod) I believe a single sip is by no means a reflection of any beer. You must savour at least a few mouthfuls before you can even appreciate the complex flavours and warmth a beer can provide, typically I use a bottle or two and I only rate a single beer in any one session. Thankfully on our site we have not had to deal with the issue of ‘single sip’ reviewing yet. But it is always something in the back of my mind on how to prevent or in the least reflect this in the ‘score’ at the time of review.

    Thank you for an interesting topic and thank you to who ever posted it on Brew Poll else I would not have seen it.

    Jeremy (from East Lansing, Michigan) – April 16, 2008 4:28 PM
    I agree that Michael Jackson had a great way of writing a review that was not particulary damning to the beer. But I think Michael Jacksons goal was slightly different than the goal of ratebeer.com.

    I have one of michael Jacksons books, and it is really fantastic. But, it didn’t help me too much when I was just getting into craft beer and I had $10 to spend at my local store that carries over 600 beers. Ratebeer steered me toward certain beers back then, and still does today. I have to say that 300 beers later, I can’t say that Ratebeer has steered me wrong too many times.

    While I agree that we need to remember that there is a brewer behind each beer, there should also be room to remember that brewers are selling a product, and there are certain beer products that are not worth buying. Are their flaws to the system? Of course. And we should keep the discussion alive. But sweeping comments that incriminate thousands of the most loyal beer fans should probably be avoided. There should be a balance to the discussion, illuminating both the postives and the negatives of such rating sites. When we are posting comments for everybody to see online, we need to excercise balance and discretion. The same balance that raters could benefit from, might also benefit those who blog their thoughts online.

    Ratebeer Guy (from Breckenridge, Michigan) – April 16, 2008 4:57 PM
    As a member of Ratebeer for 6 years now, I have sampled and rated well over 1,500 beers in that time. Does this make me a beer expert? Someone once said, “there are no beer experts, only beer drinkers with opinions.” Although I agree with this statement to a certain degree, I admit that there are technical aspects of beer and brewing that are quite conducive to varying degrees of expertise. I might not be able to correctly identify the hop or malt variety used in every beer I drink, but that does little to detract from my ability to tell you whether it’s a good beer or not. To that end, I believe I am well more qualified than the general public to know what’s a quality beer and what’s not.

    The anonymous brewer’s comment, “They (Ratebeerians) are dominated by a handful of posters that don’t reflect the opinion of the general public”, is a good thing. The general public doesn’t know what a good beer is, whereas the majority of raters on Ratebeer do. The opinions of the general public may reflect the types of beer you might want to brew to make the most money, but catering to those opinions will lead to brewing an inferior product. My experience has been that good brews love Ratebeer for what it has done for the craft beer movement and that bad brewers hate it because they don’t like the poor ratings their beers get on the site. I’m guessing I know what camp anonymous brewer falls into.

    Alan – April 16, 2008 6:39 PM
    The general public doesn’t know what a good beer is, whereas the majority of raters on Ratebeer do.

    See, I can’t agree with that. That would me a well made pale ale or ordinary porter which is entirely accessible somehow bad because it was more popular than something rare and experimental. Both are expressions of what they are supposed to be.

    And your guess is entirely misplaced: good brewer. From what I have seen in this discussion, it may be that it is actually fans of rating forums who are the ones who have issues with criticism.

    chris_o – April 16, 2008 7:30 PM
    http://www.ratebeer.com/View-User-6832.htm
    The vast majority of the beer-drinking general public here in the UK would not know what an ordinary porter is, much less what constitutes a well-made example.

    Wilson – April 17, 2008 9:09 PM
    http://www.brewvana.wordpress.com
    “From what I have seen in this discussion, it may be that it is actually fans of rating forums who are the ones who have issues with criticism.”

    You hit the nail on the head, Alan. With regard to this discussion, and many others. Cheers!

    Matthew – April 18, 2008 8:51 PM
    http://theeremite.com
    I think my reply got moderated. I’ll just say that Boak had a good point way up there. Rating sites strike me as similar to wikipedia. The point of the sites aren’t to be the most accurate source commercial beer style guidelines. It’s user opinions. It’s supposed to be subjective, just like Wikipedia isn’t supposed to be moderated. And, all things considered, they both work pretty well at what they do… most of the time.

    ALan – April 18, 2008 10:04 PM
    Matthew – if your comment didn’t show up I apologize. I get about 50 spam comments a day and the filters are set pretty much at “HIGH!!!” and it may be in the net.

    kp (from Atlanta, Georgia) – April 22, 2008 6:37 PM
    http://www.ratebeer.com/View-User-5356.htm
    I’m the guy you’ll see at a beer fest standing off to the side and taking notes while rating a 4oz sample of beer. I’ll have my nose stuck in the glass seeing how much aroma I can get from the beer. I’m holding it up to a light source to check clarity. I taking sips and notes and sips and notes as I evaluate just how much I like the product. I spend just a few minutes with the beer before checking it off my hit list and rushing off to the next table to try the next new beer.

    But the whole time I’m going through my drawn out routine to decide how much I like this beer, I’ll see several members of the “general public” get their 4oz sample, take a sip, say “yuck!”, and dump the rest. Maybe with enough practice I too can achieve this enlightened quick decision method.

    Alan – April 22, 2008 8:29 PM
    Ha ha! Nothing like seeing the other side of a discussion. Maybe with enough practice you might work up to a third of a pint.

    kp (from Atlanta, Georgia) – April 23, 2008 11:44 AM
    A 4oz festival pour *is* 1/3 of a “pint” considering the number of places that use 14oz shaker glasses with a finger of head.

    If you don’t think a 4oz sample is enough to fairly judge your beer, why bring it to a beer fest where hundreds of people will be exposed to small samples and get the wrong impression. And certainly do not pay to send it to any professional contests like BJCP, World Beer Cup, etc, where panel of judges work from 4oz or smaller samples in flights of a dozen or more similar beers to determine if yours is worthy.

    But don’t take my word for it. I have a simple experiment you can do yourself. Go into a bar and order a (gasp) full pint of beer. Drink 3/4 of it. Then try sending it back saying it is not good. If the bartender asks why you drank 3/4 of it before deciding to send it back, explain to them that it took you that long to realize it was bad. Try this at a few different bars and then report back to me how well that works for you.

    Alan – April 23, 2008 12:16 PM
    No, I won’t be engaging with that. As with your non-reflection.

    Stonch – April 23, 2008 9:10 PM
    http://stonch.blogspot.com
    kp, the elaborate and drawn out process you describe doesn’t convince at all. You can’t judge a beer from a tiny sample. You’re kidding yourself.

    John – April 25, 2008 12:28 AM
    “You can’t judge a beer from a tiny sample. You’re kidding yourself.”

    Guess I’d better tell all the competition judges out there that their method of judging is inaccurate since when they judge a competition, they’re judging on less than that. Better throw out all those GABF and World Beer Cup medals too, brewers.

    Al – April 27, 2008 8:53 PM
    http://hop-talk.com/
    I think beer rating sites suffer from the same thing that any collaborative, amateur site/system/group does, and no more than any other either.

    Systemic bias.

    Digg, Slashdot, Wikipedia, et cetera all have the same issue.

    Wikipedia, for example. The type of people who are drawn to contribute there are geeks. Is it any wonder, then, that the article for Star Trek is longer than the article on The Enlightenment or the article on Gary Gygax is longer than the article on Ban Ki-moon.

    The people who care enough to share their opinions on beer at these rating sites are going to tend toward people who are trying to make their beer-drinking experiences as varied as possible. People who aren’t interested in trying lots of new beers wouldn’t care to take the time to participate, and anyone who likes industrial-brewed American light lagers probably doesn’t even know–or care–that such sites exist. It doesn’t surprise me in the least that there is a tendency toward negativity for industrial-brewed beers and beer that appear to be similar in style/flavor profile/color/etc.

    Certainly not everyone who participates in these sites is like that. If you look at reviewers individually you probably wouldn’t see such bias. But, taken in aggregate the tendency is there.

    That’s my view anyway. I tried, for a time, to rate beers as I tried them, but it just didn’t work for me. For one thing, I generally try to pick things up that I know I’ll like, so rarely was a rating low. For another, it skewed my thinking. If I rate this beer 3 and that one 3½, do I really like that one better than this one? Do I like it less than the one I gave a 4 last week? Do I like this one exactly the same as another I gave a 3 to a month ago? Too much of a pain.

    One other thing before I wrap up: I’m a big NFL fan. I love my Giants and Sundays in the fall are sacrosanct. Have you ever listened to a sports-radio call-in show or even just a bunch of fans in a bar? My golly, if professional football players took what these fans said to heart they wouldn’t be able to get out of bed in the morning. And, for all their griping about how bad this guy is or that team is, they still watch every week. Maybe brewers should be more like professional athletes.

    Alan – April 27, 2008 9:20 PM
    What? Like the Bengals?!?

    Al – April 28, 2008 11:51 AM
    http://hop-talk.com/
    Well, not in the I-have-millions-of-dollars-and-no-moral-compass way, but in the I-appreciate-the-fans-that-come-out-to-see-me-play-but-the-one-or-few-that-vociferously-disparage-me-don’t-mean-anything way.

    Of course, if the world made any kind of sense, teachers would make more than professional athletes.

    Troy – June 2, 2011 12:28 PM
    gasp! people post negative, uniformed opinions on the internet?!?

    And other people are upset about that?!?

    whu’da thunk it.

Leave a Reply

Your email address will not be published. Required fields are marked *