Wine Competitions: It Won a Medal – It Must be Good!

By
[ 12 ] Comments
Share




winningwineMy first-ever restaurant review also included reviews of some of the wines. And though we do not usually write negative reviews of the wine we receive as samples, we do occasionally include reviews of less-than-stellar wines that we pay for. This is especially true when the restaurant has been hyped in all the local press; the chef has been touted as the Best Sommelier in the city, and we really didn’t think much of the pairings. If a venue is going to use awards for marketing and as criteria for long lines and upper-end prices, they had best not take issue with the accompanying scrutiny that goes with it. The same can be said about wine competitions, medals and awards.

A local wine writer took issue with that review and used it as a basis for personal attack. The winery owner was alerted. Rather than use it as an opportunity to change the writer’s mind, the winemaker piled on with the kind of visceral acrimony I haven’t seen since our days of running a college football message board. It got to the point where he was “spamming the site” with a list of the awards his wine had won.

And that got me to thinking, just what do these awards mean, and how should our readers use them when choosing wine?

Research Shows No Concordance in Awarding the Gold

A research study published in a recent issue of the Journal of Wine Economics found an interesting aspect of wine competitions that might be disturbing to the aforementioned winemaker as well as to consumers who choose their purchases solely based on medals and ribbons: A gold medal in one competition does not mean a gold medal in another.

In “An Analysis of the Concordance Among 13 U.S. Wine Competitions,” author, researcher and winemaker Robert Hodgson analyzed 4,000 wines entered in 13 wine competitions in the United States. Hodgson who is also Professor Emeritus, Department of Oceanography, Humboldt State University, found that of the 2,440 entered in three or more competitions, a little less than half (47%) of them received Gold medals. But about 8/10 (84%) of these same winners received no Gold in another competition. And his findings go a step further:

There were 3,347 wines entered in two or more competitions. Considering the case when the results of the competitions were perfectly concordant, i.e., when a wine received the same score in every competition entered, we find that there were only 132 cases. However, virtually all of these occurred when the wine was entered in just two competitions. For wines entered in more than three competitions, there were no straight Golds, out of 3,400 entries. There were 2,440 wines entered in more than three competitions. Of these, there were 1,142 Gold medals awarded (47%). Of the 1,142, 957 (39%) also received at least one No Award. Of the remaining 185 Gold awards, 165 received a low score of Bronze in one or more other competitions. Thus, 98 percent of Gold medal wines were regarded as just above average or below in another competition.

Wine Competitions are All About Marketing

I have always found competitions (in any industry) a bit suspect for reasons both anecdotal and evidential. First and foremost, my years in marketing and public relations give me an insight that the average rust-specialist-turned-wine-writer does not have. Competitions are all about marketing, and you typically are not going to send your product to multiple competitions where it’s going to come up a loser. You’re going to spend your money on entry fees that guarantee at least some sort of “honorable mention.” Sure you can write the entry fee and the cost of the product off as marketing expenses on your books, but the goal is to come back with something you can use on your website or in your product literature.

Anecdotally, I have tasted wines that were deemed worthy of award that were positively awful – and not in the way that “fills one with awe.” One in particular tasted like it was fermented in Grandma’s Cedar Chest; another like it came from a box of Toni Home Perm, and yet another like that bottle of White Zinfandel we all had at age…well…less than legal. And then there were others that I, as well as recognized experts had thought very good to fabulous placing equally or losing to the same “winning wines.”

Competitions even use this to entice wineries to enter. A website tab called, “Marketing Benefits” practically promises skyrocketing sales from entering their competition:

When you reach the top, word travels quickly. Just ask Adam LaZarre, winemaker Hahn Estates, or Dan Kleck, owner/winemaker at Silver Stone Wines. Each of their wines received top honors at the Los Angeles International Wine & Spirits Competition, and within days, each of their wines sold out.  The marketing value from the competition can have a tremendous impact. We partner with Ralphs Grocery Company, a major Southern California grocery chain and sponsor of the competition, as well as offer marketing and promotional collateral to retail and specialty stores and restaurants touted our award-winners.

Now personally, I like Hahn Estates’ wines, and I haven’t had Silver Stone Wines. Their awards are probably well deserved. But being able to download winner bottle seals and shelf talkers doesn’t just go to the Best of the Best – and the promise of a potential sellout is pretty alluring.

Judging Wines is Inconsistent and Highly Subjective

Secondly, wine tasting is subjective and judging is inconsistent. You get 6 judges in a room, and even if they are sextuplets with the exact same palate, exact same wine tasting experience, had exactly the same amount of sleep and taste all the wines in the same order – you may still not get the identical ranking of wines.

Judges’ palates and experience, the number of wines tasted, and a particular wine’s order in the competition is going to affect the wine’s perceived worthiness of winning.

In another study, “An Examination of Judge Reliability at a major U.S. Wine Competition,” Professor Hodgson found judging in the nation’s oldest wine competition to be inconsistent. In the 2003 California State Fair commercial wine competition, judges had no idea that one of the wines in each of the three flights given them was identical. During the first two flights, they rejected the wine as undeserving of any kind of award. However, during the third flight, they unanimously agreed that it warranted a gold medal.

I talked to a number of people working for and in the wine industry to get their take on the meaning of a Wine competition medal. Nobody really wanted to go “on the record” but I did receive permission to quote some of them anonymously.

“There are some European competitions that I find are worthwhile, but by and large I find that the winners are always the middle of the road wines that please the palate of a judge that has just gone through 100 rough, insipid, and uninspired wines. Once they find a wines that is “passable” it all of a sudden becomes a Gold Medal Winner!” said one quite familiar with the process.

I was particularly interested in the number of winners versus total entrants. For example, judges tasted 3,400 wines from 931 wineries in the 2009 Los Angeles International Wine & Spirits Competition.  There were a total of 116 Best of Class Awards, 404 Gold Medal Awards, 943 Silver Medal Awards and 1,030 Bronze Medal Awards. Nearly 75% of all wines entered (2,493) won awards. So your entry fee of $85 ($75 if you got it in early) and cost of 6 bottles of your wine gives you nearly a 75% chance of winning something. If Joe had those kinds of odds for the Texas Lottery, we would probably own a winery or two.

Not All Competitions are Equal

Given that the three-tier system means distributors determine what wines get into the local retail venue, wine competitions can provide an excellent opportunity for smaller, unknown wineries, or those with small productions to get their name out in front of wine drinkers. Many of the competitions also have a “public tasting” where the wine is poured to all who purchase a reasonably priced ticket. This can give good, but relatively unknown wines with a smaller marketing budget a chance to introduce themselves in a new region. A number of wines we love have entered and won awards in such competitions.

Other competitions favor the local or well-heeled wineries. The Houston Livestock Show & Rodeo International Wine Competition requires its wines to be available in Texas. It also requires the wineries winning double gold, champion or reserve champion awards provide enough wine to be poured at the Rodeo Uncorked! Round Up Wine and Best Bites Competition. Top winners must also reserve 40 – 50 cases of wine to potentially be purchased by the Rodeo for sale at the Champion Wine Garden in March of the following year. Other winners are required to provide 9-liter etched bottles for sale at the Champion Wine Auction. Rules for entry such as these may bar a smaller winery from entering. And if the wine isn’t available in Texas, the winery cannot enter at all.

Step Right Up – Everyone’s a Winner

And then there are those competitions where everyone wins a prize. When I was in high school marching band, our director would take us to the most out-of-the-way, travel dirt roads over a one-lane bridge, Podunk competitions so that he could add another trophy to the glass case in the hallway. People did not get behind the locked glass doors to see the trophy came from the “Little Sisters of Possum Holler (population 307) Invitational” – they just saw a line of trophies.

Likewise, there are some wine competitions the Big Boys don’t bother to enter. They wouldn’t add any sort of prestige to Chateau Montelena or Robert Mondavi, but they might put a feather in the cap of Big Ray’s 63rd Avenue Cellars. His wine that Wine Spectator rated a 65, might just be the one that receives the silver or gold in the “NASCAR International Motor Fuel and Wine Competition.”

Said another of my better-not-identified (because keeping one’s job in the industry is a beautiful thing) sources,

When it comes down to it, any award is better than no award to wineries. It’s absolutely a marketing tool. I know for a fact of a few wine competitions where the organizers have approached the judges and told them that they have not handed out enough awards and could they choose more wines to receive medals. I wish I were kidding.

The Gold Medal Is Just One Indicator

According to Professor Hodgson’s research the likelihood of receiving a Gold medal can be statistically explained by chance alone. I might not go quite that far. But a medal is only one indicator a consumer should use when choosing a wine. There are many she should look for. Check out the ratings from your favorite wine publication. Ask a sommelier. Read your favorite trusted blog. And most of all, taste the wine.

After all, your palate is really the only one that matters.

Cheers!

The WineWonkette

  • UncorkedVentures

    That is pretty much how I understand those ratings. They give second rate wineries something to put in their marketing efforts

    • http://twitter.com/WineWonkette Amy Corron Power

      Not always second rate wineries — some very good wines that have lower marketing budgets can get mileage from wine competitions as well. And then there is the “Keeping up with the Jones'” in the bigger competitions. I think that's why it's important to consider the competition. If a wine from California has only one award, and it's Miss Congeniality from clear across the country at some tiny wine competition – you have to take it for what it's worth.

  • joshgana

    Nice post! I completely agree, and would even argue that the traditional rating systems don't account for an accurate perspective on wines for general consumption of the drinking public.

    • http://twitter.com/WineWonkette Amy Corron Power

      i think it's a bit like obscenity – you have to look at the “community standards.” What the locals might love (hate) in one part of the country may not make it popular (or bad) in another.

  • http://www.innovativewine.com Innovative Wine

    Hi Amy,

    Anyone who has not served as a judge for a major wine competition could understandably view the results of Prof. Hodgson's study as evidence that these competitions are not perfect, which they are not. Your post seems an attempt to discredit them even further being that you view them as “suspect”.

    Wine competitions are all about marketing for the wineries. If this were not the case, there would be no incentive to spend the entry fees and waste good wine on a group of “rust experts” to evaluate. Each competition is different with the qualifications and experience level of the judges varying widely. Some require judges to undergo a testing process to qualify and others have no requirements other than knowing the organizer. Because of this variance, a broad brush cannot be used to paint the entire process.

    Competitions that promise awards damage the credibility of the entire process. Few reputable wineries would enter such a judging if they understood the implications should the competition be exposed as a fraud. Guilt by association will harm the wineries reputation where this is the case.

    If we look closely at the judging process (in reputable competitions) it is easier to see why wine judging will never be an exact science or perfectly consistent. I can state without reservation that the people who judge these competitions take their responsibility very seriously. While my palate or yours may not always agree with their evaluation, it is not due to lack of commitment on the judges part to fairly evaluate each wine.

    I agree that these awards do not mean that a particular individual will like the wine any more than does a high score from one of the major wine publications. I often taste wines recommended “experts” that I do not find appealing at best. These scores and awards only say that “someone” liked them on the day they were tasted. Your own palate is the only final judge and that will vary depending on thousands of variables such as mood, menu and situation.

    During a wine competition, a judge sits on a panel of three or four judges, each tasting the same wines in each flight. A flight of wines typically has between 30-40 wines. Each judge evaluates every wine independently, giving a score between “no award” and “gold”. Some competitions simply average the individual scores and others allow the judges to discuss each wine once all judges have finished tasting.

    Some varietals are easier to evaluate than others. Aromatic white wines, sauvignon blanc, riesling, etc, when tasted at room temperature will amplify faults and varietal characteristics better than a flight of monster petite sirrah or zinfandel. Background aromas of 40 volatilizing glasses of wine have to be filtered out in the process as well.

    Once the flight is tasted, the glasses are cleared and another flight of 40 wines is placed. The process is repeated four or five times in a typical day of judging. You could argue that fewer and smaller flights of wine would improve consistency, which I think it would, but you would end up with competitions that wineries could not afford to enter due to the cost of running such an event. The compromise is using panels of judges over individuals which does improve consistency, albeit not perfectly.

    Every judge has faced the awkward situation of giving a wine a gold medal that all the other judges gave no award. Each has also experienced giving no award to a wine that others gave high marks. While not a common occurrence, it can be attributed to the order the judge tasted the wines. They may have tasted a really bad wine just previously or a wine where some component masked a flaw in the current wine. Again, this is the benefit of panels.

    Have you ever purchased a bottle of wine that tasted great in the winery only to wonder what you were thinking when you opened the bottle at home? When we taste several wines in a day, our palates and noses become fatigued regardless of how great a taster we are. It takes a tremendous amount of concentration to taste 100 wines in a day and keep your mind and palate objective.

    Until someone invents a machine that can be programed to an individual's specific taste, wine ratings and competitions are the best they can be, subjective but a good place to start.

    • http://twitter.com/houstonwino houstonwino

      I'm not sure what your point of contention is as it seems that you concede almost every single point made in the post. I have seen very knowledgeable, talented judges sit side by side with an “expert” whose qualifications began and ended with a weekend box of Franzia. Professor Hodgson's study, as well as this post, are factual and fair. Gold medal swill is still swill. The process is tough, as is any hardcore tasting, if a judge isn't up to it, too &^%$ing bad. Competitions are what they are…fun marketing tools and not much else.

    • http://twitter.com/WineWonkette Amy Corron Power

      I realize I didn't clarified my suspicions about competitions. (I corrected it above) I mean in ALL industries where the criteria is highly subjective. College rankings? It's college administrators ranking other colleges. “Super Lawyers” rankings? The publication usually has a HUGE ad for the firm that has a number of Super Lawyers, and rumors abound that the ad buy came FIRST.

      I'm not saying that there aren't wines that won a gold and truly deserve it — I've tasted a number of medal winners that I loved! But a medal doesn't “prove” that one wine is superior, in my opinion. And that's the argument we got when we didn't like a particular wine. Even though Wine Spectator had ranked the wines by this winery in the 70s, the writer and winemaker kept pointing to the list of awards as “proof” the writers of this blog were “ignorant about wine.”

      To me, it's just ONE indicator that a consumer should use, and it shouldn't be weighted any heavier than the others we suggested: Ask a sommelier, read reviews, and by all means TASTE the wine.

  • http://www.innovativewine.com Innovative Wine

    The point of my comment was not to disagree with what you said in your post but rather to explain the process to those unfamiliar with how wine competitions work. As Joe so eloquently said, there are good judges and occasionally a few who struggle with differentiating wine from turpentine in my opinion. Panels of judges are better than single tasters unless you find one that has a similar palate.

    The study shows that there is a statistical advantage to entering many competitions which came as no surprise. Two Buck Chuck won a double gold in a major competition a few years ago which proves this well.

    There was no feeling sorry for the “poor” judges who work these competitions, only the statement that palate fatigue is a factor when large numbers of wines are tasted. Often wines with a little residual sugar do well when they should not because they feel softer and more enjoyable after tasting a group of wines with monster tannins or heavy oak. This should be a warning sign to the judge but quite a few miss it in my experience. (Re: Two Buck Chuck)

    Your post is a fair assessment and well written. I wanted to add a little explanation of the “why” on top of the “is”. Most gold medal wines I have tasted are better than average but there are exceptions.

    Of course the only people who know what good wine tastes like agree with my palate! :)

  • http://www.landoflicorice.co.uk/ Healthy Licorice

    I think it will prove to be a great post ,thanks

  • http://www.landoflicorice.co.uk/ Healthy Licorice

    I think it will prove to be a great post ,thanks

  • Pingback: Wine Competitions: Are They Utilized | Chardonnay Wine

  • Pingback: Serving Temperature Can Affect Tasting Results | another wine blog