Search Articles

Please Share

FacebookTwitterDiggDeliciousStumbleuponGoogle BookmarksRedditTechnoratiLinkedin

Subscribe to Blog via RSS

Enter your email address:

Delivered by FeedBurner

Sponsors

Search for Events

Sponsors

Most Read Articles

How Critics Taste Wines - On Blind Tasting

You’ve read thousands of wine reviews. But what do you know about the way wine critics perform their evaluations? Frankly, even wine writers don’t always know how their fellow reviewers conduct tastings.

This is the first in what will be a short series of articles revealing how several respected wine critics go about their business. My inspiration was a panel discussion at the 2014 Symposium for Professional Wine Writers held last week in Napa Valley. Moderated by Alder Yarrow of Vinography, Ray Isle of Food and Wine magazine and Lisa Perotti-Brown of The Wine Advocate described their processes in substantial detail. Other writers chimed in. I’ve interviewed still others.

Wine Reviewing Makes You Go Blind. Or Not.

320px-Blindfold hatThere is ongoing debate between wine reviewers on the pros and cons of tasting wines blind. Newsletter reviewers, such as Antonio Galloni, Doug Wilder, and those at the Wine Advocate, do not taste blind. Critics for top wine magazines—Wine Spectator, Wine & Spirits, and Wine Enthusiast—do.

In a note to Jameson Fink, Harvey Steiman, who reviews the wines of Australia, Oregon and Washington for Wine Spectator, said, “At Wine Spectator every review in New Releases is the result of a blind tasting. We believe that blind tasting insulates our judgments from any bias that might result from knowing producer or price. It’s the fairest and most objective way to allow every wine to show its true character”

In a recent blog of his own, Steiman suggests that even knowing the stated alcohol level of a wine prior to evaluation can color one’s opinion. “Some high-profile wine writers are suggesting that that they shouldn’t have to taste blind, that it’s unnecessary unless you have an agenda. I would submit that a preference for low-alcohol wines is an agenda. Just how much alcohol is present is not so obvious when you can’t see the label.”

Alcohol isn’t the only bit of information that might sway an opinion. Reviewers could potentially be biased with regard to price point, producer or certain varieties in a particular region. Even bottle weight and label design could sway perception.

Joe Czerwinski, managing editor at Wine Enthusiast where he also reviews wines of Australia, New Zealand and the Rhone, tells me that he knows what countries might be involved in his review tastings due to his beat. When tasting for a particular feature article, he may also know the specific region and varieties. He never knows the price or producer though.

At Wine & Spirits, reviewers have three pieces of information. They know the region, the variety and whether or not the wine costs less than $15. The latter helps them earmark wines for “best buy” designations. Knowing the region and variety lets them judge typicity, eg. does a particular Dry Creek Zinfandel meet expectations as such.

Some reviewers, myself included, often taste blind but don’t do so exclusively. In an email to me, Jon Bonné of the San Francisco Chronicle said, “I’m increasingly of the opinion that blind tasting isn’t really that useful when tasting critically.” The Chronicle’s panel tastings are typically conducted blind but, when tasting on his own, Bonné usually sees what he’s pouring. This is common with solo reviewers. The logistics of managing the blind tasting of a multitude of wines by one’s self makes it highly impractical.

Bonné also poses the question, “Why shouldn’t a critic evaluate a wine the way his or her readers do?” I’m sympathetic to that view and try in other respects, such as selection of glassware, to taste wines the way I think enthusiasts might. Perhaps more importantly, Christophe Hedges of Hedges Family Estate suggested (in that same article by Jameson Fink) that he believes blind tasting removes important context from the evaluation process.

What context matters? Knowing the vintage can help you judge whether a wine is an excellent 2011 or a lean 2009. Region, even vineyard, and variety can indicate typicity. Knowing the producer, and their track record, provides important clues as to the likely aging profile.

Context is, along with practicality, the primary reason given by critics for not tasting blind. At the Symposium, Lisa Perotti-Brown said she tries to ensure her tastings at wineries are under controlled conditions, but she always tastes non-blind and with the winemaker. She asks a lot of questions as she tastes. One of her main goals is to get as much information as possible for her vintage reports.

In a past interview, Antonio Galloni told me, “I generally prefer not to taste blind because the questions readers ask of me require some context.” Reader questions he fields include comparisons of different vintages of a particular wine, wines made by different producers from the same vineyard, differences between vineyard blocks, etc. Therefore, he likes to taste three successive vintages of each wine: the one being reviewed, the preceding vintage and a barrel sample of that upcoming. He, and other reviewers at wineries, will also taste a variety of vineyard designates side-by-side.

As important as context is, the issue of reviewer bias always arises when discussing non-blind tasting. Can critics completely divorce themselves from prejudices of any sort? The mind is a sneaky thing. Objectivity can be easily, and stealthily, clouded. There’s a reason why doctors aren’t (officially) allowed to treat their family and why judges recuse themselves from cases.

Even if one can be totally objective, non-blind tasting creates doubt in the mind of consumers, wineries, etc. For example, going back to that Jameson Fink article one more time, Hedges assumed Wine Spectator didn’t taste blind and that Steiman’s reviews might somehow be biased against them. Neither was true, but the doubt and perception affected Hedges’ own behavior.

On the other hand, blind tasting tempts one to make guesses which may also be a distraction. Unless the tasting is not just blind but double-blind—when the reviewer has no idea whatsoever which wines are in the tasting—there’s a natural tendency to look for hallmarks of the known regions or producers in various bottles. That can lead to false assumptions that skew results more than non-blind tasting would have.

That’s why magazines’ office-based tastings are generally double-blind. They also have enough staff to be able to receive, catalog and pour bottles without involvement from the reviewers. Home-based reviewers can’t do that unless they have a part-time helper. [Reviewers may taste 50 wines per day, six days per week. Setting all that up is very time-consuming and even the most gracious spouse or friend won’t want to spend 20 hours a week doing that over the long term.]

Jon Bonné adds this on blind vs. non-blind, “One significant thing: When I looked at how Robert Parker and the Advocate was tasting, it was clear that they increasingly opted for non-blind tasting, often with the producers themselves. Context seemed to be more important. Over and over again, my conclusion was that non-blind tasting led to better criticism.” Bonné and Wine Advocate are famously at odds with each other. His praise for their methodology therefore speaks well of that system and Bonné’s own objectivity.

In reality, neither tasting method is perfect. What we see is that the method used depends on the practicality of tasting blind, the intended use of the evaluation and the intended audience.

If the critiques will be going into a large compendium of capsule reviews without much context, such as those found at the back of magazines, then blind tasting makes sense. It evens the playing field, creates the perception of fairness and most readers are simply looking for a list of solid wines from which to select a few bottles for purchase anyway.

The newsletters address a different type of buyer. These subscribers may be purchasing in case volumes or higher. They are looking at more expensive wines on average and have more concerns about aging potential, resale value, etc. Such readers are also more likely to be using their wines for formal tastings, perhaps verticals or horizontals. Then, the added context provided by a Galloni or Perotti-Brown educates and provides creates discussion points.

Wine evaluations for inclusion within a regional or producer profile are different still. This is the type of writing Ray Isle typically does for Food & Wine. In this case, readers are primarily interested in the story or learning about the overall topic. Notes on the wines are used almost adjectivally, adding color or supporting a broader point. Tasting non-blind does no harm and is often essential.

The good news is that, regardless of their tasting methodology, the vast majority of professional critics are just that—professional. They do very the best they can in every circumstance to be objective. Personal taste, be it for intense, mouth-filling wines or lithe, high-acid ones, plays a much larger role in the scores than does knowledge of producer, price point or region.

Follow NorCalWine on Twitter for wine news, information on events and more, or friend me on Facebook. This article is original to NorCalWine.com. Copyright 2014. Photo of blindfolded woman by Isarra. All rights reserved.

Comments   

Tim Teichgraeber
#1 Tim Teichgraeber 2014-03-11 18:29
Very even-handed treatment of a sensitive topic. Well done, Fred! I think that without at least some context, blind tasting doesn't generate a lot of critical insight. At least a little bit of context (knowing the variety, vintage, price and/or place of origin)is invaluable in judging typicity or value.
Quote
Wilfred Wong
#2 Wilfred Wong 2014-03-11 19:15
Nicely done article, in the end, however a review does his or her job, reconciliation comes in the final statement on whatever the wine in question is...sometimes a blind tasting yields great results and other times, a blind tasting give confusing conclusions. Critics have to be true to themselves and understand when they performing well and not.
Quote
Michele Good
#3 Michele Good 2014-03-12 19:25
Nice article. I think for consumers blind vs. non-blind ultimately doesn't matter as long as the reviewer is consistent. If consumers find a reviewer, whether Spectator,Heimo ff or Parker, that tends to agree with their personal wine preference then how the wine is tasted becomes somewhat irrelevant.
Quote
Fred Swan
#4 Fred Swan 2014-03-12 22:22
Thank you all for the excellent comments. I agree with each of your conclusions.

One thing that's dangerous about blind tasting is that it forces the reviewer to draw conclusions in the absence of sufficient data. For example, I tasted a wine at IPOB with the winemaker. It was a wine I'd tasted previously, not blind but in the absence of any genuine context.

Based on my first tasting, I'd made certain conclusions about how the wine had been made and those turned out to be incorrect. Either way, it's a good wine. But it turned out that, contrary to my initial impressions, the wine was much more about terroir than technique.
Quote
Michael Cervin
#5 Michael Cervin 2014-03-12 23:08
As wine writers we all strive to do our jobs exceedingly well. When I review wines for everything from regional to international publications, I am keenly aware of the wines. As a frequent wine judge at competitions, we taste wines blind, with occasional minimal information provided. What always amuses me at blind judgings is the number of fellow judges who are surprised at the wines they ended up liking, ones that on occasion had they tasted non-blind they probably would have given a lower score.
Quote
Thomas Matthews
#6 Thomas Matthews 2014-03-14 19:37
Fred,

It's your blog and you can do what you want, but I'm curious to know why you deleted my comment.

Thomas Matthews
Executive editor
Wine Spectator
Quote
Fred Swan
#7 Fred Swan 2014-03-16 05:27
Thomas,

I never saw your comment. If you left one and it was deleted then it was in error while eliminating spam comments from faux pharmaceutical companies, etc. Please repost and sorry for the trouble.
Quote
Bob Henry
#8 Bob Henry 2014-05-09 03:34
Thomas,

I'd be interested in reading your comment.

If for nothing else than my edification as a wine consultant to stores and restaurants, I invite you to re-post it.

You have me on the edge of my seat . . .

~~ Bob
Quote
Bob Henry
#9 Bob Henry 2014-05-09 06:58
Synopsis from The Wall Street Journal "The Informed Reader" Column
(November 8, 2007, Page B7):

"Don't Discount Wine Critics Fooled by Blind Tasting"

Excerpted from Slate
(November 7, 2007)

Wine critics should get a break when they pan a wine during a blind tasting that they normally praise to the skies, says Slate wine columnist Mike Steinberger.

Few things give satisfaction like catching a wine snob preferring the bottle of swill over a storied chateau in a blind tasting, Mr. Steinberger concedes.* And in the wine world, assessing an anonymous glass is considered a critic's ultimate trial by fire and the most democratic way to let the flavor of new wines compete against reputable brands.

Still, the label represents a significant track record of quality that wouldn't necessarily emerge from a single tasting, Mr. Steinberger says. Knowing the wine's name helps the critic assess how it compares with the taste of prior vintages at a similar stage, making for better predictions of how a wine will turn out. Mr. Steinberger recommends using both blind and open approaches to evaluating a vintage.

(* Someone once asked Harry Waugh, the acclaimed English wine writer, if he had ever confused a Bordeaux with a Burgundy. "Not since lunch," was his reply. . . . )


Go here for the full article . . .

Excerpt from Slate
(November 7, 2007):

"In Blindness Veritas?;
Tasting wine blind isn't all it's cracked up to be."

[Link: slatetv.com/.../#page_start]

By Mike Steinberger
"Drink: Wine, beer, and other potent potables" Column

Don't drink the label, drink the wine. Among the many (unwritten) rules of wine appreciation, this is easily the most important. It's also the hardest one to follow. Even the most discriminating oenophiles find it difficult not to be influenced by the name on the bottle, particularly if the name is a hallowed one. For this reason, many people believe that the only reliable way to judge wines is to taste them blind -- that is, to taste them without knowing who made them. (Indeed, a blind tasting is the one occasion when drinking out of a brown paper bag is not only respectable but a sign of intellectual rigor.) When people don't have the benefit of seeing the label, the argument goes, they have no choice but to judge a wine solely on its merits. But does this approach really make for the best wine criticism? In blindness veritas?

It depends on whom you ask. The Wine Spectator proudly trumpets the fact that its tastings are done blind, presumably as a way of distinguishing itself from Robert Parker, who says only that he tastes blind "when possible."

. . .
Quote
Bob Henry
#10 Bob Henry 2014-05-09 07:06
From Wine Spectator Online
(posted September 13, 2003):

“Through a Glass, Darkly:
Riedel Pioneers Opaque Tasting Glass”

By Nick Fauchald

For wine drinkers wishing to stay completely in the dark about the wines they are tasting, Riedel Crystal has made it possible. The Austria-based stemware company has introduced the ”Blind Blind” Tasting Glass, a jet-black, opaque glass that masks a wine's color.

During a double-blind tasting, all of a wine's pedigree -- including appellation, varietal, vintage and producer -- is concealed. Going further and eliminating knowledge of a wine's color prevents tasters from judging a wine through visual cues, such as clarity and deepness of hue.

Like the rest of Riedel's premier Sommelier collection, the Blind glasses are handmade from full-lead crystal. Shaped like the collection's Zinfandel/Chian ti glass, the blackness of the glass is obtained by adding the metal manganese oxide -- which is used to tint glass purple, or black if more is added -- during the manufacturing process. The “Blind Blind” glasses retail for $59 each.


See this related article . . .

From Wine Spectator Online
(posted November 21, 2002):

“Can You Tell Red From White?”

By Thomas Matthews

The New Yorker threw down the gauntlet. Wine Spectator rose to the challenge.

In a special issue devoted to food (Aug. 19 & 26), The New Yorker magazine devoted one small story to wine. It's a shame, really. There were plenty of strong stories in the issue; I particularly enjoyed a profile of wild man chef Mario Batali of New York's excellent Babbo restaurant. Why give such short shrift to food's favorite partner? Even this august magazine just doesn't get the point.

To add insult to injury, the one wine story was written by an avowed beer lover, Calvin Trillin. Trillin may be a noted food writer, but he confessed, "when I'm trying to select a bottle of wine in a liquor store, I'm strongly influenced by the picture on the label."

After that, it came as no surprise that Trillin set out to take wine lovers down a peg or two. The story's title said it all: "The Red and the White. Is it possible that wine connoisseurs can't tell them apart?" Trillin suspects they can't. He has heard about a test in which wines are served at room temperature in glasses that mask their color, and "was definitely told," he writes with evident glee, "the tasters often couldn't tell red from white."

Can this be true? If so, it makes a mockery of the entire idea of serious winetasting. It implies that wine critics are frauds, basing oracular pronouncements about quality and origin on nothing but inside information and guesswork. It suggests that we at Wine Spectator might just as well hang up our corkscrews and look for honest work.

I decided to take the test myself. And to be fair to Trillin, I asked for volunteers to join me. They included our tasting director, who reviews 2,000 wines a year and should be able to tell the difference (and, if not, be fired), along with three copy editors and two art directors, all of whom drink wine with enthusiasm, but don't pretend to be experts.

The day came. Our tasting coordinator selected six wines -- I asked not to know how many were red, how many white -- put them in bags, numbered them and brought them all to room temperature. The tasters then sat, blindfolded, as each wine was poured into a glass and the glass was put into trembling hands.

We sniffed, we sipped, we guessed.

"This is harder than I imagined," confessed an art director, tasting his first wine. One copy editor was more confident. "That's a Chardonnay," he proclaimed of wine No. 5. "I'd say this one [No. 3] is a red Rioja," opined the tasting director. "It has a supple texture and a hint of vanilla." Another copy editor based most of his guesses simply on the aromas. "The hard part is not dribbling the wine down my chin," he said.

How did we do? Six wines times seven tasters equals 42 guesses. If Trillin's suspicions were true, and we were simply flipping coins, the odds say we'd get about half right.

In fact, we guessed correctly 40 times. There were only two mistakes.

The hesitant art director missed the first wine, probably from nerves. The other art director -- a young woman in her first job out of college -- mistook an oaky Chardonnay (yes, wine No. 5) for a red wine, possibly because barrel fermentation and aging had given it as much tannin as some softer reds. None of the copy editors missed a wine. Neither did the tasting director, nor, I'm relieved to say, did I. And by the way, wine No. 3 was indeed a red Rioja.

What did we learn?

Even an inexperienced wine taster can tell red from white. It's just not that hard. There's no reason to be intimidated by wine; our taste buds are natural guides to its flavors and its pleasures.

More than that, our palates can be educated. Someone who cares enough to study wine can figure out a lot more than whether it's red or white, even "blind" -- like which grapes went into the wine, where it was grown, how old it is. The kind of information that makes wine a uniquely complex beverage, and particularly fascinating for people who really enjoy the sensory side of life.

Calvin! You're a guy who likes to eat, and you write about food in a way that makes a reader hungry. You should give wine a second chance. And the next time The New Yorker wants a wine article, don't just scoff and theorize. Pull some corks and pay attention. You might be surprised how much you enjoy yourself, and how much you can discover in a glass of wine.
Quote
Truth
#11 Truth 2014-05-19 16:54
WS does not conduct a true blind tasting.
Quote

Add comment


Security code
Refresh