If you're in the business of fermenting grapes for money, you're going to want a medal from the California State Fair. A bronze, silver or gold from its wine competition means recognition, esteem and a solid boost to your sales figures. This is not only the oldest judging competition in the United States - it started in 1854 - but also one of the most prestigious. Every year, its 70-strong array of noses and tongues, which belong to many of the best-known critics, sommeliers, academics and winemakers in the nation, pass judgment on about 2,500 bottles.
But how meaningful are their results? A few years ago, a frustrated vintner named Robert Hodgson, who had a background in statistics, thought of a way of testing the testers. He wondered what would happen if he supplied 100 wines for consideration but, without the judges knowing, slipped each wine to them three different times. Would they notice? Surely, with their trained and articulate palates, they'd at least be consistent in how they rated the identical drinks?
Having sought the agreement of the chief judge, G.M. "Pooch" Pucilowski, Hodgson ran his study for four consecutive years. When the results were calculated, they staggered him, disappointed Pooch and infuriated others.
"Some people think I'm wacko," says Hodgson. "Some say I'm full of crap. None is a scientist."
The Hodgson studies have shaken the wine world, calling into doubt the promises of its most elevated masters.
Hodgson is a retired professor of oceanography who developed an interest in wine in his early 30s.
"I'd been making blackberry wine for fun and, in 1967, a friend in Napa Valley put me on to someone who could get grapes," he says. "We drove 500 miles [800 kilometres] in a little Volkswagen, bought 500 pounds of cabernet and made wine."
As he learnt the secrets of making it, Hodgson also began to educate himself.
"I got a bunch of people together and we would all chip in and buy expensive wines to see what they tasted like. We had - and I'm not kidding here - a [famously expensive] Château Latour, as well as a good Californian and a real cheap one."
These ad-hoc sessions led Hodgson to question much of what he'd been told about vintages on the market's fabulous end.
"We found that US$5 wine is not very good, US$10 gets better, US$15 is pretty damn good but if you go up to US$20 it doesn't get any better from there to US$200."
He was teaching in the day and making wine in his spare time. After taking a wine-judging course in Sacramento, he spent two years as a judge. And he was terrible.
"Others would take copious notes," he says. "All these notes about all these flavours! They had this ability to transcribe what was going on in their brains and make sense of it. I thought, 'I'm not in their league.' I told the chief judge I should quit."
What was Hodgson missing? What did he lack that other experts seemed to have? Is there really a special ability, possessed by some, that makes possible a marvellous and extended palate of tastes?
According to British wine critic Jilly Goolden, the person responsible for this compulsion to endlessly detect and list flavours is … Jilly Goolden. As a presenter of the BBC's Food and Drink show between 1983 and 1999, she became famous for her evocative and wildly precise descriptions of the galaxies of taste that were apparently exploding on her palate.
"The language of wine, which has been adopted internationally, is entirely down to me on that programme," she says. "I was faced with the challenge of describing what was in my glass to the people at home. I thought, 'There's no point talking about it in old-fashioned speak, using words like 'finesse' or 'backbone' or 'terroir' because that doesn't tell you what the wine actually tastes like.'"
Viewers were enchanted by her performances, which you might describe as "poshly effervescent on the palate with just a hint of loopy on the nose" and which, in a typical broadcast, ran like this: "It's so gooseberryish it's ridiculous! It's like gooseberries that have just started to simmer, they're still al dente, they've still got loads of fruit, but it's not too aggressive, loads of flavour, tart but somehow sweet, big but delicate, challenging but cosy." She described one drink as being like "a wheelbarrow full of ugli fruit", another as resembling "an aeroplane revving up" and another as "cats".
Goolden, it turns out, is exactly the kind of person that caused Hodgson to quit his judge's chair.
"When I go to a wine tasting, I write down incredibly detailed notes," she says. But rather than having superior biological equipment, she puts her ability down to the possession of a fastidiously curated memory of smells which she can go to when reaching for descriptions.
"I have a larger reference, because I remember smells," she says. "For instance, if you're running or even walking on a hot road, you get the smell of slightly melted tarmac. It becomes much more pungent when it's very hot, and the rubber on the soles of your shoes also becomes more pungent. The smell of that combination is very precise, and it's exactly the smell of gamay, which is the grape in beaujolais. I spent a while trying to precisely identify it but if I ever put my nose in a glass of wine and get it, I immediately know where I am." How many flavours like this is it possible to detect? "It's infinite."
Goolden nervously admits she's not a scientist and hurriedly adds that this is just her layperson's impression of the wine-taster's method. But James Hutchinson, of the Royal Society of Chemistry, is a scientist, as well as being a wine expert and former captain (and current coach) of Cambridge University's competitive Wine Tasting Team. He says Goolden's exactly right.
"A big part of what enables us to recognise different aromas is the sense memory we have from having encountered it before," he says. "The sorts of chemical flavour-compounds we find in wine can also be found in other parts of life. The ones that give trainers the smell of sweat are probably a similar compound to those that give a slightly sweaty aroma to some chablis."
And there are a lot of these flavour-compounds.
"Within any particular wine, you could have hundreds of different compounds contributing to its overall characteristic. That's why we find wine so inherently complicated."
And Goolden is correct, too, about not having an innate superiority.
"Wine experts, myself included, have a lot of experience interpreting the information our senses are telling us," says Hutchison. "But everyone in the public has exactly the same machinery. There's no such thing as good tasters and bad tasters."
Some researchers disagree. In 2012, John Hayes, director of Pennsylvania State University's Sensory Evaluation Centre, published evidence that suggested professional oenophiles might, in fact, have some genetic superiority. Having apparently proved that 110 wine experts had a significantly increased sensitivity to taste compared with 220 consumers, he said, "It is not just learning. Experts also appear to differ at a biological level." But whether the secret is practise, genes or a vast olfactory reference trove, poor Hodgson agreed he didn't have it, and the experts surely did. And then he did his test …
"We gave each judge a flight of 30 wines," he says. "They'd all have three samples of the same wine, but they didn't know it. These samples were arranged randomly. When I first saw the results I could hardly believe them. They scored the identical wines like they were different. It was staggering." Out of a 20-point scale (scored, for esoteric reasons, between 80 and 100), an identical drink would typically vary by four points from one tasting to the next. "About 10 per cent of the judges were really bad," he says. Their judgments of these wines ranged between 16 and 18 points. "But about 10 per cent were quite good. We thought we'd be able to use these judges as mentors to teach the others how they did it."
Extraordinary as it was to have realised that 90 per cent of the judges didn't appear to have any real consistency in their judgments, at least they'd narrowed down an anointed few who could. Well, that's what Hodgson thought until he tracked their results the following year.
"It turned out they couldn't maintain that performance. One year they might be really good, the next they were just in the middle of the group."
For his next study, Hodgson broadened his investigation.
"I looked at a set of data that showed the scores for wines that were entered into as many as 13 different competitions," he says. "I tracked the scores from one competition to another. There were, like, 4,000 wines that I looked at. Of all the ones that got a gold medal, virtually all got a 'no award' some place else. It turns out that the probability of getting a gold medal matches almost exactly what you'd expect from a completely random process."
As you might expect from a statistician, Hodgson's numbers are big enough to count. It's hard to argue with data extracted from hun-dreds of judges and thousands of wines over several years. He might be the first academic to have treated the pronouncements of the wine gurus so rigorously, but he's not the first to have come to an embar-rassing conclusion.
One French academic, Frédéric Brochet, decanted the same ordinary bordeaux into a bottle with a budget label and one with that of a grand cru. When the connoisseurs tasted the "grand cru" they rhapsodised its excellence while decrying the "table" version as "flat". In the US, psychologists at the University of California, Davis, dyed a dry white various shades of red and lied about what it was. Their experts described the sweetness of the drink according to whether they believed they were tasting rosé, sherry, bordeaux or burgundy. A similar but no less sobering test was carried out in 2001 by Brochet at the University of Bordeaux, in France. His 54 experts didn't spot that the red wine they were drinking was a white dyed with food colouring.
Despite results such as these, wine judges continue to believe in their ability to finely judge quality. As well as being a former wine buyer for British supermarket chain Tesco and author of the bestselling book The Knackered Mother's Wine Club, Helen McGinn is a judge at the International Wine Challenge.
"I think the medals are a brilliant guide," she insists. "If I'm standing in a wine shop and struggling to find things I recognise, I'll choose one that has a medal from a competition that I've heard of." However, her experience as a buyer has led her to a conclusion that's startlingly similar to the one Hodgson came to all those years ago when he was starting out. "Take a £5 [HK$65] bottle of wine. The duty you're paying on that is more than £2 [Hong Kong reduced duty on wine imports to zero in 2008]. Then there's shipping, packing, retailer margins. So you've spent no more than 50 pence on the wine in that bottle. Spend £1 more, you get twice as much money into the wine. And so it goes on. Once you hit £7 it starts to get interesting. Between £10 to £12 it gets exciting. But I rarely spend £20."
If what you're after is a nice bottle of wine, then, perhaps it's best not to worry about the label or the critics or the medals. The most rational decision is the one judged on price. But, despite the work of the scientists, it still appears as if there's very little rational behaviour to be found among wine lovers and makers.
And Hodgson is no exception. Today, he runs Fieldbrook Winery, in California's idyllic Humboldt County, and produces about 1,000 cases a year.
"I still feel good when I win a medal," he admits. "I do! I crow about it, just like everyone else."
And when he doesn't win? "Well, when I lose, I don't get upset about it. Not any more."
The Sunday Telegraph UK