Lewis Chester DipWSET on the folly of trying to apply simple metrics to a complex art.

In Part I, I touched on ‘Parkerisation’, the complexity of ageing, the failure of critics to fully grasp fine wine culture and the practical difficulties wine critics face in judging wines, among other subjects. So what are the other nuances of why fine wine scoring is a senseless anachronism?

One Good Call Does Not Make A Genius

Robert Parker made his name championing the Bordeaux 1982 vintage as an all-time great at a time when many in the wine trade thought it was good to average. It turned out Parker got it right; but one correct decision doesn’t make a genius. The Bordeaux 2001 vintage was deemed at the time of its release to be “average” (wine-speak for “don’t bother”): however, today, it has turned into a very good vintage for Bordeaux. What is clear is that taking a snap-shot of a wine at some point during its evolutionary cycle and giving it a score that will pre-determine the public’s view of that wine for many decades to come is plain stupid.

“How Football Explains The World”

One of my favourite books is How Football Explains The World by Franklin Foer. The best analogy for thinking about the 100-point scoring system is to compare it with football (or “soccer”, if you’re American). Let’s take the world’s greatest football team – which happens to be the one I also support – Arsenal Football Club. (This is a deliberate example of bias, of course, which is just as likely to be prevalent among wine critics.)

Recently, Arsenal played their arch rivals Tottenham Hotspur. At the end of the match, the commentator, Gary Neville (a former Manchester United player and Sky Sports commentator) named Granit Xhaka, a central midfielder, as his Man of the Match. Personally, I thought William Saliba (a central defender) was more deserving, and the TV audience could have opted for another two players who also had terrific games.

Two weeks after the match, if you asked that TV audience who Gary Neville named as Man of the Match in this game, less than 20 percent would have any clue. That piece of data has now become redundant. Who Gary Neville, one expert, thought the best player in one game was is neither here nor there, and has no intrinsic value. No football manager in his right mind would decide to buy Granit Xhaka based upon Gary Neville’s Man of the Match Award in one game.

In the same way, what Robert Parker, or a host of other wine critics, thought about a specific wine, on one day, is going to be neither here nor there over the course of the wine’s entire life history. It has no intrinsic value. The difference with the football analogy, however, is that currently, fine wine consumers are materially influenced by what the wine critic thought on that one particular day, whereas they shouldn’t care in the slightest. It’s an irrelevant data point in the context of the wine’s life cycle. But, in the absence of any other credible data, they often end up buying solely based upon the critic’s score.

Granit Xhaka: Gary Neville’s choice for Man Of The Match in the recent North London Derby – but the author’s second choice | © Shutterstock

Lack Of Credible Data Points

The issue we have identified exists, in part, due to the lack of credible data points that allow the fine wine consumer to make good buying decisions. It’s just not cost effective or practical to get wine critics to re-evaluate any given fine wine every twelve months, especially since there are new vintage wines that the wine critic needs to rate each year.

If evaluating fine wines based on a scorecard (whether out of 100, 20, 10 or five points) is pointless, what systems exist that satisfy the consumers’ need for an easy-to-understand, credible basis for buying their wines? If we go back to the football analogy, we know that the best football teams in the world are judged principally on the basis of how much silverware they have won, both recently and historically. Real Madrid of Spain have won more La Liga division championships and more European Cups (now called The Champions League) than any other team, both on a historical basis and on the basis of recent history.

For this reason, every football expert and supporter inherently knows that Real Madrid is one of the top football clubs in the world. For fine wine, most experts and collectors inherently know which wine estates have produced the best wines both on a historical basis and – depending upon their level of expertise and experience – on the basis of recent history. The issue is that nobody collects this data or publishes it in a readily available format for consumers. At least, until recently!

Is There A Better System for Consumers to make Buying Decisions?

The Gerard Basset Global Fine Wine Report is in its fourth edition. I founded it, along with my colleague Sasha Lushnikov, because we wanted to find out who fine wine professionals really think are the best of the best. The Report is based upon extensive polling – not of a single expert or small group of experts, but from a wide range of fine wine industry experts, including Masters of Wine, Master Sommeliers, sommeliers from Michelin-star restaurants, wine merchants, brokers and importers, wine press and media.

For the 2022 edition, 943 fine wine industry professionals from more than 100 countries voted, with an average experience in fine wine of more than 14 years. It was by far and away the largest and most diverse fine wine industry poll ever taken.

Votes were weighted based upon industry qualifications and experience: those with top academic credentials (such as the Master of Wine or Master Sommelier qualifications) or more years of industry experience received a higher weighting on their votes.

Deloitte, one of the Big Four international accounting firms, independently verified the results. The winners were given Golden Vines Awards in a number of categories, including Best Fine Wine Producer in Europe, Best Fine Wine Producer of the Americas, Best Fine Wine Producer in the Rest of the World, the World’s Best Rising Star and the World’s Best Fine Wine Producer. If you want to see the winners (and top 10), download the 2022 Report for free here.

Because the awards are given to the very best estates (and not the individual wines), it is much more likely to reach the public consciousness: assessments which award points on individual wines are impossible to remember (and, sometimes, to access unless you subscribe to the journals that award the points). The Golden Vines Award are deliberately designed to replicate the impact of the Ballon d’Or awards in football or the Oscars in film. A much bigger percentage of the population know the winners of these awards because they are simple to remember, and very accessible.

Can this System be Gamed?

The voting rules for the Golden Vines Awards are on the Liquid Icons website [www.liquidicons.com]. Deloitte are specifically requested to look for voting patterns that look suspect, and any other voting activity that could be viewed as gaming the system. We also check for duplicate votes from the same person using different email accounts and scrutinise voters’ credentials and qualifications with the Institute of Masters of Wine and the Court of Master Sommeliers (European and USA Chapters).

Unfortunately, we had to discount a fairly large number of votes this year: a not inconsiderable number of voters claimed to be Masters of Wine or Master Sommeliers but, in fact, were not. Had all the votes been included, the total number of participants would have been well over 1,150 (rather than the 943 verified voters).

Is the System Perfect?

No system for judging wines is perfect, and nor is this one. For starters, the system rates the wineries, not the individual wines, which is equivalent to rating the football teams (for example, Arsenal) and not the individual players (Granit Xhaka, to refer to my earlier analogy).

However, the logic of this assessment is that the best wineries make great wines every year across their entire range. It is much more predictable and noteworthy for a fine wine consumer to know who the best wine estates are than to try and determine the best individual wines from a given vintage. It is also likely to result in better decision making when it comes to purchasing wines, whether for consumption or investment. It also eradicates some of the subjectivity and biases around the choices of an individual wine critic or group of amateur wine lovers’ scoring.

Ultimately, it’s the weighted average of a large group of experts across a range of fine wine activities – from serving to selling, writing and studying – from a large, diverse set of countries who get to vote, which should smooth out a variety of biases that you would expect to see from smaller data sets. What you end up with is more akin to an index of fine wine estates, like the S&P 500 Index or the MSCI Global Equity Index, that is recalculated once a year after all the votes have been counted and confirmed by Deloitte.

Given that wine is slow moving – after all, there is only one new vintage per annum – recalibrating this index once a year is not detrimental to understanding who are the best fine wine estates in the world, in contradistinction to stocks if the S&P 500 Index was repriced only once a year.

Can this System throw up surprise winners?

Of course, one critique of this voting system is that you would expect the same names appearing year after year. In other words, it could be a ‘closed shop’, becoming boring to those who are following the awards. However, in actual fact, there have been some shock surprises.

In 2021, Weingut Egon Müller (pictured above), a sweet wine producer from the Mosel Valley, won the Golden Vines Best Fine Wine Producer in Europe Award. To have a winner from Germany, making sweet wines, beating the very best fine wine estates from Burgundy, Bordeaux, Champagne, Italy, Spain and elsewhere, would have surprised many. In fact, Egon’s wines are sensational and widely appreciated, and he was a worthy winner.

In the same award category, we had an even bigger shock this year. István Szepsy, another sweet wine producer from Tokaji in Hungary, won the award. To have an Eastern European producer of sweet wine beat the best of the rest in Europe is even more extraordinary, and certainly unpredictable. (When Deloitte sent through the results, I almost fell out of my chair.) I personally love István’s wines, but I didn’t realise so many others did as well.

Many of the votes came from sommeliers around the world who know Szepsy’s wines through their wine list. If last year is anything to go by – Domaine Arnoux-Lachaux won the Golden Vines World’s Best Rising Star Award, and saw the secondary market prices of his wines treble after the announcement – you have to be thinking that it might be a good time to buy István Szepsy’s wines.

Who Won the 2022 Golden Vines Awards?

Having changed the question from “What are the best ways for scoring fine wines?” (to which the answer is “None”) to “What are the best ways for scoring fine wine estates?” (to which the answer is “A wide-ranging industry poll”), I can now satisfy your intellectual curiosity and give you the top-10 score rankings for 2022 by voting category – see below.

A note: the logic behind how slow moving the fine wine world is dictates that the positions of the leading estates voted in the 2022 poll will likely not move markedly in the 2023 or 2024 polls; it could take many years for a leading wine estate to fall from grace, and vice-versa, which allows for greater predictability in decision-making for fine wine consumers should their buying decisions be predicated on the outcome of the poll.

Europe

Szepsy Winery (Tokaj, Hungary)

The Americas

Ridge Vineyards (California, USA)

Rest Of The World

Penfolds (South Australia, Australia)

Rising Star

The Sadie Family Wines (Swartland, South Africa)

World’s Best

Domaine de la Romanée-Conti (Burgundy, France)

Innovation

Wine-Searcher (London, UK)

Hall Of Fame

István Szepsy (Tokaj, Hungary)

—

Lewis Chester DipWSET is a London-based wine collector, member of the Académie du Champagne and Chevaliers du Tastevin, co-founder of Liquid Icons and, along with Sasha Lushnikov, co-founder of the Golden Vines® Awards. He is also Honorary President and Head of Fundraising at the Gérard Basset Foundation, which funds diversity & inclusivity education programmes globally in the wine, spirits & hospitality sectors.

Why Wine Scoring Should Be Consigned To The Scrap Heap: Part II