How Tastings Tastes Wines, Beers, and Spirits
We’re aromatic, deeply extracted, full-bodied, muscular, and jump from the glass with lovely legs and an exotic lingering aftertaste. Now if you want to know how we conduct our reviews, read on...
Our Tasting Lab
To make our information as consistent as possible,Tastings powered the Beverage Testing Institute uses a dedicated tasting lab in Chicago. This room was specially designed to minimize external factors and maximize our panelists’ concentration. Tasting at the same time of day practically every weekday morning, under the same ideal conditions, is far better than working out of a suitcase or at a producer’s facility. We only taste 25-30 wines, 25-35 beers, or 12-18 spirits per day to prevent palate fatigue. Tasting more than these amounts in a single day, as most competitions and many magazines do, negatively effects the accuracy and quality of alcoholic beverage evaluations. And of course, we always spit.
We rely heavily on highly experienced, professional guest tasters who are either retailers, restaurateurs, or prominent writers that are especially knowledgeable about the beverage category being reviewed. All panelists are rigourously screened and audited and then trained in our proprietary blind tasting methodology. Our director, Jerald O’Kennard, moderates the panels. For a recent list of panelists, click here.
A basic tenet of qualitative evaluation is that the shorter the scale, the easier it is for tasters to repeat their scores — and repeatability is the essence of accuracy. Our tasters now use a simple scale with four bands. In the first round we use this range:
With this scale, tasters needn’t concern themselves with the difference between an 88 and an 86, or a 90 and a 91. Wines with two scores of "4" are sent to the "merit round," whose range is as follows:
This allows our tasters to consider general quality, without the distraction of adding points. Products breaking the critical 90-point barrier are tasted twice without exception, virtually guaranteeing that they will deserve their accolades. Also, after the first round, those with a wide disparity in score are re-tasted. We also re-taste many that scored poorly, to make sure a bad sample was not at fault.
The scores are then translated onto a modified 100-point scale. We think the five "bands" below more accurately reflect the quality of products in today’s market. It roughly corresponds to a five-star system:
To help translate the panels’ scores, we use a different statistical method, relying on the mode (the most frequent score) rather than the mean (the average score). This helps reduce the impact of maverick scores; the mode is much closer to what the panelists are really saying. If, for instance, a product in the first round receives three scores of "3," it is placed in the upper center of its band (85-89) and given a final score of 88 points. Should the third score be a 4 or a 2, the product in question would be given an 89 or an 86, respectively. The third score is used to move the total score up or down within the same band. Again, controversial permutations are re-tasted. Those falling in the lowest band (less than 80) are simply listed as "not recommended"; no score is assigned.
Best Buy. Wines or spirits which provide uncommon value.