If anything can kick start me writing about beer, it’s nerdy maths shit.
The 2017 BGONZAs (Brewer’s Guild of NZ Awards) have been announced and for the first time the individual award data has been released with them. Phil Cook has done an excellent review of the data and derived some statistics. To be honest it’s pretty great, go read it.
He comes up with two statistics that give a good baseline for determining who did well at the awards. Here they are in Phil’s own words:
Medal Percentage (MPC) is the proportion of all beers entered that earned any kind of medal at all. The overall medalling rate for the competition was 52% (up slightly on the previous year) and it turned out fairly well distributed; breweries in the middle of the rankings scored around this level. It’s worth noting that a beer that earns nothing is either significantly faulted or disqualifyingly “out of style” or both — though we don’t know which without access to the judges’ notes.
Points Per Entry (PPE), on the other hand, assigns gold medals 3 points, 2 for silver, and 1 for bronze and divides the result by how many beers the brewery entered all up. This scoring reflects the fact that the medals aren’t unique, like at the Olympics; a category might have a half-dozen golds, for example ― and it’s the scoring the Guild itself uses to calculate the overall Champion Breweries. This year, a PPE over 1.0 put a brewery in the top third of the field as a whole.
They’re simple and do what they say on the tin. But PPE requires a weighting and weightings can be disputed, which is what Ralph Bungard of Three Boys’ Brewing did on Beertown.
I thought this: the simple one, two and three points of bronze, silver and gold does not really reflect how I, and probably every brewer, feel about the value of those awards. I think I would rather have one silver than two bronze medals, but think I might take four bronzes over one silver – you see where I’m going here? I reckon that if a bronze had a value of one point, then a silver should maybe be three points and a gold say six points and of course zero for no award. This probably reflects how most brewers would largely value those awards. I’m pretty sure statisticians would call these ‘weighted’ values for those awards and if we average those weighted values for every brewery it would be a weighted average.
He’s right, that’s what we call them. What’s interesting is, it sort-of doesn’t matter. As long as we all agree that gold is worth the most, silver the second most and bronze the least, then your rankings are going to shuffle out roughly the same.
However, Ralph touched on an interesting point: what is a gold medal worth? As he says above, “I think I would rather have one silver than two bronze medals, but think I might take four bronzes over one silver.”
But is a bronze in one category worth more than silver in another? Or even more than gold? I decided to try a new form of derived statistic that I have called “worth”.
Worth
Firstly, if I was doing this seriously I would be trying to pull in sales data. The literal value of having a medal on your label or having the publicity of an award could be measured in dollars. But that would take a lot of work and would also require every brewery happily handing over their sales data.
Lacking both time and influence, I simply ran the award numbers and came up with two measures.
Medal Worth (WMED) For each award category, I took the total number of winners (w), the total number of entries (n) and then 1-(w/n). This gives me an idea of how difficult it was to get a medal in each category (with 1 as the maximum value). For example, European Ale had 31 medals from 48 entries, giving a value of 0.35; whereas International Lager had 59 entries but only 22 medallists, for a value of 0.63.
So for each brewery you take the number of beers that won in a category, multiplied by the value for that category, then sum across all the categories. To get their MWED you divide by the number of beers they entered.
Relative Medal Worth (WGSB) This calculation is similar to MWED except more complicated. WGSB is based on the value of gold, silver and bronze in a category. The gold value of each category is based on the number of golds awarded per number of entries. The silver is the same but based on the number of silvers AND golds. The bronze values are calculated the same way as MWED.
In this way you can see that was harder (and hence more valuable) to get a silver in the Cider & Perry category (0.90), than a gold in the Wheat (0.89).
To get the final WGSB for a brewery, you sum all the values multiplied by medals and then average across all beers entered.
The idea behind this is to capture the value of a medal in a fashion that isn’t an arbitrary number. The single gold from a field of 59 International Lagers is worth a lot more than either of the two golds from 18 entrants in the Wheat category. Relatively speaking that is, they’re all good beers.
The downside of this method, as well as PPE, MPC and Medal-x (Ralph’s weighting), is that it favours breweries who have only entered a few beers. In Phil’s analysis he only included breweries that entered 10 or more beers, which seems fair. The average number of beers entered is 9, so I’ll choose that as my cut-off. (As an interesting side note the most common number of beers to enter was two).
Before I do cast those breweries adrift I should mention that Epic and Altitude both did very well under all the systems.
Results
First up here are the values for each category:
WMED | WGSB | |||
Class | General medal | Gold | Silver | Bronze |
British Ale | 0.47 | 0.96 | 0.73 | 0.47 |
Cider & Perry | 0.52 | 1.00 | 0.90 | 0.52 |
European Ale | 0.35 | 0.90 | 0.73 | 0.35 |
Flavoured | 0.40 | 0.93 | 0.72 | 0.40 |
Flavoured Cider & Perry | 0.59 | 0.91 | 0.77 | 0.59 |
International Lager | 0.63 | 0.98 | 0.80 | 0.63 |
New Zealand Lager | 0.41 | 0.94 | 0.73 | 0.41 |
Pale Ale | 0.54 | 0.95 | 0.77 | 0.54 |
Specialty | 0.36 | 0.91 | 0.70 | 0.36 |
Stout & Porter | 0.57 | 0.93 | 0.81 | 0.57 |
Strong Pale Ale | 0.56 | 0.93 | 0.80 | 0.56 |
US Ale | 0.41 | 0.93 | 0.64 | 0.41 |
Wheat | 0.67 | 0.89 | 0.83 | 0.67 |
The rare outlier is gold for Cider & Perry, because there weren’t any. Making this the most valuable medal. Oddly while the Wheat category was the hardest one to win a medal in, the value of each individual medal is lower. While US Ale sees the biggest drop in value from gold to silver, with only 5 golds, but 21 silvers awarded.
How do these values work out to the overall breweries?
WGSB | WMED | |||
1 | North End | 0.64 | Liberty | 0.38 |
2 | Liberty | 0.62 | Sawmill | 0.38 |
3 | Sawmill | 0.62 | McLeod’s | 0.36 |
4 | McLeod’s | 0.61 | North End | 0.35 |
5 | Brave | 0.59 | Brave | 0.34 |
Now I don’t think for a second that my crazy numbers should replace what the BGONZAs currently do. Nor do I think that any of those breweries should challenge Garage Project for overall champion. These calculations are simply to show these breweries that, “You probably thought you did well, and you really did!”