Comments on: WBC Scoresheets – a few thoughts

By: Devin

Devin — Thu, 24 Dec 2009 15:30:22 +0000

I definitely agree with you on the wording / number scoring. If I were to judge the espressos I can get around me (Albany, NY area) most would receive a 0, a few would receive a 1, even less a 2, and only one would receive a 3 or a 4.

Only 50% score for that one good espresso? I think they deserve a bit better than that. And I hate having to give a miserable 33% to my favourite coffee house. Oh dear.

Like or Dislike: 0 0

By: Rob Dunne

Rob Dunne — Fri, 11 Dec 2009 02:48:36 +0000

I’m agree d on that one, if judges are to judge, they themselves should be able to explain the details, offer constructive advice and I think its more credible if they have or can preform the task at hand.. producing well made coffee & milk. This would solve a lot of frustration on the Barista’s side of thing’s. If the industry as a whole is to evolve and improve, the judging itself needs to evolve along with the every charging Barista’s approach to it.

Like or Dislike: 0 0

By: swiss made

swiss made — Mon, 30 Nov 2009 13:35:27 +0000

My 2 cents as a judge for drum competitions:
I think you will always fail with wording if it has to fit for all parts of judging. I recommend separate and significant explanatory keywords for each part like taste, tactile, crema etc. (eventually more than once for each point).
Such an “explanation table” will be more helpful for the judges on stage but also for the competitors after getting their scoresheets. It decreases also the amount of notes on the scoresheets (which is still a good thing) and for regionals each keyword can correlate to some basic advices.

A lot of them is just a “write down of what is going on in a judge’s head” but it will also causes some interesting debates I think…

Like or Dislike: 0 0

By: Anette

Anette — Mon, 30 Nov 2009 09:41:27 +0000

Had to rummage around the dictionary a bit, trying to find words that I as a non native English speaker could sort of see fitting onto a scale of 0-6… Not easy to find the ones that have the right connotations for everyone, some of them sound a bit harsh and others just condescending… How to choose the right ones that everyone will relate to and not be upset by?

Based on a few school and university grading systems, a few ideas:

0- Fail/Poor/Inadequate/Insufficient/Unacceptable/Unsatisfactory
1- Marginal/Adequate/Sufficient/Passable
2- Fair/Satisfactory/Acceptable
3-Good
4-Very Good
5-Excellent
6- Outstanding/Superior/Exceptional

Like or Dislike: 0 0

By: Erik Fooladi

Erik Fooladi — Sun, 29 Nov 2009 12:14:25 +0000

With the risk of making myself a laughing stock, I’ll take my chances on commenting anyway. I’ve in fact never been to a barista competition, but this discussion resembles what occurs in school grading and maybe the latter might be of relevance anyway, although posing a slightly alternative view on the main topic.

Scale
======
A major question in Norwegian school system lately has been: Is a graded scale contructed from the top and downwards, or is it a scale in which each grade has its own well defined value? In evaluating an espresso: for each tasting, are you looking for how far below 6 that espresso is, or are there well defined criteria for the different grades? The two extremes of giving i.e. grade 3 would be:

either

“because this cup lacks this, this, that, that and this virtue, it is three grades below 6 and is thus given grade 3″

“this cup has got this, this, this, this and that virtue. Hence we would give it a grade 3. For it to reach grade four, it would in addition need to have this and this virtue”.

The difference is: the first way of seeing it would be to define the result as how far one is from the ideal (rating 6). The second would be to value it for what it is and give indications of how to do better next time (ref. Anthony’s comment above). The last version is considered to be more constructive that the first, helping the student (barista) to make an even better effort next time.

Of course, the last model is quite labourious for the judges in beforehand. Criteria must be constructed for all grades prior to the competition. However, when this is done the scoring might be both more consistent and transparent. In that respect, I’d say that Tristan’s descriptions in parentheses are more precise and should be easier to judge by than the ones in bold (the words in parentheses are indeed one step on the way to consistent criteria).

Yes, there are indeed great differences between baristas attending a competition and pupils attending compulsory education. But who knows, maybe there might be relevant meeting points anyway.

Like or Dislike: 0 0

By: Tristan

Tristan — Sun, 29 Nov 2009 11:20:38 +0000

Wording should be more like this I reckon (with dictionary definitions just to show it is a linear progression):

0 – Unacceptable (not satisfactory)
1 – Poor (worse than usual, expected, or desired of)
2 – Acceptable (adequate, satisfactory)
3 – Good (desired or approved of)
4 – Fine (of high quality)
5 – Excellent (extremely good)
6 – Exceptional (unusually good, outstanding)

The only problem with this wording is that ‘Poor’ is sandwiched between ‘Acceptable’ and ‘Unacceptable’, boding the question: is it acceptable of not?!

I did thin about using ‘hopeless’ or ‘crappy’ instead of unacceptable, but thought against it! The fact of the matter is that there has to be a word in between acceptable and unacceptable otherwise the leap is just too big. For that reason I placed ‘poor’ in the no-mans land and I think it works, especially when backed up by the definitions.

I don’t think that there needs to be descriptors for half points. The descriptions are there to give a numeric reference, not the other way round. If a drink is too good to be ‘Good’, but not good enough to be ‘Fine’, it has to be scored a 3.5, exactly what English word that translates too is irrelevant, the points have been scored.

I am of course in agreement that the scoring should be backed up by notes as much as possible.

Like or Dislike: 0 0

By: Anthony

Anthony — Sat, 28 Nov 2009 21:08:53 +0000

As a competitor I’ve always looked at the comments on scoresheets as an overture to further conversation. Even for those of us who take care to time the presentation of information appropriately, there’s not always enough time for a judge to listen to what I’m saying, then experience the beverage, and then make extensive notes without possibly missing crucial details of what I’m saying. When judges are available to me at length after a competition, my scores improve the next time out.

Personally, the numbers never mean much to me.

Like or Dislike: 0 0

By: James Hoffmann

James Hoffmann — Sat, 28 Nov 2009 17:35:05 +0000

I am pretty sure Fritz is a judge isn’t he? You can’t forget the Storm….

I didn’t go for WBC certification for very practical reasons – either Anette and I could travel to the certification and we thought she was the better candidate. I hope to get more involved in the future. I will give Gwilym a nudge too!

I will also put together a few suggestions for the R&R committee in the next couple of days.

Like or Dislike: 0 0

By: Eirik

Eirik — Sat, 28 Nov 2009 10:04:02 +0000

I’ve certainly been frustrated about the words in the scoresheet many times, as I like to use them as a guidance and maybe also as communication back to the competing barista.

The most problematic one is “Average”. What does that mean? Average of what? All the espressos served in the world? In the WBC? In this particular competition? I’ve always wanted to change that to “Mediocre”.

I like your suggestions for changes in the scoring sheet. I like the space for more comments, and I agree that it is really important to give good comments, and some hints. However, I see Nickchos point on the judges not playing barista..

Interesting discussion anyway.

Best,
E.

Like or Dislike: 0 0

By: Brent

Brent — Sat, 28 Nov 2009 08:33:18 +0000

Some good points and suggestions here. The WBC scoresheets could absolutely benefit from a little re-wording and consolidating. I would agree that a scale is needed and the words help judges find some meaning/relativety in the numbers. As Nick noted, calibration is more important than the wording. Its all relative, and I discourage people from comparing scores between competitions. The scores can be inflated or deflated by a lot of different factors, including the overall quality of the competitors, who goes first, etc.

I would agree that the circles probably don’t belong on the score sheet as they relate to only one score in each category and they encourage judges to create notes and artistic renderings that we have never provided training on. (Why don’t we just use photos anyhow to help better communicate to the competitors what the actual beverage appearances were? We have the technology.) I’d rather that they focus their time on the taste balance, a much more valuable score, than trying to draw a picture of the beverage. (And who can draw a nice looking espresso with a pencil anyhow?)

One item in wording that I see many judges get hung up on is in the list of words that follows Taste Balance of Espresso. “Sweet, Acidic, Bitter.” As was explained to me by the person (who claims to have) created that list, it was never meant to be an exhaustive list of words. It was just a list of possible examples. In the rules, we changed the language to say “…a balance of sweet, acidic AND/OR bitter.” Still, I see some judges (esp. new ones) using that as a checklist. A balanced shot does not have to have ALL THREE OF THOSE to be a balanced shot and I think the score sheet words should make that more obvious. (And its not intended to be an exhaustive list of the possible elements that create balance. Maybe it even needs an “etc.”)

As part of the newer judges training programs we are piloting in some NB’s, we ARE in fact requiring judges to prepare shots of espressos in teams. We have also tried having tech judges pull shots, not necessarily in a flight of 8 drinks as the logistics of that are not always reasonable. However the next time new WBC Judge certifications are done, its very likely there will be more hands-on and baristaing requirements for WBC tech judges. Considering the current pool of seven WBC certified tech judges, I am highly confident that ALL OF THEM could pull 8 espressos and capps successfully. The best tech judges I know are also excellent baristas.

I agree that tech judges should be able to offer some suggestions with their feedback, such as “try using more coffee?” or “try slowing down the shot/fining up the grind?” but that has to be based on communicating with the sensory judges, as you suggested, to get a complete picture of what happened. And these suggestions are probably best delivered face-to-face vs. on the score sheet.

Some of these comments and suggestions would be worth communicating on the WBC website where they can be submitted to the WBC R&R Committee:
http://www.worldbaristachampionship.com/contact/rules-submissions

And finally James, why are you not a WBC Certified Judge??? (And why do we only have ONE former WBC Champion – Morrissey – who is, or attempted to be, a WBC Judge?) :) We would all love to see more of the former champions judging in the WBC!

Like or Dislike: 0 0