Video 8 – No to numbers

17 Comments

  1. This is the second video I’ve watched (2nd since the crema skimming debut). I have two thoughts:

    1) I’d like to put some distance between the sort of scoring that you SEEM to be referring to (scores attached to consumer-oriented reviews) to professional coffee scoring.

    I’m somewhat with you, if you’re referring to the former (consumer-oriented) idea of scoring. More on that in my #2.

    Professional cupping scores are a different animal altogether. Be it the SCAA cupping form, the CoE form, or some other formally established cupping protocol, there is some level of training and/or familiarity that those cupping protocols require, which (ideally) will yield mostly objective scores. Either a coffee has sweetness or not (and to a certain degree). Either there is acidity or not. Either it’s clean or not. There is such a thing, as I know you agree, as “objective quality.” On the most part, that’s what professional cupping scores pertain to. Those protocols are designed to be as impartial as possible, and (again ideally) shouldn’t vary much from cupper to cupper. “Ideally” has come up a lot, and the practical reality does admittedly differ quite a bit from the ideal. Still, I think you understand my point.

    That said, I’d ask that you clarify which of the two scoring types you’re addressing here, or whether you mean ALL scoring.

    2) A friend, Ms. Linda Smithers, who in her long specialty-coffee industry career has worn many hats (most recently as a marketing person for Daterra) including past president of the SCAA, shared with me her frustration with coffee scoring. Her point was that there exists no established scoring system or protocol that is designed for consumer education or marketing.

    In professional circles, someone might say, “X-coffee is the best I’ve ever cupped! I gave it a 93!” To a consumer, a 93 is 7 points lower than what they’d like to be drinking. It’s similar to the complaints that accompany WBC barista competitions, where someone might say, “The best I’ve ever tasted! I gave it a 5 out of 6!” The current scoring methodologies do not translate to the consumer at all.

    Linda proposes a consumer-oriented scoring system that deducts points based on flaws, shortcomings, or defects. I don’t remember details beyond that, but I think the point is made: a consumer-oriented scoring system must be designed quite differently from the professional cupping protocols in existence.

  2. Scoring coffee is really arbitrary and quite annoying for me. I think theres more value in describing coffees, forming an opinion on what you like and dislike and then using tasting notes to pick the coffees that you want to drink. Great video James, 98/100 ;)

  3. I like your thoughts. As a person new to the coffee scene, learning about cupping scores has somewhat puzzled me especially when comparing two different regions. I don’t understand how they can be objectively compared on a point system. Maybe it’s just that I’m still learning and am still ignorant of many things about coffee.

    This may be a stretch, but when I cup and compare coffees, it makes more sense to me to think about it on a video game level. You know how on video games (usually fighting or sports) often you have several characters to choose from and you get to see their “stats” One player has power 9 speed 4 stamina 4 ability 5. The other has Power 4 speed 8 stamina 6 ability 7. Now if you add those numbers up (22 for the first and 25 for the second) it doesn’t really tell you who is better, it just tells you what happens when you add the numbers up. You eventually choose your character based on what you like about them. Do you like a brute, or someone with finesse and style? It’s pretty subjective, but that’s why you get to chose among several and pick the one that you enjoy the most. Just a thought.

  4. Scoring coffee seems to be a really subtractive or reductive way of thinking about coffee…distilling everything you think is great (or not so great) in to a number.
    In my humble opinion, descriptive flavor profiles are much more useful as a way of determining whether the coffee is going to be great for your particular tastes.

  5. I guess I didn’t really know what you were on about until you mentioned Coffee Review… In selling coffee over a counter to people, it’s completely useless to use scores, because as you pointed out, customers don’t have the same experience as CoE judges and they don’t really care. For them to know it did really well compared to its peers is useful, and that you paid a bucket load of money for the coffee is too.

    Isn’t the idea of Q graders to have their scores between them standardised (hence the rigourous testing)? If their scores are actually consistent between them, then a scoring system is useful for gauging the quality of a coffee from a particular country over a short time span (the inflation argument here). It is not useful for determining if you will in fact enjoy the coffee (i.e. “it’s good for a Sumatran coffee”), and it seems it is not useful for civilians where words describing flavour (and texture!) should be used.

  6. Following up on the video game analogy…

    Often somebody will tell me about basketball game, “Did you see the Lakers best the Celtics? 62 to 58!”

    Which is great. Now I see how those teams matched up – but it doesn’t tell me what went on at all. What good plays were there? How did they move? Well I don’t know any of that – but I do know that when they stack up that is what happens.

    In coffee it’s similar. A score doesn’t tell me what goes on, just the result. But what other way do we have of immediately knowing how well a coffee did? And as a roaster, how do I know which coffees are even worth trying? Not exclusively do I use numbers… but it has a lot to do with which ones I choose to sample (and therefore carry.)

    Sometimes tasting notes don’t tell you how good it is. Sometimes scores don’t tell you which “vibe” it has. Both of these systems need something else to give me a good representation.

  7. Robert Parker…
    In the States we love to quantify everything. Including quality. Which is an absurdity. But then I remember Robert Parker. Since I have a friend in the wine importing business, I know just enough to be dangerous. After tasting many wines I have come to respect and trust Robert Parker’s judgement above all others. When I see a “Robert Parker: 92 points” for $12.99 you bet your ass that’s the wine I’m buying.

    Problem with Grocery Store Coffee…
    But I don’t think scores would be useful or meaninful or even correct in the grocery store where most coffee is purchased by consumers, due to the obvious problems with freshness that wine is not subject to. Coffee Review will continue to ply its trade and have its followers and detractors. Their repuation and trustworthiness will determine their future, just has Robert Parker’s has determined his.

    Let’s Not Sweat it…
    The rest of us who cup and score for our own internal decisions will continue to do that to our best ability and sometimes compare our 92′s with Ken David’s or some other experienced peer. Scores are just tools and thankfully we have many in the box (I hope). All types of scoring needs to be taken with a grain of salt. Take what you can get and leave the rest. But mostly, enjoy.

  8. Well, aside from thinking that scoring outside a process like the COE cupping is like throwing darts blindfolded, a score does not tell me (as a consumer) a whole lot about the coffee. For example, I was at a very well respected Roastery to buy coffee and they proposed one coffee which was their favorite. It was obviously pretty good, but it just didn’t “fit” me. You can’t easily put a one-dimensional score on a multi-dimensional thing like the taste of coffee, especially when people “look” for different things they expect. So, some people (like me) like Aricha microlots, others don’t, and there’s nothing wrong with that.

  9. i agree with your thoughts and many of the others. Distilling a coffee down to numbers is next to useless for the average person, and even to most of us new baristi. Flavour profiles seem to be the easiest and most useful way of describing a coffee, letting a consumer know what you thought of the coffee when you tasted it, or even objectivly what flavour notes were present in the coffee. A ’94′ point coffee on its own tells you nothing about it. And combined with a flavour profile it just looks like a selling point.

    The easiest guidance i’ve found is usually the bean grading plus a Flavour profile from someone you trust. Especially if there tastes are similar to yours.

    The only other useful thing i could think off is a relativly small scale for body, acidy and strength which alot of us already utilise along side a profile.standardised scales would allow printed numbers for packages.

  10. You can’t judge a book by its cover…But you DO buy it based on limited but trusted knowledge.
    The process of describing coffee will never replace the way a person experiences it…nor should it try.
    But setting up a trusted system to guide people into a higher probability of satisfaction should be what
    scoring, descriptions, and judging is all about.

    -Chris

  11. We have never used a cupping score to describe our coffees at the shop. In our first year, I decided that in our area (basically devoid of truly specialty coffee for 35 miles in every direction), a score would not really educate our customers as to the virtues of of any particular bean. Something we have done lately, however is to use descriptors from the “fast and dirty” score sheet we use to decide whether we want to pursue a sample in further cupping ans assessment and finally into a purchase. It goes something like this:

    Quick Cupping Form

    MENTALITY (overall character)
    CALM BALANCED WILD

    CURVES (body)
    PARIS ANGELINA SALMA

    WIND (nose)
    ZEPHYR GUSTY TEMPEST

    MOTION (acidity)
    COUCH POTATO JOGGER FASTER PUSSYCAT KILL, KILL

    INTELLIGENCE (complexity)
    GOMER JOHN Q. EINSTEIN

    MEMORY (aftertaste)
    WHAT? THERE IT IS TOTAL RECALL

    GRADE
    F D C B A

    We circle one from each category with a note or two out to the side for particular flavors and end up with a grade-type quick score that is based totally on our enjoyment of the coffee. If a majority of us didn’t give it an “A”, we drop it from the list of potentials. It may all seem a little silly, but we have seen good returns in our customers being able to identify different characteristics in their cups.

  12. What score did last year’s Best Picture winner at the Oscars get? What score did last year’s Booker Prize winner get? Does it matter that we don’t know? Does it really matter, other than from a marketing perspective, who even won? It still doesn’t mean we’re going to like either.

  13. Forgive my injection here about origin, for the coffee buyers world (exporters, importers and roasters) score is extremely useful for communicating with producers and formulating purchasing protocols. In fact in my work it is essential. The reality of hundreds of samples needing to be efficiently classified at origin leads us to score.

    When a set of cuppers become “calibrated” (meaning they all agree upon basic descriptions of levels of good) then they can assign points and use the “Cup of Excellence” style quality discovery, and assign appropriate pricing.

    I absolutely believe that amongst experienced cuppers there is a reliable common understanding and recognition of what “good” resembles in washed arabica coffees, even when evaluating multiple origins. Correspondingly, I believe that our point assessments are probably within 3 points of each other between 75 and 90; and more distanced the lower and higher we go. So for purchasing specialty washed arabicas the point system and pricing is a very valuable and workable model for us.

    Where I concede this all breaks down is when alternative processing methods (some represented by particular origins) are employed. Fruity, Winey, Fermenty, Earthy . . . (natural, semi natural, milled moist . . .) The relativity of cupping goes wild.

  14. My feeling is that coffee knowledge is too disjointed, and without known labelling (down to estate/lot/microlot) and less subjective judging, the numbers are rubbish. Added to different brewing methods – coffee varies wildly from siphon espresso (and even in espresso from HX DB, group, preinfusion, brewing temp etc).

    If we’re cupping one bean against another from the same region (eg. a Sidamo against another Sidamo) using the same scoring method and the same, the scores break down well. Cup-tasting espresso adds more complexity and thus difficulty, and we’ve not as an industry got the same level of familiarity with regions/estates/soils/processing methods.

    perhaps greater accuracy in what we’re actually communicating we’ve tasted (and how we got there) is actually more important, before we start tells consumers “that’s a 91 pointer, while that’s an 88 pointer”.

  15. G’day James,
    I agree with your immediate questions of any score from a reviewer: ‘What have you tasted’ and ‘what criteria did you use’. Like wine, coffee taste is essentially personal and numbers make what are in reality subjective judgments appear objective – but that is where the ‘super reviewers’ you mentioned (I think in a negative sense -?) perhaps can be of some benefit. From my experiences with wine (I haven’t come across too many coffee scores yet – but I keep exploring) I find that if a number of well known reviewers pass judgment on a particular drop, you can get some very useful insights into what will appeal based on the differences in marks presented. Of course, the wine industry has existed for so long that the wine-reviewing industry now exists almost in its own right. I’m not sure the coffee industry is in a state yet where there are enough established reviewers to make scores of much use as a communication tool between producers and consumers. Word of mouth, and personal trial and error seem to be much more effective – particularly at the consumer’s end of the market where you can start tasting and experimenting for the price of a cup of coffee… (it’s cheaper than a bottle of wine!)

  16. Yes i have to agree i work for Masteroast in Peterborough and they have just gone to a number system and i dont like it

  17. This is why I love this blog. First off (not saying they are predictable), I see a trend in all the videos that I can use to predict most of the themes. What I have noticed from your views is that coffee is a personal experience. It is NOT black and white, good or bad. It’s what it is and what the person makes of it. We all like different foods prepared differently, so who’s to say coffee is any different?

    Love the video

Submit a comment