Vancouver is a lovely city, but it seems it likes to rain. So we’ve all been cooped up inside testing the six machines submitted (I am glad I am not the one paying to run 6 three groups all day!)
People probably want to know how this works and I will try and explain, but I warn you it isn’t massively exciting. As a tester I have no idea how the machines performed. The Scace devices run straight into a datalogger and then into a laptop which is kept out of view. I will never get to see the data, unless a manufacturer chooses to publish. Also a manufacturer will only see their data and not the other machines.
The testing protocol is pretty simple. Each round consists of 14 shots pulled at varying intervals, with the device removed at certain points to simulate grinding, as well as knocking out the puck and flushing. Each machine has four rounds of testing: middle group only, middle group and steaming, 1st and 3rd group only and 1st and 3rd gorup with steaming. With the latter rounds two people worked the machine, even though it was replicating one barista using this. We did this to make sure we could be as consistent as possible in the collection of data across machines. With the steaming we had tested each machine to see the time it took to steam 10oz of milk up to 60C, and then that time was used during the testing.
It hasn’t been the most fascinating testing to watch – every manufacturer was present with a technician during the testing and I think the constant counting down each action, second by second, was close to driving everyone a little crazy!
I think the testing has been very fair, I think the protocols are very good (though I don’t think anyone is claiming that it is beyond improvement). If anyone has any questions I will try and answer them though if you have issues or complaints I am probably the wrong person to speak to. Everyone seems very happy with the fairness of it, which is good.
Next up is grinders which will take a while, and I will write a bit more about that after we have done it.