Testing and Evaluation: Our Protocol for Espresso Machine Reviews

Why a standardized test protocol?

When buying an espresso machine, you are faced with an overwhelming selection: from a compact thermoblock machine for 400 euros to a dual-boiler monster for 4,000 euros – how are you supposed to compare them? That is exactly where our test protocol comes in.

We don't just want to tell you "this machine is great" or "that one is bad." Our goal is to provide an objective, measurable, and comparable assessment based on hard facts. We focus on the result in the cup, complemented by measurable technical parameters and a fair economic classification.

All results are rated on a scale from 0 (unsatisfactory) to 10 (outstanding) and then aggregated into a total score using weighted values.

Overview of all espresso machines we have tested.

How to read this protocol

1. Look at the total score first – it gives you a quick overview of the overall quality
2. Identify your personal priorities – What is important to you? Espresso potential? Steaming performance? Energy efficiency?
3. Focus on the relevant individual scores – Ignore categories that are unimportant to you
4. Only compare machines in similar price ranges – A score of 7.5 at 800 euros can be better than 8.0 at 2,500 euros

Weighting: Not all criteria are equally important

To do justice to the different relevance of the aspects, the categories contribute to the overall rating with different factors:

Factor 3 (High – The Core)

Espresso potential (3×): The most important criterion. How good is the espresso that can be squeezed out of the machine at best? Here, technical prerequisites merge with sensory validation.

Factor 2 (Medium – Technology & Haptics)

These categories significantly influence the daily experience and technical performance:

• Temperature management (2×): Consistency and adjustability
• Volumetrics (2×): Precision of automatic volume programming
• Steaming quality (2×): Power and speed when steaming
• Build quality (2×): Material, workmanship, and longevity
• Usability (2×): Workflow, cleaning, and ergonomics
• Heat-up time (2×): Time to reach full operational readiness
• Power consumption (2×): Energy efficiency during operation

Factor 1 (Low – Additions & Special categories)

These aspects are relevant but less decisive for most users:

• Noise level (1×): Noise during operation
• Accessories (1×): Quality and scope of delivery
• Volume potential/Catering (1×): Performance during high-volume use
• Price-performance ratio (1×): The economic classification

Calculating the total score

The sum of the factors is 19. The result is a score between 0 and 10.

The quality classes

Depending on the total score, we assign the machine to a quality class:

The heart of our test

Why this category is the most important

In the end, only one thing counts: How does the espresso taste? A machine can be technically impressive, but if the result in the cup is disappointing, everything else is secondary. That's why espresso potential contributes to the overall rating with factor 3 – more than any other category.

Our hybrid approach: Technology + Sensory analysis

Espresso potential results from two pillars:

• 1. Technical prerequisites (max. 8 points) – objectively measurable
• 2. Sensory validation (max. 2 points) – panel test

Technical prerequisites (8 points)

A. Temperature performance (0-3 points)

The score from Chapter 6 (Temperature Management) is used as a basis here:

• Score ≥ 9.0 in Ch. 6 → 3.0 points
• Score 8.0-8.9 → 2.5 points
• Score 7.0-7.9 → 2.0 points
• Score 6.0-6.9 → 1.5 points
• Score 5.0-5.9 → 1.0 points
• Score < 5.0 → 0.5 points

Temperature stability is the basis for good espresso. A machine with excellent temperature management already has a major advantage.

B. Profiling & brewing control (0-3.5 points)

Manual control (0-3 points):

• 3.0 pts: Flow + pressure control: everything is possible. Formulation of if-conditions, stages, volumetric control
• 2.5 pts: Programmable pressure profile control
• 2.0 pts: Adjustable pre-infusion (time/pressure) OR simple pressure profiles
• 1.5 pts: Fixed (sensible) pre-infusion available
• 1.0 pts: Manual profiling possible via paddle/lever
• 0 pts: No profiling options

Smart profiling bonus (+0.5 points):

Some modern machines have adaptive systems that monitor the flow during extraction and intervene automatically. If the machine detects too fast a flow (an indicator of too coarse a grind), for example, it automatically reduces pressure or extends the pre-infusion. This significantly increases error tolerance and enables even less experienced users to achieve consistently high-quality results.

Maximum: 3.5 points

C. Hardware hygiene (0-2 points)

Shower screen quality & water distribution (0-1 point):

• 1.0 pt: High-quality shower screen, high-quality filter baskets, uniform distribution
• 0.5 pt: Solid standard shower screen, average quality filter baskets
• 0 pt: Defective distribution, susceptibility to channeling

Thermal mass & pressure build-up (0-1 point):

• 1.0 pt: Heated portafilter, stable pressure build-up
• 0.5 pt: Partial thermal mass or slight pressure fluctuations
• 0 pt: Cold portafilter or unstable pressure

Sensory validation (2 points)

Technical values are the foundation – but can you taste it?

Panel test by Q-Grader Benjamin Hohlmann with reference coffees whose ideal profile is known.

Evaluation:

• +2 points: Machine fully implements technical potential. Espresso shows balanced harmony, no extraction errors
• +1 point: Technical potential is largely implemented, minimal deviations
• 0 points: Significant discrepancy between technology and cup result

We evaluate the machine's ability to produce fine-pored, homogeneous microfoam for latte art. This assessment is carried out by our professional barista Michel Indelicato, who has steamed with around 100 different machine types in test scenarios.

Categories (max. 10 points)

A. Steam power

Pressure and the ability to swirl 300ml of milk in a 0.6l pitcher. The more tilted a pitcher has to be held, the less powerful the machine is.

• Pitcher vertical (0°) → Strong power
• Slightly tilted → Good power
• Steeply tilted → Weak power
• No rotation → Unsatisfactory

B. Speed

Time required to heat 300ml of milk from 8°C (fridge temperature) to 60°C. In this scenario, the milk is deliberately not frothed, only heated (steam wand inserted deeply).

C. Workflow & ergonomics

• No-burn steam wand or insulated design
• Ball joint (360° freely movable)
• Rotary valve (precise steam quantity control) vs. simple toggle switch

D. Condensation water

How much condensation water or residual moisture is ejected by the machine before steaming can begin? Too much condensation water dilutes the milk.

The robustness test

We test not only whether volumetrics are present, but how robust they are under changing conditions.

The 5-stage stress test procedure

Setup:

• Coffee: Apas Espresso (medium roast)
• Grinder: Mahlkönig E64 GbW
• Brew ratio: 1:2.5
• Target size: Dependent on filter basket size, remains set constant

The 5 stages:

1. Hardware basis (5×): Scace portafilter (simulated resistance) – testing pure technology without the influence of the coffee puck
2. Reference espresso (5×): Real conditions with standard grind
3. Profile check (5×): Test with complex profiles (e.g., pre-infusion) to check if the logic remains stable with pressure fluctuations
4. "Coarser" stress test (3×): Grind is set a full notch coarser (faster flow)
5. "Finer" stress test (3×): Grind is set a full notch finer (slower flow)

Goal: Does the machine recognize the changed flow (flow meter) or does it stop stubbornly by time?

Evaluation grid (0-10 points)

Base points (Technology type):

• 0 pts: Manual (no volumetrics)
• 3 pts: Time-based control only

Precision points (based on average deviation):

• 4 pts: ±4g deviation
• 6 pts: ±2g deviation
• 8 pts: ±0.5g deviation (scale level)

Bonus (+2 points max):

• +1 pt: Exact UI input (grams/milliliters instead of impulses)
• +1 pt: Integrated scale

For linked scales, we perform the protocol twice: once without, once with the scale.

This category evaluates how pleasant the machine is in everyday life. A technically brilliant machine that is annoying every day loses points here.

Point distribution (max. 10 points)

• Interface & controls (max. 3 points + 0.5 bonus)
• Daily routine (max. 3 points)
• Cleaning & maintenance (max. 3 points)
• Documentation (max. 1 point)

A. Interface & controls (0-3.5 points)

For machines WITH display/software:

• 3.0 pts: Intuitive, no manual needed, fast reaction time (<0.5s), easy to read
• 2.5 pts: Good, short familiarization, display still readable in sunlight
• 2.0 pts: Solid, manual occasionally needed, acceptable readability
• 1.5 pts: Cumbersome menu navigation, frequent reference required
• 1.0 pts: Confusing, slow reaction, poor readability
• 0.5 pts: Frustrating to operate

For machines WITHOUT display (mechanical/analog):

• 3.0 pts: High-quality controls, clear pressure point, precise rotary knobs
• 2.0 pts: Solid haptics, functional
• 1.0 pts: Wobbly feel, imprecise settings
• 0 pts: Cheap plastic, massive play

Bonus: +0.5 pts for programmable shortcuts/function keys

B. Daily routine (0-3 points)

Each fulfilled criterion gives 0.5 points (max. 3):

• ✓ Water tank easy to remove/fill without jamming
• ✓ Water level clearly visible at all times or "Last Shot" technology
• ✓ Drip tray sufficiently dimensioned (>500ml) and easy to remove
• ✓ Portafilter moves smoothly, locks securely without effort
• ✓ Integrated water filter in the tank present
• ✓ Ergonomic arrangement of all daily used elements

C. Cleaning & maintenance (0-3 points)

• 3.0 pts: Automatic cleaning programs present AND shower screen easily removable with a spoon or provided tool
• 2.5 pts: Automatic programs present, disassembly partially tool-free, tool not included
• 2.0 pts: Manual cleaning easy, clear machine indications for maintenance cycles
• 1.5 pts: Manual cleaning possible, partially fiddly
• 1.0 pts: Cumbersome disassembly, unclear maintenance instructions
• 0.5 pts: Hard-to-reach parts, no maintenance instructions

D. Documentation (0-1 point)

• 1.0 pt: Excellent manual (clearly structured, good illustrations, complete)
• 0.5 pt: Acceptable manual with minor gaps
• 0 pt: Defective or incomprehensible manual

Penalty: -1 point for significant "annoyance factors" (excessively loud signal tones, unnecessarily complex procedures, long inexplicable waiting times)

The technical foundation

Why temperature is so important

Brew temperature is one of the most critical parameters in espresso. A difference of just 2°C can drastically change the taste: too cold = sour and under-extracted, too hot = bitter and burnt. We do not rely on manufacturer specifications, but measure ourselves – under standardized conditions.

Our measurement setup

Temperature measurement:

• Device: Scace 2 with data logger
• Calibration: Before every test series with boiling water
• Measurement: Directly in the portafilter

Power measurement:

• Device: Christ CLM1000 Professional (Plus)
• Resolution: 0.1 Wh (0.0001 kWh), readings every second
• Logging: Saved as a CSV file

Environmental conditions:

• Water temperature in the tank: 20°C (±2°C)
• Target temperature: 93°C

The 93-degree standard and offset calibration

Our reference point is a brew temperature of 93°C.

With offset: If the machine has adjustable temperature control (PID), we adjust the offset so that the machine delivers 93°C on average as accurately as possible in a stable state (shots 3-5).

Without offset: If the temperature is not adjustable, we measure the machine in its factory setting.

When is a machine considered "heated up"?

Manufacturer claims are often optimistic ("ready in 5 minutes"). We define the status "heated up" more strictly:

Example: If the machine settles at 92.5°C after one hour, it must deliver at least 91.5°C on the first shot after the heat-up time.

Special case: For light portafilters (plastic/silicone insert), we allow a deviation of 1.5 degrees, as they work thermally differently than our measurement portafilter.

We determine the real heat-up time in the cold-start procedure in 5-minute intervals (5, 10, 15, 20 min) until this criterion is met.

The two test protocols

The KM protocol – The everyday test

Simulates typical home use: After the heat-up time, we pull 5 double espressos at intervals of one minute each.

Procedure:

1. Insert Scace
2. Start shot: Extract for 27 seconds
3. Dwell time: Leave Scace inserted for 15 seconds
4. Remove
5. Pause: Leave it removed for 45 seconds
6. Rinse: 2-second flush
7. Repeat (total of 5 shots)

Goal: To check if the first shot is on target (cold-start performance) and whether the machine keeps the temperature constant over a series.

The WBC protocol – The stress test

According to the specifications of the World Barista Championship, the machine must perform under heavy load: 14 shots in short intervals plus parallel milk steaming. This shows if the machine has reserves for guests or catering use.

The WBC protocol is carried out after one hour of operation.

The 3-cluster scoring model (max. 10 points)

Cluster A: KM everyday suitability (max. 5.3 points)

A1. Cold-start behavior / First shot (max. 3.3 points)

Comparison: Average shot 1 vs. average shot 3-5 (steady-state)

• Difference < 0.5°C → 3.3 points
• Difference < 1.0°C → 2.7 points
• Difference < 1.5°C → 2.0 points
• Difference < 2.5°C → 0.7 points
• Difference > 2.5°C → 0.0 points

A2. Consistency & calibration (max. 2.0 points) – FAIRNESS UPDATE

Check the stability of shots 3, 4, and 5 among each other.

Scenario A (Stable): Delta of averages < 0.5°C?
→ 2.0 points (full points for technical precision, regardless of the absolute value)
→ Mandatory: Point out offset correction in the report

Scenario B (Unstable): Delta > 0.5°C?
→ Evaluation based on deviation from target (93°C)

• ±0.5°C → 2.0 points
• ±1.5°C → 1.3 points
• >1.5°C → 0.0 points

Cluster B: Intra-shot stability (max. 2.7 points)

What happens during the 25 seconds of extraction?

Time-to-temperature (TTT):
By second 10 at the latest, the target corridor (average ±2°C) must be reached.
If later ("late riser"): Automatically max. 0.7 points

Curve shape:

• Flatline: Horizontal (Δ < 0.5°C) → 2.7 points (Technical ideal)
• Drop: Slight decline (< 2.0°C) → 2.0 points (Sensory good – prevents bitterness)
• Rise: Slight increase (< 2.0°C) → 1.3 points (Sensory critical – aggressive extraction)
• Drift/Unstable: Change > 2.0°C → 0.7 points
• Chaos: → 0.0 points

Cluster C: WBC performance (max. 2.0 points)

Does the machine also last for the garden party?

We fire off 14 espressos in quick succession and steam milk in parallel.

Evaluation:

• Pro level: Window < 1°C (max-min) → 2.0 points
• Household level: Window < 3°C (max-min) → 0.7 points
• Failure: Window > 3°C → 0.0 points

Evaluation grid for heat-up time (evaluated separately as its own category)

The evaluation goes far beyond the first visual impression. We carry out a systematic inspection.

Categories (max. 10 points)

A. Surface quality

• Visual inspection of all housing parts (paintwork, polish, coating)
• Inspection for uniform gaps (< 1mm = very good, > 1.5mm = problematic)
• Tactile check of all edges for sharpness or burrs
• Accuracy of fit of attachments (drip tray, tank lid, cup rail)

B. Material & haptics

• Identification of the primary materials used (metal, plastic, wood)
• Evaluation of material thickness (e.g., thickness of sheet metal parts >1.5mm = high quality)
• Haptic check of controls (pressure point, firmness, play)

C. Construction & stability

• Evaluation of housing torsional rigidity
• Checking stability
• Quality of visible screw connections and mechanical links

D. Interior quality (if visible)

• Cleanliness of cable/hose routing as an indicator of production care

Note on longevity

We do not currently evaluate the effective longevity of an espresso machine. We simply cannot reliably do this for several machines. We therefore refer to reports from people who have bought an espresso machine. The sentiment on the internet gives a good overview of whether an espresso machine is prone to errors.

If we nevertheless hear about increased problems with test machines from our own use or from the experiences of our home barista course participants, we mention this in the context of our tests.

Noise level is a decisive comfort factor, especially in open-plan kitchen-living areas. We do not rely on subjective hearing, but measure the sound pressure level under standardized conditions.

Measurement scenario:

• Distance: 20 cm at brew group height
• Environment: Film studio (controlled acoustic conditions)
• The maximum value during an espresso shot under load (against blind basket or coffee puck at 9 bar) is measured

Evaluation grid (0-10 points)

Subjective correction: In extreme cases, the score can be corrected by max. 1 point if the type of noise is particularly annoying (high-frequency whistling, rattling) or particularly pleasant (sonorous, deep humming). This must be explicitly justified in the test report.

We do not evaluate the amount of plastic waste, but the quality of the tools. A precise tamper is more valuable than five poorly made filter baskets.

Evaluation system based on points (max. 10 points)

Basic equipment (mandatory elements)

• Double-spout portafilter present: +1 pt
• At least 2 baskets (different sizes): +1 pt
• Tamper included: +1 pt
• Blind basket for backflushing: +0.5 pt

Quality of core components

• Tamper precise (fit with <1mm play in the basket): +2 pts
• Tamper precise (fit with 1-2mm play): +1 pt
• Stainless steel portafilter: +0.5 pt
• Baskets free of burrs, high-quality stamping: +1.5 pts
• Baskets acceptable: +0.5 pt
• Portafilter high-quality (metal, good haptics): +1 pt

Additional accessories

• Single-spout portafilter: +0.5 pt
• Bottomless portafilter: +1 pt
• Milk pitcher (only if high-quality): +0.5 pt
• Cleaning agent/cleaning brush: +0.5 pt
• Water plug-in filter: +0.5 pt
• Cups: +0.5 pt
• Sensible extras (leveler, WDT tool, apron etc.): +0.5 pt per item (max. +1 pt total)

Penalty

• Tamper too small (>2mm play): -1 pt
• Burr formation on baskets: -0.5 pt
• Non-functional cheap accessories: -0.5 pt

Maximum: 10 points (if exceeded, it will be capped)

Evaluates the performance of the machine in continuous operation ("stress test"). Less relevant for most home baristas, but important if you regularly entertain guests.

Point distribution (max. 10 points)

A. Temperature stability under continuous load (0-4 points)

Result from the WBC protocol (Chapter 6):

• 4 pts: WBC protocol passed with window <1°C
• 3 pts: WBC protocol passed with window <3°C
• 2 pts: Slight instability, but still usable
• 1 pt: Significant drop in performance
• 0 pts: Failure under load

B. Steam mass potential (0-3 points)

Test: 5 milk steaming processes (300ml each) in a row at 90-second intervals

• 3 pts: All 5 rounds without loss of performance, constant steam power
• 2 pts: Slight drop in performance from round 4-5, but still usable
• 1 pt: Significant drop in performance from round 3
• 0 pts: Machine cannot manage 5 rounds

C. Capacities (0-3 points)

• +1.5 pts: Fixed water connection
• +1 pt: Drip tray with drain
• +0.5 pts: Large cup warming tray (>6 cups)

The Kaffeemacher energy score

Energy efficiency is no longer a niche topic in modern espresso machines, but a central quality characteristic. To create fair comparability between a small thermoblock and a large dual boiler, we have developed a differentiated evaluation system. Further explanations on the measurement system here. Our power protocol is now used by various espresso machine manufacturers in development.

Our measurement setup

Measurement device: Christ CLM1000 Professional (Plus)

• Resolution: 0.1 Wh (0.0001 kWh)
• Logging: Readings every second as a CSV file

Start conditions:

• Machine at rest for at least 6 hours
• Water tank filled to the maximum with room-temperature water (20-24°C)
• Cold start: Machine unplugged, then plugged in and switched on

The complete measurement protocol

We carry out a comprehensive measurement protocol for every machine to cover all usage scenarios:

The evaluation principle: "Espresso First"

Although we carry out the complete protocol, we evaluate the machine based on the typical everyday scenario:

The principle of best possible efficiency applies: If a machine (e.g., a dual boiler) has the function of switching off the energy-hungry steam boiler, we evaluate it in this economical "espresso mode". We want to reward technological progress and energy-saving functions.

Transparency: Additional readings in our tests

In our detailed test reports, we additionally show:

For classic heat exchangers or machines without a shut-off function for the steam boiler, the espresso value is identical to the cappuccino value (minus the actual steam pull) – which is reflected accordingly in a lower score.

Evaluation scale (1-10 points)

The scale is based on current technology standards and the value "heat up + 1 double espresso":

The fair value formula

The evaluation of the price-performance ratio (PPR) is not subjective, but based on a mathematical "fair value" model. This sets the objectively measured overall quality of the machine in relation to its current market price.

Why a formula?

A 3,000-euro machine doesn't have to be three times as good as one for 1,000 euros to have a good PPR. This is due to the economic law of diminishing marginal utility: every additional euro invested brings less additional quality. A perfect 10/10 machine for 5,000 euros can have a worse PPR than a solid 7/10 machine for 800 euros.

The calculation model

Step 1: Determine quality basis (0-100 scale)

We take the weighted average of all technical and haptic performance categories (Chapters 1-10, WITHOUT the PPR category itself) and scale it to 0-100:

Step 2: Calculate raw value (square root correction)

To avoid linearly penalizing expensive high-end machines, the price is "softened" via the square root:

Example:

• Machine A: Quality basis 80, price 1,600 EUR
  → Raw value = (80 / √1600) × 10 = (80 / 40) × 10 = 20.0
• Machine B: Quality basis 70, price 900 EUR
  → Raw value = (70 / √900) × 10 = (70 / 30) × 10 = 23.3

→ Machine B has a better PPR despite lower quality!

Step 3: Scaling to a 0-10 score (Fair Value 2025)

The raw value is converted into a score based on current market data:

The anchor (score 5.0): A raw value of 15.0 defines the current market standard for a fair price (e.g., solid mid-range dual boilers).

The gradient: A change in the raw value of 3.0 points leads to a change in the score of 1.0.

Formula:

Limit: Min. 0 points, Max. 10 points

Interpretation

• Score > 8 Price-buster. Above-average performance for the price
• Score 5 – 6.5 Fair value. The price is appropriate for the performance offered
• Score < 5 Enthusiast surcharge. You pay a disproportionately high premium for nuances, design, or brand

How to interpret our test results

1. The total score is just the beginning

A 7.5 means "very good" – but perhaps the machine is weak in categories unimportant to you and excellent in the important ones. Always look at the individual scores!

2. Only compare within similar price ranges

A machine for 800 EUR with a score of 7.5 is a different caliber than one for 2,500 EUR with a score of 7.5. Use the price-performance score for fair comparisons.

3. Identify your "must-haves"

Ask yourself:

• Do I drink mainly espresso or milk-based drinks? (→ espresso potential vs. steaming quality)
• Do I use the machine daily or on weekends? (→ heat-up time, usability)
• Is sustainability important to me? (→ power consumption)
• Do I have guests often? (→ volume potential)

4. Use the individual evaluations as a checklist

Each category has sub-points. If, for example, you know that smart profiling is important to you, check Chapter 1 (Espresso potential) directly to see if the machine offers it.

5. Wait for the individual weighting (coming soon)

Transparency and continuous improvement

This test protocol is a living document. We are continuously adjusting it based on your feedback and our experiences. Version 2.3 is already a significant development over our first tests.

What we do not yet test fully

The exact point distribution in the categories steaming quality and build quality will be added as soon as we have collected sufficient comparative data. However, the categories and evaluation criteria are already defined.

What you can do

• Give us feedback
• Write to us about which aspects are particularly important to you
• Share your experiences with tested machines

Our promise

We remain objective, measurable, and transparent. Every machine is tested according to the same protocol. Every machine is bought by us or, in individual cases, rented by us for a rental fee in the case of very expensive espresso machines.

We take our time. Good tests take time.

Concluding remarks

A good test protocol is like a good recipe: it standardizes the process but leaves room for nuances. We hope that with this guide, you not only understand our tests better but can also make a well-founded purchase decision.

In the end, only one thing counts: the espresso in your cup. All measurements, scores, and formulas are just a means to an end – namely to help you find the machine that suits your life, your demands, and your budget.

Good luck with your machine search!

Andrea Perin, Michel Indelicato, Benjamin Hohlmann