Testing and Rating: Our Protocol for Espresso Machine Tests

Why a Standardized Testing Protocol?

When buying an espresso machine, you face an overwhelming selection: From the compact thermoblock machine for 400 euros to the dual boiler monster for 4,000 euros – how do you compare? This is exactly where our testing protocol comes in.

We don't just want to tell you "This machine is great" or "That one is bad." Our goal is to provide an objective, measurable, and comparable evaluation based on hard facts. The result in the cup takes center stage, complemented by measurable technical parameters and a fair economic assessment.

All results are rated on a scale from 0 (insufficient) to 10 (outstanding) and then weighted together into an overall score.

Overview of all espresso machines we've tested.

How to Read This Protocol

1. Look at the overall score first – it gives a quick overview of overall quality
2. Identify your personal priorities – What matters to you? Espresso potential? Frothing performance? Energy efficiency?
3. Focus on the relevant individual scores – Ignore categories that are unimportant to you
4. Compare only machines in similar price ranges – A score of 7.5 at 800 euros can be better than 8.0 at 2,500 euros

The Weighting: Not All Criteria Are Equally Important

To reflect the varying relevance of aspects, categories flow into the overall rating with different factors:

Factor 3 (High – The Core)

Espresso Potential (3×): The most important criterion. How good is the espresso that can be maximally extracted from the machine? Here, technical prerequisites merge with sensory validation.

Factor 2 (Medium – Technology & Haptics)

These categories significantly influence the daily experience and technical performance:

• Temperature Management (2×): Consistency and adjustability
• Volumetrics (2×): Precision of automatic volume programming
• Frothing Quality (2×): Performance and speed when frothing
• Build Quality (2×): Materials, craftsmanship, and durability
• Usability (2×): Workflow, cleaning, and ergonomics
• Heat-up Time (2×): Time until fully ready for use
• Power Consumption (2×): Energy efficiency during operation

Factor 1 (Low – Additions & Special Categories)

These aspects are relevant but less decisive for most users:

• Noise Level (1×): Noise development during operation
• Accessories (1×): Quality and scope of what's included
• Volume Potential/Catering (1×): Performance with many consecutive shots
• Price-Performance Ratio (1×): The economic assessment

Calculating the Overall Score

The sum of factors is 19. The result is a grade between 0 and 10.

Quality Classes

Depending on the overall score, we assign the machine to a quality class:

The heart of our test

Why This Category Is Most Important

In the end, only one thing counts: How does the espresso taste? A machine can be technically impressive, but if the result in the cup disappoints, everything else is secondary. That's why espresso potential contributes with Factor 3 to the overall rating – more than any other category.

Our Hybrid Approach: Technology + Sensory

Espresso potential consists of two pillars:

• 1. Technical Prerequisites (max. 8 points) – objectively measurable
• 2. Sensory Validation (max. 2 points) – panel test

The Technical Prerequisites (8 Points)

A. Temperature Performance (0-3 Points)

The score from Chapter 6 (Temperature Management) is used as a basis here:

• Score ≥ 9.0 in Ch. 6 → 3.0 points
• Score 8.0-8.9 → 2.5 points
• Score 7.0-7.9 → 2.0 points
• Score 6.0-6.9 → 1.5 points
• Score 5.0-5.9 → 1.0 points
• Score < 5.0 → 0.5 points

Temperature stability is the foundation for good espresso. A machine with excellent temperature management already has a major advantage.

B. Profiling & Brew Control (0-3.5 Points)

Manual Control (0-3 points):

• 3.0 pts: Flow + pressure control: everything possible. Formulation of if-conditions, stages, volumetric control
• 2.5 pts: Programmable pressure profile control
• 2.0 pts: Pre-infusion adjustable (time/pressure) OR simple pressure profiles
• 1.5 pts: Fixed (sensible) pre-infusion present
• 1.0 pts: Manual profiling via paddle/lever possible
• 0 pts: No profiling options

Smart Profiling Bonus (+0.5 point):

Some modern machines have adaptive systems that monitor flow during extraction and automatically intervene. If the machine detects, for example, a too-fast flow-through (indicator for too coarse grind), it reduces pressure or extends pre-infusion automatically. This significantly increases fault tolerance and enables even less experienced users to achieve consistently high-quality results.

Maximum: 3.5 points

C. Hardware Hygiene (0-2 Points)

Shower Screen Quality & Water Distribution (0-1 point):

• 1.0 pts: High-quality shower screen, high-quality baskets, even distribution
• 0.5 pts: Solid standard shower screen, medium-quality baskets
• 0 pts: Deficient distribution, channeling susceptibility

Thermal Mass & Pressure Build-up (0-1 point):

• 1.0 pts: Heated metal portafilter, stable pressure build-up
• 0.5 pts: Partial thermal mass or slight pressure fluctuations
• 0 pts: Cold portafilter or unstable pressure

The Sensory Validation (2 Points)

The technical values are the foundation – but can you taste it?

Panel test by Q-Grader Benjamin Hohlmann with reference coffees whose ideal profile is known.

Rating:

• +2 points: Machine fully realizes technical potential. Espresso shows balanced balance, no extraction errors
• +1 point: Technical potential largely realized, minimal deviations
• 0 points: Clear discrepancy between technology and cup result

We evaluate the machine's ability to produce fine-pored, homogeneous microfoam for latte art. This assessment is made by our barista professional Michel Indelicato, who has frothed in test scenarios with around 100 different machine types.

The Categories (max. 10 points)

A. Steam Power

Pressure and ability to set a 300ml milk quantity in a 0.6l pitcher into rotation. The more tilted a pitcher must be held, the less powerful the machine.

• Pitcher vertical (0°) → Strong power
• Slightly tilted → Good power
• Strongly tilted → Weak power
• No rotation → Insufficient

B. Speed

Time required to heat 300ml of milk from 8°C (refrigerator temperature) to 60°C. In this scenario, the milk is deliberately not frothed but only heated (steam wand inserted deep).

C. Workflow & Ergonomics

• No-burn steam wand or insulated design
• Ball joint (360° freely movable)
• Rotary valve (precise steam volume control) vs. simple toggle switch

D. Condensation Water

How much condensation water or residual moisture is expelled by the machine before frothing can begin? Too much condensation water dilutes the milk.

The robustness test

We don't just test whether volumetrics are present, but how robustly they work under changing conditions.

The 5-Stage Stress Test Procedure

Setup:

• Coffee: Apas Espresso (medium roast)
• Grinder: Mahlkönig E64 GbW
• Brew ratio: 1:2.5
• Target size: Dependent on basket size, remains constantly set

The 5 stages:

1. Hardware baseline (5×): Scace portafilter (simulated resistance) – testing pure technology without coffee puck influence
2. Reference espresso (5×): Real conditions with standard grind setting
3. Profile check (5×): Test with complex profiles (e.g., pre-infusion) to check if logic remains stable with pressure fluctuations
4. Stress test "Coarser" (3×): Grind setting adjusted one full number coarser (faster flow)
5. Stress test "Finer" (3×): Grind setting adjusted one full number finer (slower flow)

Goal: Does the machine detect the changed flow (flowmeter) or does it stubbornly stop by time?

Rating Grid (0-10 Points)

Base points (technology type):

• 0 pts: Manual (no volumetrics)
• 3 pts: Pure time control

Precision points (based on average deviation):

• 4 pts: ±4g deviation
• 6 pts: ±2g deviation
• 8 pts: ±0.5g deviation (scale level)

Bonus (+2 points max):

• +1 pt: Exact UI input (grams/milliliters instead of pulses)
• +1 pt: Integrated scale

With scale coupling, we run the protocol twice: once without, once with scale.

This category evaluates how pleasant the machine is in daily use. A technically brilliant machine that annoys you daily loses points here.

Point Distribution (max. 10 points)

• Interface & control (max. 3 points + 0.5 bonus)
• Daily routine (max. 3 points)
• Cleaning & maintenance (max. 3 points)
• Documentation (max. 1 point)

A. Interface & Control (0-3.5 Points)

For machines WITH display/software:

• 3.0 pts: Intuitive, no manual needed, fast response time (<0.5s), easy to read
• 2.5 pts: Good, short familiarization, display still readable in sunlight
• 2.0 pts: Solid, occasionally manual needed, acceptable readability
• 1.5 pts: Cumbersome menu navigation, frequent consultation needed
• 1.0 pts: Confusing, sluggish response, poor readability
• 0.5 pts: Frustrating to operate

For machines WITHOUT display (mechanical/analog):

• 3.0 pts: High-quality controls, clear pressure point, precise rotary knobs
• 2.0 pts: Solid haptics, functional
• 1.0 pts: Wobbly feel, imprecise adjustment
• 0 pts: Cheap plastic, significant play

Bonus: +0.5 pts for programmable shortcuts/function keys

B. Daily Routine (0-3 Points)

Each fulfilled criterion gives 0.5 points (max. 3):

• ✓ Water tank easily removable/fillable without jamming
• ✓ Water level always clearly visible or last shot technology
• ✓ Drip tray sufficiently dimensioned (>500ml) and easy to remove
• ✓ Portafilter smooth-running, secure locking without force
• ✓ Integrated water filter in tank present
• ✓ Ergonomic arrangement of all daily-used elements

C. Cleaning & Maintenance (0-3 Points)

• 3.0 pts: Automatic cleaning programs present AND shower screen easily removable with spoon or included tool
• 2.5 pts: Automatic programs present, disassembly partially tool-free, tool not included
• 2.0 pts: Manual cleaning simple, clear machine hints on maintenance cycles
• 1.5 pts: Manual cleaning possible, partially fiddly
• 1.0 pts: Cumbersome disassembly, unclear maintenance hints
• 0.5 pts: Hard-to-access parts, no maintenance hints

D. Documentation (0-1 Point)

• 1.0 pts: Excellent manual (clearly structured, good illustrations, complete)
• 0.5 pts: Acceptable manual with minor gaps
• 0 pts: Deficient or incomprehensible manual

Penalty: -1 point for significant "annoyance factors" (excessively loud signal tones, unnecessarily complex processes, long inexplicable wait times)

The technical foundation

Why Temperature Is So Important

Brewing temperature is one of the most critical parameters for espresso. Just 2°C difference can drastically change the taste: too cold = sour and under-extracted, too hot = bitter and burnt. We don't rely on manufacturer specifications but measure ourselves – under standardized conditions.

Our Measurement Setup

Temperature measurement:

• Device: Scace 2 with data logger
• Calibration: Before each test series with boiling water
• Measurement: Directly in the portafilter

Power measurement:

• Device: Christ CLM1000 Professional (Plus)
• Resolution: 0.1 Wh (0.0001 kWh), per-second readings
• Logging: Saved as CSV file

Ambient conditions:

• Water temperature in tank: 20°C (±2°C)
• Target temperature: 93°C

The 93-Degree Standard and Offset Calibration

Our reference value is a brewing temperature of 93°C.

With offset: If the machine has adjustable temperature control (PID), we adjust the offset so that the machine delivers an average of exactly 93°C in stable state (shots 3-5).

Without offset: If temperature is not adjustable, we measure the machine in its factory setting.

When Is a Machine Considered "Heated Up"?

Manufacturer specifications are often optimistic ("Ready after 5 minutes"). We define the status "heated up" more strictly:

Example: If the machine settles at 92.5°C after one hour, it must deliver at least 91.5°C on the first shot after heat-up time.

Special case: For light portafilters (plastic/silicone insert), we allow a 1.5-degree deviation, as they work thermally different from our measuring portafilter.

We determine the real heat-up time in the cold-start procedure in 5-minute intervals (5, 10, 15, 20 min.) until this criterion is met.

The Two Test Protocols

The KM Protocol – The Daily Test

Simulates typical home use: After heat-up time, we pull 5 double espressos at one-minute intervals.

Procedure:

1. Insert Scace
2. Start shot: extract for 27 seconds
3. Standing time: leave Scace locked in for 15 seconds
4. Unlock
5. Break: leave unlocked for 45 seconds
6. Flush: 2 seconds flush
7. Repeat (total 5 shots)

Goal: Check if the first shot is on target (cold start performance) and if the machine holds temperature consistently over a series.

The WBC Protocol – The Stress Test

According to World Barista Championship specifications, the machine must perform under rapid fire: 14 shots at short intervals plus parallel milk frothing. This shows whether the machine has reserves for guests or gastro use.

The WBC protocol is conducted after one hour of runtime.

The 3-Cluster Scoring Model (max. 10 Points)

Cluster A: KM Everyday Suitability (max. 5.3 points)

A1. Cold Start Behavior / First Shot (max. 3.3 points)

Comparison: Average shot 1 vs. average shots 3-5 (steady state)

• Difference < 0.5°C → 3.3 points
• Difference < 1.0°C → 2.7 points
• Difference < 1.5°C → 2.0 points
• Difference < 2.5°C → 0.7 points
• Difference > 2.5°C → 0.0 points

A2. Consistency & Calibration (max. 2.0 points) – FAIRNESS UPDATE

Testing stability of shots 3, 4, and 5 among themselves.

Scenario A (Stable): Delta of averages < 0.5°C?
→ 2.0 points (Full points for technical precision, regardless of absolute value)
→ Requirement: Report must mention offset correction

Scenario B (Unstable): Delta > 0.5°C?
→ Rating by deviation from target (93°C)

• ±0.5°C → 2.0 points
• ±1.5°C → 1.3 points
• >1.5°C → 0.0 points

Cluster B: Intra-Shot Stability (max. 2.7 points)

What happens during the 25 seconds of extraction?

Time-to-Temperature (TTT):
By second 10 at the latest, the target corridor (average ±2°C) must be reached.
If later ("late riser"): Automatically max. 0.7 points

Curve shape:

• Flatline: Horizontal (Δ < 0.5°C) → 2.7 points (Technical ideal)
• Drop: Slight decline (< 2.0°C) → 2.0 points (Sensorially good – prevents bitterness)
• Rise: Slight increase (< 2.0°C) → 1.3 points (Sensorially critical – aggressive extraction)
• Drift/Unstable: Change > 2.0°C → 0.7 points
• Chaos: → 0.0 points

Cluster C: WBC Performance (max. 2.0 points)

Does the machine hold up even at the garden party?

We fire off 14 espressos in quick succession and froth milk in parallel.

Rating:

• Professional level: Window < 1°C (max-min) → 2.0 points
• Home level: Window < 3°C (max-min) → 0.7 points
• Failure: Window > 3°C → 0.0 points

Rating Grid Heat-up Time (evaluated separately as its own category)

The evaluation goes far beyond the first visual impression. We conduct a systematic inspection.

The Categories (max. 10 points)

A. Surface Quality

• Visual inspection of all housing parts (painting, polishing, coating)
• Check for even gap dimensions (< 1mm = very good, > 1.5mm = problematic)
• Touch test of all edges for sharpness or burrs
• Fit accuracy of attachments (drip tray, tank lid, cup rail)

B. Material & Haptics

• Identification of primarily used materials (metal, plastic, wood)
• Evaluation of material thickness (e.g., sheet metal thickness >1.5mm = high-quality)
• Haptic testing of controls (pressure point, solidity, play)

C. Construction & Stability

• Assessment of housing torsional rigidity
• Check of stability
• Quality of visible screws and mechanical connections

D. Interior Quality (if visible)

• Cleanliness of cable/hose routing as indicator of manufacturing care

Note on Longevity

We currently do not evaluate the actual longevity of an espresso machine. We simply cannot do this seriously for multiple machines. We therefore refer to reports from people who have purchased an espresso machine. The mood picture on the internet gives a good overview of whether an espresso machine is prone to defects.

If we nevertheless hear from our own use or through experiences from our home barista course participants about increased problems with test machines, we mention this in the context of our tests.

Noise level is a decisive comfort factor, especially in open-plan kitchens. We don't rely on subjective hearing but measure the sound pressure level under standardized conditions.

Measurement scenario:

• Distance: 20 cm at brew group height
• Environment: Film studio (controlled acoustic conditions)
• Measured is the maximum value during an espresso shot under load (against blind basket or coffee puck at 9 bar)

Rating Grid (0-10 Points)

Subjective correction: In extreme cases, the score can be corrected by max. 1 point if the type of noise is particularly disturbing (high-frequency whistling, rattling) or particularly pleasant (sonorous, deep humming). This must be explicitly justified in the test report.

We don't evaluate the amount of plastic waste, but the quality of the tools. A precisely fitting tamper is more valuable than five poorly crafted baskets.

Scoring System by Point Collection (max. 10 points)

Basic Equipment (Required Elements)

• 2-spout portafilter present: +1 pt
• At least 2 baskets (different sizes): +1 pt
• Tamper included: +1 pt
• Blind basket for backflushing: +0.5 pt

Quality of Core Components

• Tamper precisely fitting (<1mm play in basket): +2 pts
• Tamper precisely fitting (1-2mm play): +1 pt
• Portafilter made of stainless steel: +0.5 pt
• Baskets burr-free, high-quality punching: +1.5 pts
• Baskets acceptable: +0.5 pt
• Portafilter high-quality (metal, good haptics): +1 pt

Additional Accessories

• 1-spout portafilter: +0.5 pt
• Bottomless portafilter: +1 pt
• Milk pitcher (only if high-quality): +0.5 pt
• Cleaning agent/cleaning brush: +0.5 pt
• Water filter insert: +0.5 pt
• Cups: +0.5 pt
• Useful extras (leveler, WDT tool, apron, etc.): +0.5 pt per item (max. +1 pt total)

Penalties

• Tamper too small (>2mm play): -1 pt
• Burr formation on baskets: -0.5 pt
• Non-functional cheap accessories: -0.5 pt

Maximum: 10 points (capped if exceeded)

Evaluates the machine's performance in continuous operation ("stress test"). Less relevant for most home baristas, but important if you regularly entertain guests.

Point Distribution (max. 10 points)

A. Temperature Stability Under Continuous Load (0-4 points)

Result from WBC protocol (Chapter 6):

• 4 pts: WBC protocol passed with window <1°C
• 3 pts: WBC protocol passed with window <3°C
• 2 pts: Slight instability but still usable
• 1 pt: Clear performance drop
• 0 pts: Failure under load

B. Steam Volume Potential (0-3 points)

Test: 5 milk frothing sessions (300ml each) consecutively at 90-second intervals

• 3 pts: All 5 rounds without performance loss, constant steam power
• 2 pts: Slight performance drop from round 4-5 but still usable
• 1 pt: Clear performance drop from round 3
• 0 pts: Machine doesn't manage 5 rounds

C. Capacities (0-3 points)

• +1.5 pts: Fixed water connection
• +1 pt: Drip tray with drain
• +0.5 pt: Large cup shelf with warming function (>6 cups)

The Kaffeemacher Energy Score

Energy efficiency is no longer a niche topic for modern espresso machines, but a central quality feature. To create fair comparability between a small thermoblock and a large dual boiler, we've developed a differentiated rating system. Further explanations of the measurement system also here. Our power protocol is now used by various espresso machine manufacturers in development.

Our Measurement Setup

Measuring device: Christ CLM1000 Professional (Plus)

• Resolution: 0.1 Wh (0.0001 kWh)
• Logging: Per-second readings as CSV file

Starting conditions:

• Machine at rest for at least 6 hours
• Water tank maximally filled with room temperature water (20-24°C)
• Cold start: Machine unplugged, then plugged in and turned on

The Complete Measurement Protocol

We conduct a comprehensive measurement protocol for each machine to cover all usage scenarios:

The Evaluation Principle: "Espresso First"

Although we conduct the complete protocol, we evaluate the machine based on the typical everyday scenario:

The principle of best possible efficiency applies: If a machine (e.g., a dual boiler) has the function to turn off the energy-consuming steam boiler, we evaluate it in this economical "espresso mode." We want to reward technological progress and energy-saving functions.

Transparency: Additional Measurement Values in Our Tests

In our detailed test reports, we additionally show:

For classic heat exchangers or machines without steam boiler shut-off function, the espresso value is identical to the cappuccino value (minus the pure steam shot) – which is reflected accordingly in a lower score.

Rating Scale (1-10 points)

The scale is based on current technology standards and on the value "Heat-up + 1 double espresso":

The Fair Value Formula

The evaluation of price-performance ratio (PPR) is not subjective, but based on a mathematical "Fair Value" model. This puts the objectively measured overall quality of the machine in relation to its current market price.

Why a Formula?

A 3,000-euro machine doesn't have to be three times as good as one for 1,000 euros to have a good PPR. This is due to the economic law of diminishing marginal utility: Each additional euro invested brings less additional quality. A perfect 10/10 machine for 5,000 euros can have a worse PPR than a solid 7/10 machine for 800 euros.

The Calculation Model

Step 1: Determine Quality Base (0-100 scale)

We take the weighted average of all technical and haptic performance categories (Chapters 1-10, WITHOUT the PPR category itself) and scale it to 0-100:

Step 2: Calculate Raw Value (Square Root Correction)

To not linearly penalize expensive high-end machines, price is "defused" via the square root:

Example:

• Machine A: Quality base 80, price 1,600 EUR
  → Raw value = (80 / √1600) × 10 = (80 / 40) × 10 = 20.0
• Machine B: Quality base 70, price 900 EUR
  → Raw value = (70 / √900) × 10 = (70 / 30) × 10 = 23.3

→ Machine B has better PPR despite lower quality!

Step 3: Scaling to 0-10 Score (Fair Value 2025)

The raw value is converted into a score based on current market data:

The anchor (score 5.0): A raw value of 15.0 defines the current market standard for a fair price (e.g., solid mid-range dual boiler).

The slope: A change in raw value of 3.0 points leads to a change in score of 1.0.

Formula:

Limit: Min. 0 points, Max. 10 points

Interpretation

• Score > 8 Price killer. Above-average performance for the price
• Score 5 – 6.5 Fair value. Price is appropriate for performance offered
• Score < 5 Enthusiast premium. You pay a disproportionate premium for nuances, design, or brand

How to Interpret Our Test Results

1. The Overall Score Is Just the Beginning

A 7.5 means "Very Good" – but maybe the machine is weak in categories unimportant to you and excellent in the important ones. Always look at individual scores!

2. Compare Only Within Similar Price Ranges

A machine for 800 EUR with score 7.5 is a different caliber than one for 2,500 EUR with score 7.5. Use the price-performance score for fair comparisons.

3. Identify Your "Must-Haves"

Ask yourself:

• Do I mainly drink espresso or milk beverages? (→ Espresso potential vs. frothing quality)
• Do I use the machine daily or on weekends? (→ Heat-up time, usability)
• Is sustainability important to me? (→ Power consumption)
• Do I often have guests? (→ Volume potential)

4. Use Individual Ratings as a Checklist

Each category has sub-points. If you know, for example, that smart profiling is important to you, look directly in Chapter 1 (Espresso Potential) to see if the machine offers it.

5. Wait for Individual Weighting (Coming Soon)

Transparency and Continuous Improvement

This testing protocol is a living document. We adapt it based on your feedback and our experiences continuously. Version 2.3 is already a significant evolution from our first tests.

What We Don't Fully Test Yet

The exact point distribution in the categories Frothing Quality and Build Quality will be provided once we've collected enough comparison data. However, the categories and evaluation criteria are already defined.

What You Can Do

• Give us feedback
• Write us which aspects are particularly important to you
• Share your experiences with tested machines

Our Promise

We remain objective, measurable, and transparent. Every machine is tested according to the same protocol. Every machine is purchased by us or, in individual cases with very expensive espresso machines, rented by us with a rental fee.

We take our time. Good tests take time.

Closing Words

A good testing protocol is like a good recipe: It standardizes the process but leaves room for nuances. We hope that with this guide you not only understand our tests better, but can also make your own informed purchase decision.

In the end, only one thing counts: The espresso in your cup. All measurements, scores, and formulas are just means to an end – namely to help you find the machine that fits your life, your demands, and your budget.

Good luck with your machine search!

Andrea Perin, Michel Indelicato, Benjamin Hohlmann