Measuring User Experience with the UEQ: a practical guide and case study in the hospitality sector

How a 26-item scientific questionnaire turns subjective user impressions into actionable design data

There is a question every digital product manager asks sooner or later, often in the wake of an expensive design overhaul or a disappointing launch: how do I know whether the experience I am delivering is actually good? The intuitive answer — "I ask users" — points in the right direction but falls short in practice. Gathering qualitative feedback, reading reviews and running unstructured usability sessions yields valuable insights, but insights that cannot be compared, aggregated or replicated over time.

To answer that question with rigour, UX research has developed a set of standardised measurement instruments. Among these, the User Experience Questionnaire (UEQ) stands out for a rare combination: brevity, scientific validity, and the ability to capture both the pragmatic and the hedonic dimensions of experience. This article describes how the instrument works, its theoretical grounding, and analyses it through a real-world study of mobile hotel-booking applications — a sector in which Esario works on a daily basis.

Measuring user experience: why qualitative feedback is not enough

Quantitative UX measurement addresses a concrete need: making evaluations collected at different times, on different products and from different user samples mutually comparable. Without a standardised metric, knowing that "users seem confused at checkout" says nothing about the degree of confusion, nor does it reveal whether a subsequent release improved or worsened the situation.

The landscape of standardised instruments is broad. The System Usability Scale (SUS) provides a concise, robust measure of overall perceived usability. AttrakDiff2 separates pragmatic and hedonic quality, with a focus on identity and stimulation. The PSSUQ/CSUQ measures end-of-session satisfaction. Each tool answers slightly different questions. As illustrated in Esario's evaluation service, the choice of instrument is never arbitrary: it depends on the construct to be measured, the user population and the phase of the product lifecycle.

The UEQ occupies a specific niche within this ecosystem: it simultaneously measures the six principal dimensions of UX — both task-oriented and non-task-oriented — through a single 26-item questionnaire that takes 3 to 5 minutes to complete. That is its competitive advantage.

How the UEQ is built: the scientific foundations

Origins and construction process

The UEQ was developed in 2005 by Bettina Laugwitz, Theo Held and Martin Schrepp (SAP AG) using a rigorously empirical approach. The process began with brainstorming sessions involving 15 usability experts, from which 229 potential terms related to user experience emerged. This pool was reduced to 80 items through expert evaluation, then further refined through studies on real products — statistics software, mobile phone address books, collaboration tools — with 153 participants in total. A principal component analysis (PCA) with varimax rotation ultimately extracted the six definitive scales.

The validity of the instrument was verified in 11 usability tests with 144 participants and in an online survey with 722 participants, achieving sufficiently high Cronbach's Alpha values across all scales. Subsequent construct validity studies confirmed the expected relationships between UEQ scales and those of the AttrakDiff2.

The six scales: structure and meaning

Each UEQ item takes the form of a semantic differential: two terms with opposite meanings separated by a 7-point scale (coded from –3 to +3). The positive/negative order of each term pair is randomised to reduce response biases.

The distinction between pragmatic quality (task-oriented) and hedonic quality (non-task-oriented) reflects Hassenzahl's theoretical framework, which distinguishes between what a product does and what a product is for the user. Attractiveness is a pure valence dimension: it does not describe the reasons for a judgement, but its overall affective direction.

Interpreting scale values

Mean scale values are interpreted according to empirically established thresholds: values above 0.8 indicate a positive evaluation; values between –0.8 and 0.8 represent a neutral zone; values below –0.8 signal a negative evaluation. In practice, real-world applications typically yield values in the range of –2 to +2.

Each scale is accompanied by a 95% confidence interval (the error bars in the chart produced by the Excel Tool), which describes the precision of the estimate: narrow bars indicate high agreement among respondents, wide bars signal heterogeneity or an insufficient sample. Statistical significance for product comparisons is assessed with a two-sample t-test, available in the dedicated Excel workbook.

The UEQ evaluation process: six operational phases

Case study: evaluating hotel-booking apps with the UEQ

Context and research objective

Kresnanto, Hanggara and Prakoso (2020) applied the UEQ methodology to compare two mobile applications for budget hotel booking in Indonesia: RedDoorz and Airy. Both apps operate in a highly competitive market segment — affordable hotels with digital booking — and had received strong store ratings (4.6 and 4.7 stars on Google Play respectively). Yet user reviews revealed dissatisfaction with specific aspects of the interface and the booking flow. The study set out to quantify that dissatisfaction with validated instruments.

The sample comprised users with heterogeneous backgrounds, aged 18 to 34, who had already used at least one of the two applications. Data were collected via an online UEQ questionnaire and subsequently analysed using the official Excel Tool.

Results: mean scores by scale

A critical reading of the results

A superficial reading of the data might conclude: Airy is better. A design-oriented reading, however, is considerably richer.

Novelty is the most discriminating scale. RedDoorz scores only 0.30 (neutral zone), Airy 0.88 (just above the positive threshold). The gap of 0.58 points is the widest in the entire comparison. Users perceive RedDoorz as conventional and lacking design innovation.

Perspicuity and Efficiency are almost identical across the two apps: this suggests that both applications handle the core functional task — finding and booking a hotel — competently, but with different levels of perceived appeal and innovation.

Stimulation is the only scale on which RedDoorz outperforms Airy (1.34 vs 1.33), albeit by a negligible margin. Both applications are perceived as motivating.

Dependability shows a meaningful gap (1.08 vs 1.29): RedDoorz users perceive less control and predictability in their interactions — a precise design signal pointing to issues with feedback, error handling and system-expectation alignment.

Against the general UEQ benchmark: RedDoorz rates "Good" on Perspicuity and Stimulation, "Above Average" on Attractiveness and Efficiency, "Below Average" on Dependability, and "Bad" on Novelty. Airy rates "Good" on four scales and "Above Average" on Dependability and Novelty.

Implications for redesign

UEQ data do not specify what to change in a product — they indicate where to look for it. In the case of RedDoorz, the pattern is clear: the interface works (high Perspicuity and Efficiency) but underperforms on the hedonic-innovation axis (low Novelty) and on the sense of control (Dependability in the critical zone). A redesign intervention should prioritise:

• The visual feedback system and error handling, to recover Dependability
• Updating the visual language and interaction patterns, to recover Novelty
• Reviewing the information architecture of the booking flows, to consolidate existing strengths

For Airy, conversely, the overall positive profile suggests maintaining design coherence while working at the margins — with particular attention to deepening long-term satisfaction, an aspect that the standard UEQ configuration does not capture.

How Esario integrates the UEQ into its evaluation process

Esario applies the UEQ as a component of a multi-level evaluation framework, not as a standalone instrument. Esario's Usability, UX and Accessibility Evaluation service combines standardised scales, cognitive and behavioural performance metrics, and emotional metrics in a multidimensional approach that delivers a complete picture of the experience.

Within this ecosystem, the UEQ plays a specific and well-defined role: it is the instrument of choice for simultaneously measuring pragmatic and hedonic quality, and is particularly well suited to the hospitality and e-commerce sectors, where enjoyment of use and the perception of innovation are as determinative of conversion as functional efficiency.

In a UX evaluation project for a client in the hotel sector — precisely the context of the study examined in this article — Esario typically works in a three-movement cycle: baseline evaluation with UEQ and usability testing, scale-pattern-driven redesign, and post-intervention re-evaluation to measure the delta. This approach makes the return on UX design investment as measurable as the ROI of an advertising campaign.

The experience accumulated over more than twenty years of consulting and research, combined with an academic background in mathematics and university-level UX teaching, enables Esario to select the most appropriate instruments for each context, interpret data with methodological rigour and translate results into concrete design guidance. Measurement is never an end in itself: it is the means through which knowledge becomes design.