Evaluate an experience

Evaluation is not asking whether a mockup is pretty. It reduces uncertainty: can a person find information, understand the state, finish a task and recover from an error?

Each method has a question and a limit. Use only enough fidelity to answer the question; do not make a prototype more impressive than necessary.

Choose the method

Question	Useful method	Evidence
Do categories match expectations?	Card sort or tree test	Groups, words and destinations
Is the flow understandable?	Prototype and task test	Actions, hesitation, errors and strategies
Does a principle fail?	Heuristic review	Located issue, impact and fix to test
Does behaviour change in production?	Instrumentation and analysis	Rates, volume, time, segments and trend
Why does a signal appear?	Interview or observation	Context, language, motivation and constraints

Do not ask a metric to answer a qualitative question. Do not ask five interviews to represent an entire population.

Write a testable hypothesis

A useful hypothesis contains an intervention, a situation and an expected effect:

When a person returns after an interruption,
if we show the last room, its status and “Resume”,
then they will resume without help more often.
We will observe the first click, status explanation and navigation errors.

“Users will like the new version” describes neither behaviour nor a decision criterion.

Prepare a usability test

Write the question, recruit people who meet the situation, prepare an outcome-based task, explain that you test the product rather than the person, request recording consent when needed and define what you will observe before the session.

During the session, ask “What are you trying to do?” and let the person try. Note first look, clicks, pauses, backtracking, help requests, errors and words. Separate:

• fact: the person opened the menu and went back;
• quote: “I thought this link was about tests”;
• interpretation: the label may not match the mental model;
• decision: compare a new label in another test.

Heuristic review

Review visibility of system status, match with the person’s language, control and undo, consistency, error prevention, recognition over recall, flexibility, sober hierarchy, recoverable errors and help at the moment of need.

For each issue, record scenario, principle, impact, severity, fix and verification method. A list of violations without context does not help prioritise.

Define a metric correctly

A metric needs a name describing behaviour, numerator, denominator, population or segment, period, data source and a possible decision.

Unaided resume rate
= correctly resumed sessions / eligible interrupted sessions
Population: new learners on mobile
Period: seven days after version 2
Decision: if it does not improve, observe the resume label and state

A high click rate may mean interest, confusion or a necessary control. A short time may mean efficiency or abandonment. Connect measurement to the task outcome and investigate contradictory signals.

Prioritise without false precision

Combine severity, observed reach, confidence in the evidence, effort and reversibility, and risk to accessibility, security or trust. Do not turn these dimensions into a magic score. A rare issue blocking assistive technology can deserve more priority than a frequent but harmless visual detail.

Experiments and comparison

A comparison is interpretable only when you know what changed, for whom and for how long. Do not change traffic, device, content and instruction at the same time. Preserve a rollback and watch side effects on errors, abandonment, accessibility and trust. A conversion increase does not justify an unclear or irreversible choice.

FlashLearning workshop

The new “resume” block increases clicks by 20%, but navigation errors also rise. Write the facts to check, a competing hypothesis, a usability task, two metrics with denominators and the decision if signals remain contradictory.

One possible answer

Check segment, period, eligible sessions, first click and actual destination. A competing hypothesis is that the block is more visible but its label sends people to an old session. Measure correct resume rate and navigation-error rate, both over eligible interrupted sessions. If clicks rise but correct destination does not, do not declare success: check data source and label, then retest.

Evaluation report

Question: …
Hypothesis: …
Method and limits: …
Task: …
Observed facts: …
Separate interpretations: …
Metrics and denominators: …
Decision: …
Next verification: …

Checklist

• The method answers a precise question.
• The task does not give away the expected path.
• Participants and constraints are relevant.
• Facts, quotes and interpretations are separate.
• Limits and bias are written down.
• Metrics have numerator, denominator, population and period.
• Every signal leads to a decision or next question.
• Accessibility, trust, security and privacy side effects are monitored.
• The next version and rollback are identifiable.

Reliable sources

• GOV.UK Service Manual — user research
• Nielsen Norman Group — usability testing 101
• Nielsen Norman Group — 10 usability heuristics
• W3C WAI — planning and managing accessibility

See how evaluation fits the product in the design lifecycle, then practise in the interactive UI/UX path.