The most damaging usability-testing mistakes happen before and after a participant touches the product: a vague research question produces unfocused tasks, a poor-fit sample distorts what the team sees, and weak analysis turns observations into overconfident conclusions. Avoid them by designing the study around a specific decision, recruiting people who reflect the intended users, letting participants attempt realistic tasks without being steered, and treating findings as evidence to investigate and retest—not as universal statistics.
1. Starting without a focused research question
“Test the app” is not a research question. It gives the team no clear basis for choosing tasks, deciding whom to recruit, or interpreting what happens. Before scheduling sessions, state the product decision the study should inform and the uncertainty that stands in its way.
- Too broad: “Find usability problems in the service.”
- More useful: “Can first-time customers find the delivery date before paying, and what makes them hesitate?”
Limit the study to a manageable set of related questions. Nielsen Norman Group cautions that adding goals can dilute insight on the others; Digital.gov also identifies an overly broad purpose as a study-design weakness. If the team has several unrelated questions, separate them into studies or prioritize the decision with the greatest immediate consequence.
2. Recruiting whoever is easiest to reach
A convenient group of colleagues, friends, or product experts can be useful for checking a prototype for obvious defects, but it is a poor stand-in for ordinary users. They may know the product, share the team’s assumptions, or be unusually motivated to help. Recruit actual or likely users based on the needs, behaviors, and goals relevant to the study.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errors#1 Best Overall
- Used Book in Good Condition
- Define the characteristics that matter to the research question rather than recruiting by demographics alone.
- Consider who recruitment channels, session times, location, language, and format may exclude.
- For accessibility research, plan time to recruit people with relevant access needs and offer suitable communication support.
- Avoid repeatedly relying on the same participants; familiarity with the product or test format can affect behavior.
Government Digital Service guidance on finding research participants discusses recruitment routes and accessibility support. Third-party recruitment or accessibility organizations can help reach people a team cannot readily recruit itself, but choose a route based on participant fit, geography, access coverage, privacy, and support needs—not convenience alone.
3. Treating “five users” as a universal sample-size rule
There is no single correct number for every usability study. The right sample depends on whether the team is exploring design problems or estimating performance, how many distinct user groups are in scope, and how much variation matters to the decision. These published recommendations describe different purposes and should not be blended into one rule.
| Study purpose | Published guidance | How to interpret it |
|---|---|---|
| Qualitative usability testing | 5 to 6 participants; Office for Health Improvement and Disparities, 2020 | A practical recommendation for qualitative discovery, not a guarantee that every issue or user group will be represented. |
| Traditional qualitative study | 5 participants; Nielsen Norman Group checklist, originally published about 2016 | Use to surface design issues; recruit more or run additional rounds when distinct groups or unresolved questions require it. |
| Quantitative work or eyetracking | At least 20–30 participants in each target user group may be needed; Nielsen Norman Group checklist, originally published about 2016 | A different scale and purpose from small iterative qualitative testing. |
| Usability benchmarking | 30 to 60 actual or likely users; Government Digital Service, 2018 | Guidance for benchmarking a website or service, not a requirement for every usability session. |
These figures come from practical guidance, not a universal sample-size law. The Office for Health Improvement and Disparities also describes an EPIC HIV example with 29 participants across 4 rounds; that case illustrates iterative refinement and contextual recruitment rather than a recommended total for other studies. Be explicit about the study’s purpose, target groups, and limits whenever reporting results.
4. Writing tasks that give away the answer
A task should describe a plausible goal in the participant’s language, without naming the control, route, or sequence the team expects them to use. A task that says “Click the Settings menu, then choose Billing” tests whether someone can follow instructions—not whether they can find billing information.
Recommended Free Tools
- Give one task at a time, using neutral and consistent wording.
- Describe a believable need or situation, not the interface element to use.
- Make the task challenging enough to expose friction but not so vague that participants cannot understand the goal.
- Pilot the wording with someone outside the study team to catch ambiguity and accidental clues.
For example, “You need to know when your order will arrive. Find that information” leaves the route open; “Open Order History and tap Track Package” does not. If participants misunderstand the scenario, revise it rather than coaching them toward the intended path.
5. Helping too much or asking leading questions
Participants can feel that they are being tested personally. At the start, reassure them that the service or prototype is being tested, not them, and explain that there are no right or wrong answers. During a task, allow time for them to try. Silence may feel awkward to the moderator, but intervening too soon can hide the very difficulty the study is meant to reveal.
The Office for Health Improvement and Disparities’ GOV.UK guidance, “Usability testing: qualitative studies” (2020), says: “Give the participant a task and then let them complete it. Try to resist influencing how they engage with the prototype or giving them too many instructions.”
- Neutral: “What were you expecting to happen?”
- Leading: “Did you notice the blue button that takes you to checkout?”
- Neutral: “Can you tell me what you’re thinking as you decide what to do next?”
- Leading: “Wouldn’t it be easier if this were at the top?”
Ask open-ended follow-ups about what you observed. Avoid praising a particular route or suggesting a solution, as either can nudge later behavior. A note-taker can record actions and context so the facilitator can focus on the session.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstall6. Choosing a format or setup that hides the real problem
Choose moderated or unmoderated, remote or in person, and lab or natural setting according to the question—not habit. Each choice changes what the team can observe and who can participate.
| Choice | Useful when | Trade-off to account for |
|---|---|---|
| Moderated | The team needs to clarify what happened or ask follow-up questions. | The moderator’s timing and wording can influence behavior. |
| Unmoderated | Faster participation or reaching people who are difficult to schedule matters. | There is less opportunity to clarify an unexpected action in the moment. |
| In person | Physical context or subtle interaction cues matter to the question. | The location and setup may differ from normal use. |
| Remote | Participants need access from their own setting or broader reach is useful. | It may be harder to guide participants or interpret interaction. |
| Natural setting | Environment, interruptions, or the participant’s usual setup materially affects use. | Conditions may vary between participants. |
| Lab setting | Consistent conditions are important to the task or comparison. | A lab may omit contextual barriers found in everyday use. |
Where relevant, let participants use their own devices and assistive technology. GOV.UK guidance notes that configured assistive tools can be difficult to reproduce in a lab. A generic test setup may therefore conceal a real barrier or create an artificial one.
7. Treating accessibility as an afterthought
Include people with relevant disabilities and assistive-technology use in the target group when those experiences matter to the service. Plan appropriate access arrangements and recruitment time instead of trying to add them at the last minute. Match the session to participants’ actual setups where possible.
One person’s experience is evidence about that person’s interaction, not a proxy for an entire disability group. Usability sessions can reveal barriers, but they do not replace evaluation against applicable accessibility standards. Use both kinds of evidence for their respective purposes.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →8. Overloading the session or measuring the wrong thing
Too many tasks can exhaust participants and leave too little time to observe meaningful behavior. For benchmarking, Government Digital Service guidance suggests no more than 5 tasks per participant and up to 10 minutes per task as a rule of thumb. These are benchmarking guidelines, not fixed limits for every exploratory session.
Choose measures that answer the study question. For a benchmark, task success and time can help compare performance; also note abandonment and cases where participants believe they succeeded when they did not. In qualitative work, use metrics to add context, not to imply population-wide precision from a small sample.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.9. Recording without consent or treating observation as proof
Explain whether a session will be recorded, why the recording is needed, and how participant information will be handled. Obtain informed consent before recording and protect personal information. A recording is not a substitute for a clear research question or careful analysis.
Rank #4
- Used Book in Good Condition
Real user data can make a task more contextual, but use it only when the service can handle it securely. Otherwise, create realistic dummy data. Bring together observed actions, participant comments, recordings, and relevant analytics carefully; describe what each can and cannot establish, and document limitations in the findings.
10. Failing to turn findings into changes
A study is useful when its evidence informs a decision. After sessions, identify repeated task failures, common errors, and the context in which they occurred. Share the findings with the people who can act on them, and translate challenges into design opportunities rather than treating a participant’s suggested fix as the answer.
Retest meaningful changes. For benchmarking comparisons, keep tasks and conditions consistent enough between rounds to interpret differences, while reviewing whether the service or user behavior has changed enough to require an updated study.
Documenting a web experience between sessions
A static capture can help a team annotate a particular page state or compare a rendered page during design review. It cannot show a participant’s path, pauses, confusion, or assistive-technology interaction, so it is supplementary documentation—not a usability test or a replacement for session evidence. ScreenshotNeo is a website screenshot API and MCP server for developers; use it only when a reproducible page capture is useful to the workflow.
Or skip the browser setup
One GET request can return a screenshot or PDF. For a page capture, replace the example target URL and provide your API key. See the ScreenshotNeo API documentation for request options.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpScreenshotNeo accepts cookie or consent banners as a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and responses identify the page verdict and billing status in headers. Its MCP server offers take_screenshot, get_page_info, and capture_pdf for AI agents and MCP clients. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. These captures can support page review, but they do not establish how users behave.
Sign up free for 1,000 screenshots a month, with no card required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
