Overcoming 5 Common Obstacles in User Research

Lessons on avoiding and overcoming some of the common trials and tribulations when running UX research

At Fuzzy Math, we practice a user-centered design process during which we learn about and advocate for the goals and needs of our users. The most effective way that we find out about user goals is by hearing about them from users directly, both at the beginning and the end of a design sprint.

First and foremost, we conduct user research sessions with users as part of the discovery phase. This takes place prior to beginning any design work. Contextual inquiry, or observing users working in their environment, is ideal for understanding what users do and why they do it, but we are often running remote interview sessions to gather this information.

When we reach a point in the process where we have a design that has been approved by the client, we validate this design, as well as future iterations of it, through a series of remote interviews where we share our screen with the user to show our designs and get feedback.

Through years of experience and hundreds of research sessions across all sorts of users, industries, and products, we’ve developed a rich understanding of what it means to run and participate in research. No two sessions are the same, and there are many obstacles that can come up throughout the research process.

Here are a few of the challenges that we’ve seen in the past, and the lessons we’ve learned from each:

Lesson 1: Scheduling is Often the Hardest Part

In most design research, talking to people is the fun part. You can plan out your session, craft detailed questions, and think through the “what ifs” for all sorts of scenarios. But what if you can’t get someone to talk to you? While there are “guerrilla research” techniques that can work in some cases to avoid scheduling issues, many of our projects focus on business software with a niche audience — we can’t simply set up a table at the coffee shop and expect to find people with the sort of detailed experience and knowledge we need. In most cases, we’re working in conjunction with our clients to source, and if needed, coax their customers to participate in research. Once people have been identified, it’s another challenge to get everyone to respond, agree on a time, and show up.

Scheduling requires planning, patience, and some creative workarounds.

• Talk with stakeholders to ensure everyone is on the same page regarding who’s handling recruitment and scheduling, and set appropriate expectations about how long it’ll take for everything to be completed. Assume you will have no-shows and cancellations, so recruit standbys if needed.
• Learn your time zones. If your users live in different countries, scheduling requires an extra layer of coordination. Ensure when scheduling that you’re explicit about time zones, and have some good coffee nearby for those early morning and late evening calls.
• Market yourself and the project to participants — if they feel like their voice is important, and they’re excited about the project, they’ll be much more likely to find some extra time in their schedule.
• If you’ve just had a great discussion with someone, ask if they know anyone else who you should talk to. A positive introduction from a friend or colleague is an invaluable recruiting opportunity, especially when dealing with people in very specific roles or industries.

Situation: You need to learn about a specific role within a certain industry, and are seeing a lot of non-responses.

• Rethink how you’re approaching participants. Review your outreach correspondence to ensure it’s simple and clear, and make sure you’re illustrating the value of participation to the user (whether monetary and/or a better experience based on their feedback).
• Reach out to the participants you’ve already talked to and see if they can put you in touch with colleagues.
• Take note of the scheduling slowdown, and account for it in any future research with similar participants. Start recruitment as far in advance as possible.

Lesson 2: Technical Difficulties Happen

We’re often using the phone and/or Webex (online screen-sharing meetings) to conduct remote research sessions with our participants. Once in a while, technical issues will come up: either side could have issues with audio, phone, or the Webex meeting connection they use to see or show their screen.

In times of technical difficulty, manage the down time by making something valuable happen. After all, you only have a certain amount of time with each person.

• Finding and scheduling users to participate in research can be challenging; it is often a collaborative effort between you and the client.
• Since users’ opinions matter most, take advantage of this time. Adjust with the technical difficulties to learn what you can from them.
• Plan ahead, and have a fallback ready. Ensure participants know if they’ll need to use any new software, including screensharing applications, before the session so they can set everything up in advance. If screensharing fails, have files prepared which can be emailed to the participant and talked through as needed.

Situation: You’re trying to validate a design with a user, and the person is unable to connect to see your screen.

• Have a conversation instead.
• Ask the participant questions around their process, role, communication, and day-to-day tasks.
• Come up with a scenario that applies to the application you’re working on. If possible, the scenario should be one that is addressed in the new design.
• Ask questions around what they do currently or would do in that scenario, and what they would expect to happen next.
• Discussing a scenario could give clues about whether the new design could work for the participant, even if they’re not able to see the design itself.

Lesson 3: Not Everyone is a Talker

With every batch of UX research, there’s always a toss-up regarding who you’re going to get. Some people talk freely with little prompting, while others need a tad more prompting before they are able to yield the information you’re looking for.

Guide and prompt those “difficult” participants to help them give you the information you need.

• For people who aren’t talking much, probe with open-ended questions and try to identify areas they’re interested in.
• Ask grounded questions about their experience, such as asking them to recall the last time they performed a certain action. By focusing on a recent, real example, they can more easily begin to tell a story and open up a dialogue.
• Through experience, start to recognize different communication styles people use so you can pick up on this as quickly as possible and adjust tactics as needed.

Situation: You’ve just started your session and have asked for first impressions on the first page of the design you’re showing. After you’ve given the person time to review and take in the page, the participant isn’t saying anything.

• Point out a particular area or feature of the page and ask them a question specifically about that area.
• The best tip for conducting user interviews is to try to keep your questions open-ended. This will help avoid one-word answers: Asking participants a question such as, “Do you think this would be useful?” might solicit a one-word answer; while asking them, “What would you find most useful here?” might help the participant give you more meaningful answers.
• Ask them about a common task they do in their current tool or workflow, and how or where they might go in the new design first to start the process.
• Take note of this personality type, and consider it a valuable data point. It could be representative of other users, too.