-or-
Overcoming common obstacles preventing UX Research
In the last few years I’ve had several people come to me seeking advice about setting up easy-to-use, inexpensive, and effective usability testing kits. This includes an Amazon Research Team, a Facebook Research Manager, and a Principal User Experience Designer at REI. Why are these and others reaching out to me? Because they understand that research is vital to the success of their products and they don’t want technology, time, or money to prevent them from doing research.
These requests got me thinking about some common obstacles that prevent UX research from happening and what can be done to overcome them. Guided by the wisdom of one of the UX forefathers, Steve Krug, other seasoned UXers, and my experiences in UX tech and research I’d like to share three truths to show that you can overcome these obstacles and ensure UX research is not overlooked or dismissed.
You can easily create your own inexpensive and effective “UX Lab.”
Obstacle #1: Equipment is expensive and/or complicated.
During my time at Blink assisting with the technology side of UX research I was tasked with supporting numerous projects with needs beyond simple usability testing of a website. ‘Wizard of Oz’ testing of a smart kitchen appliance, testing of a app connected to a newly-designed gas-pump interface, testing the in-run experience of a redesigned running app, testing several iterations of components connected to the U by Moen smart shower, and a two-week diary study of a voice-activated speaker in participants’ homes are just a few of the more unique setups I’ve designed creative solutions for. While these projects have unique technical challenges, the majority of projects I’ve supported or led as a UX researcher are much simpler. The technical essentials are usually capturing a computer or phone screen, the participant’s face, and of course, audio from both the participant and the moderator.
I’ve utilized and I recommend two techniques for effectively capturing research sessions. With a few pieces of equipment and some training anyone can put these to use.
Technique 1 - Utilize Web Conferencing Software
Web conferencing software options are abundant and many of them are free or inexpensive. The tool that I recommend and have had great success with over the years is Zoom.
Utilizing Zoom to run remote moderated usability testing.
Zoom is designed to effectively capture whatever is needed during your research session. Zoom will automatically capture the participant’s face and audio and give you options for sharing either the participant's desktop or even mobile device. Additionally, the moderator is able to share their screen and give the participant control of the mouse. This is very effective when testing an HTML prototype or new designs you don’t want participants to have access to after the research session. Zoom also has the ability to automatically or manually record the meeting to your computer or in the cloud so, at the conclusion of your session, you have a PiP-configured MP4 file ready to be shared out or cut it highlight reels.
Getting stakeholders to watch sessions couldn’t be easier; just send them the Zoom meeting link and they can view from anywhere. The only downside is that there is the potential for your participant to feel like 10 people are watching them, especially if people are coming in and out of the meeting during the session. One way to limit this is to make an easy ‘Zoom Lab + Observation Room.’ Just do the following: the participant uses one computer in the 'lab' and one observer joins the meeting (muted w/ no video) and connects their computer to a large monitor, giving you a simple but effective 'observation room.'
Zoom’s many features, including all their screen sharing options make it a great, off-the-shelf research tool and the free version may be all you need.
Technique 2 - Stream Your Sessions to YouTube
If you’re feeling a bit more tech savvy and want more video/capture options then utilizing the free video production software OBS in conjunction with YouTube is another great way to capture and share research sessions.
Field kit for usability testing of a mobile app. Photo by Greg Hansen.
This technique takes advantage of free software as well as a free and very accessible location for live viewing and storing sessions. OBS is a free and open-source software designed for video recording and live streaming. For UX Research only the most basic features are needed but if you do have a study that has unique requirements it should be able to accommodate them.
OBS has designed live streaming to YouTube built right in and there are numerous resources out there to learn how to properly set everything up. Viewing sessions on YouTube is easy and secure; just ensure your live streams are 'Unlisted' so only those with the proper link have access, not matter where they are. This eliminates the 'many meeting participants' problem discussed earlier. The downsides to this technique are that there is more setup between sessions, some basic knowledge of setting things up in OBS (but I don't believe it's too complicated), and an investment in dedicated equipment.
You can find the ‘right’ participants provided you take an iterative approach.
Obstacle #2: Participants are hard to find and/or expensive.
In his book, Don’t Make Me Think, Steve Krug lays out several true things he knows about testing:
“Testing one user is 100 percent better than testing none.”
“Testing one user early in the project is better than testing 50 near the end.”
“The importance of recruiting representative users is overrated.”
“Testing is an iterative process.”
Some common perceptions around research or testing are that many participants are needed and that you have to find the exact right people to test with. This tends to make research into more of a high-stakes process when it doesn’t need to be. Krug’s philosophy is that testing should be early and often.
He believes that it’s good test with people similar to those who will use your product or service but more weight should be put on making testing an iterative process. The following diagram illustrates how testing twice with three participants in each test will identify more problems then doing just one test with eight participants. The key difference is that the problems identified during the first test are fixed before the second test.
Jakob Nielsen, of the Nielsen Norman Group, reiterates this philosophy in his article, “Why You Only Need to Test with 5 Users”. Through years of research and some sophisticated math, Nielson and another researcher found that five participants will provide approximately 85% of the insights your study will uncover. After the fifth participant, “you are wasting your time by observing the same findings repeatedly but not learning much new.”
When you follow the iterative testing process Krug and Nielson lay out research doesn’t need to be a high-budget, high-stakes thing. Multiple tests with a smaller number of participants will help make research a routine part of your design and development process and identify more insights for the same or even less cost.
With the right tools and equipment (discussed earlier), research costs should be lower and studies can be done anywhere; in your ‘Zoom Lab + Observation Room,’ with remote participants, or out in the field.
The iterative approach evangelized by Krug and Nielson should also reduce the anxiety of finding the ‘right’ research participants. Because testing is happening more frequently there should be less pressure compared to finding the perfect fifteen participants for a ‘big, all-or-nothing research study.’
When it comes to finding people to participate there are numerous creative ways to find them as well as industry tools specifically designed to supply you with research participants. I’ve had great success with Dscout, UserTesting, and Respondent, which are just some of the tools you can utilize.
With some creativity and persistence you can build a culture of informed decision making.
Obstacle #3: There isn’t enough time for research.
This may be the hardest obstacle to overcome and the one that doesn’t have clearly defined solutions. Creating a culture where research is prioritized is hard work but it can be done. I was fortunate to work on a team at Microsoft where design and development wanted to make decisions informed by research before too much time was devoted to design efforts or before elements went live.
Each organization is going to differ on how much they prioritize research and the techniques used to create more buy-in will differ from situation to situation but there are some tried-and-true methods worth noting.
Get People to Watch Research Sessions
While there may be some truth in the obstacle of time, I don’t believe people are lacking interest. So it is crucial you give them an opportunity to watch research happening. I’ve heard of and seen several tactics practiced that are effective.
Invite all your stakeholders to the research sessions. Set up calendar invites, make posters, send out all-company emails, personally invite them; just get them in the room. Steve Krug believes, “try to get [them] to at least drop by; they’ll often become fascinated and stay longer than they planned.”
Set up live viewings in common areas. Alaska Airlines broadcasted research sessions in one of their cafes and found it a great way to evangelize research efforts happening on their website and mobile app.
Make viewing interactive—this will keep observers engaged and can even help with analysis. Provide an observation worksheet or something like the Rainbow Spreadsheet to fill out or have them jot down observations related to key questions on sticky notes—perfect for affinity diagramming later. Their insights are valuable and can help with analysis and recommendations.
Again, with the right tools and equipment, your stakeholders and observers can watch live UX research via Zoom or even YouTube.
Once they’ve had the opportunity to observe participants experiencing frustration with an account verification code thought to be working perfectly or the joy of successfully using a voice command to activate an IOT device you’ll have them hooked.
Create Compelling/Interesting Research Deliverables
Research reports are often stereotyped as dry and boring but there’s no reason they have to be. The insights and findings uncovered in research can have profound impacts so do what you can to make engaging, interesting, and empathy-inducing deliverables.
I’m particularly fond of highlight reels. They are quickly viewed, can be easily shared, and can have a huge impact on those watching them. For a multi-part study focused on DIY soap makers I created this highlight reel of participants sharing how soap making has impacted their lives. The stories shared in this video helped to build excitement around the design and research work we were doing.
When it comes to effectively communicating your work don’t be afraid to build on the successes of others. There are numerous ways to learn from others who’ve gone before us. I’m fortunate to work with tremendous researchers and designers who are eager to share the lessons for creating compelling work they’ve learned along the way. Seek out UXers around you for advice and wisdom. I’m found that most are willing and excited to help others in the UX industry. I’ve also found the podcast Mixed Methods and their Slack group to be very helpful. Additionally, Medium is full of articles like this one, with great tips on how to ensure your research isn’t boring and people will read it.
Try Something New
When I was completing my User-Centered Design Graduate Certificate I had two instructors, Rebecca Destello and Justin Marx, who had previously worked together at a start-up where they developed a unique solution to their organization’s lack of research buy-in. The need to “quickly instill a culture of user validation,” as well as other needs, prompted them to develop a strategy they called “Witness Wednesdays” or “usability sprints.”
Rebecca and Justin define usability sprints as “a series of rapid-fire user studies, emphasizing team collaboration and organizational buy-in.” These sprints would take five days to run and were often executed as a four-week series. The basic outline for one week looked like this:
Monday
Researchers and designers collaborate on the week’s session guide and prototype.
Observers are reminded of their upcoming obligation on Wednesday.
Tuesday
Researcher finalize the session guide and ensure the lab/observation room are ready.
Designers make any necessary tweaks to ensure the prototype matches the session guide.
Wednesday
Researcher conducts five, one-hour sessions.
Observers record issues, quotes, ideas, surprises, etc. on sticky notes.
Team debriefs and does quick affinity diagramming after each session.
Thursday
Researchers, designers, and stakeholders discuss the results and ideate on top-line issues.
Researchers generate a brief email report of the findings for the whole organization (because everyone on the email was in the observation room, an email is all that is needed!).
Friday
Discussion and ideation continues, if necessary
Given time, researcher begin next week’s session guide, and designers begin iterations to the prototype.
While this system had its challenges it also produced numerous great results for their organization. According to Rebecca and Justin, the biggest outcome of “Witness Wednesdays” was that it broke “the stereotype that ‘research take too long!’” Check out their talk at Convey UX 2018 for more about the unique system they developed.
⬡ ⬡ ⬡
Pivotal to overcoming these obstacles and deploying these creative tactics is ensuring that your research can be observed. Whether it’s during a collaborative live-viewing or in a compelling highlight video it all starts with an effective UX lab. Don’t have a UX lab? That’s not a problem! You can see there are simple, elegant, and effective solutions out there ready to adapt to your needs and the creative solutions you develop to promote UX research.
What are the ways you’ve overcome these obstacle? Please share your UX tech tips, strategies, and creative solutions by leaving a comment below. Thanks!