CURIOSITY: A CORE VALUE

“Tell me more…” One of my UX research mentors is the master of this phrase. I observed numerous sessions where these three simple words were expertly used to seek more from a research participant. His goal was to gain a better understanding of whatever the participant had just said and ultimately, uncover the answers to the research questions he was tasked with finding.

This act of digging deeper to uncover more or learn is one of my favorite parts of research. It is also another core value of my UX career; CURIOSITY.

Last week I connected with a college student who recently began her own pursuit of a UX career. She reached to me “to get a more authentic understanding of this field,” and for some advice on seeking a design or research internship.

One question she asked reminded me of the importance of curiosity in UX research. “What have you found to be the most crucial questions to ask participants in usability testing, and have any results been surprising? (In that maybe some participants have experienced/noticed something that the designers/researchers didn't think the same way about.)”

Here’s how I answered her questions:

The questions asked during testing are so dependent on the product, project goals, and research questions that there is no one or two crucial questions that should be asked. That said, this is one crucial thing you should do in all your research sessions: when something a participant says (or does) that seems interesting or is somewhat unclear, say, "Tell me more." UX research is about finding the "why." You might ask a participant, "On a scale from 1 to 7, how easy or difficult was it to find the information you wanted? (Where 1 is very difficult and 7 is very easy.)" but if you don't ask them "why" you're only getting half the answer.

For me, the “why” is what drew me to research and I love getting to learn more about people's understanding, expectations, habits, thoughts, actions, interpretations, frustrations, ideas, etc. Knowing this enables informed design decisions, resulting in products and services that better serve their users.

In addition to understanding the "why," research should uncover those things that end users experience or notice that the designer or researcher didn't anticipate. The designer isn't always going to be a power-user of a product or they may not use it the same way as others. (This is where observation and empathy are crucial.) Research helps to guide initial designs and should be used again and again in an iterative process to point out issues and opportunities which can help create the best products possible.

---

During research sessions there are often many moving parts, lots to track, and key questions to cover so, in midst of balancing all that, it never hurts to have friendly reminder of staying curious. In our remote research labs (labs dedicated to sessions with remote participants) I’ve placed this inspirational “artwork” to help keep “Tell me more…” top of mind.

A friendly reminder…

NO UX LAB? NO PROBLEM!

-or-

Overcoming common obstacles preventing UX Research

In the last few years I’ve had several people come to me seeking advice about setting up easy-to-use, inexpensive, and effective usability testing kits. This includes an Amazon Research Team, a Facebook Research Manager, and a Principal User Experience Designer at REI. Why are these and others reaching out to me? Because they understand that research is vital to the success of their products and they don’t want technology, time, or money to prevent them from doing research.

These requests got me thinking about some common obstacles that prevent UX research from happening and what can be done to overcome them. Guided by the wisdom of one of the UX forefathers, Steve Krug, other seasoned UXers, and my experiences in UX tech and research I’d like to share three truths to show that you can overcome these obstacles and ensure UX research is not overlooked or dismissed.

You can easily create your own inexpensive and effective “UX Lab.”

Obstacle #1: Equipment is expensive and/or complicated.

During my time at Blink assisting with the technology side of UX research I was tasked with supporting numerous projects with needs beyond simple usability testing of a website. ‘Wizard of Oz’ testing of a smart kitchen appliance, testing of a app connected to a newly-designed gas-pump interface, testing the in-run experience of a redesigned running app, testing several iterations of components connected to the U by Moen smart shower, and a two-week diary study of a voice-activated speaker in participants’ homes are just a few of the more unique setups I’ve designed creative solutions for. While these projects have unique technical challenges, the majority of projects I’ve supported or led as a UX researcher are much simpler. The technical essentials are usually capturing a computer or phone screen, the participant’s face, and of course, audio from both the participant and the moderator.

I’ve utilized and I recommend two techniques for effectively capturing research sessions. With a few pieces of equipment and some training anyone can put these to use.

Technique 1 - Utilize Web Conferencing Software

Web conferencing software options are abundant and many of them are free or inexpensive. The tool that I recommend and have had great success with over the years is Zoom.

Utilizing Zoom to run remote moderated usability testing.

Zoom is designed to effectively capture whatever is needed during your research session. Zoom will automatically capture the participant’s face and audio and give you options for sharing either the participant's desktop or even mobile device. Additionally, the moderator is able to share their screen and give the participant control of the mouse. This is very effective when testing an HTML prototype or new designs you don’t want participants to have access to after the research session. Zoom also has the ability to automatically or manually record the meeting to your computer or in the cloud so, at the conclusion of your session, you have a PiP-configured MP4 file ready to be shared out or cut it highlight reels.

Getting stakeholders to watch sessions couldn’t be easier; just send them the Zoom meeting link and they can view from anywhere. The only downside is that there is the potential for your participant to feel like 10 people are watching them, especially if people are coming in and out of the meeting during the session. One way to limit this is to make an easy ‘Zoom Lab + Observation Room.’ Just do the following: the participant uses one computer in the 'lab' and one observer joins the meeting (muted w/ no video) and connects their computer to a large monitor, giving you a simple but effective 'observation room.'

Zoom’s many features, including all their screen sharing options make it a great, off-the-shelf research tool and the free version may be all you need.

Technique 2 - Stream Your Sessions to YouTube

If you’re feeling a bit more tech savvy and want more video/capture options then utilizing the free video production software OBS in conjunction with YouTube is another great way to capture and share research sessions.

View fullsize

Field kit for usability testing of a mobile app. Photo by Greg Hansen.

This technique takes advantage of free software as well as a free and very accessible location for live viewing and storing sessions. OBS is a free and open-source software designed for video recording and live streaming. For UX Research only the most basic features are needed but if you do have a study that has unique requirements it should be able to accommodate them.

OBS has designed live streaming to YouTube built right in and there are numerous resources out there to learn how to properly set everything up. Viewing sessions on YouTube is easy and secure; just ensure your live streams are 'Unlisted' so only those with the proper link have access, not matter where they are. This eliminates the 'many meeting participants' problem discussed earlier. The downsides to this technique are that there is more setup between sessions, some basic knowledge of setting things up in OBS (but I don't believe it's too complicated), and an investment in dedicated equipment.

For more specifics regarding building out your own adaptable and capable UX lab see my recommendations list.

You can find the ‘right’ participants provided you take an iterative approach.

Obstacle #2: Participants are hard to find and/or expensive.

In his book, Don’t Make Me Think, Steve Krug lays out several true things he knows about testing:

• “Testing one user is 100 percent better than testing none.”

• “Testing one user early in the project is better than testing 50 near the end.”

• “The importance of recruiting representative users is overrated.”

• “Testing is an iterative process.”

Some common perceptions around research or testing are that many participants are needed and that you have to find the exact right people to test with. This tends to make research into more of a high-stakes process when it doesn’t need to be. Krug’s philosophy is that testing should be early and often.

He believes that it’s good test with people similar to those who will use your product or service but more weight should be put on making testing an iterative process. The following diagram illustrates how testing twice with three participants in each test will identify more problems then doing just one test with eight participants. The key difference is that the problems identified during the first test are fixed before the second test.

View fullsize

Jakob Nielsen, of the Nielsen Norman Group, reiterates this philosophy in his article, “Why You Only Need to Test with 5 Users”. Through years of research and some sophisticated math, Nielson and another researcher found that five participants will provide approximately 85% of the insights your study will uncover. After the fifth participant, “you are wasting your time by observing the same findings repeatedly but not learning much new.”

View fullsize

When you follow the iterative testing process Krug and Nielson lay out research doesn’t need to be a high-budget, high-stakes thing. Multiple tests with a smaller number of participants will help make research a routine part of your design and development process and identify more insights for the same or even less cost.

With the right tools and equipment (discussed earlier), research costs should be lower and studies can be done anywhere; in your ‘Zoom Lab + Observation Room,’ with remote participants, or out in the field.

The iterative approach evangelized by Krug and Nielson should also reduce the anxiety of finding the ‘right’ research participants. Because testing is happening more frequently there should be less pressure compared to finding the perfect fifteen participants for a ‘big, all-or-nothing research study.’

When it comes to finding people to participate there are numerous creative ways to find them as well as industry tools specifically designed to supply you with research participants. I’ve had great success with Dscout, UserTesting, and Respondent, which are just some of the tools you can utilize.

With some creativity and persistence you can build a culture of informed decision making.

Obstacle #3: There isn’t enough time for research.

This may be the hardest obstacle to overcome and the one that doesn’t have clearly defined solutions. Creating a culture where research is prioritized is hard work but it can be done. I was fortunate to work on a team at Microsoft where design and development wanted to make decisions informed by research before too much time was devoted to design efforts or before elements went live.

Each organization is going to differ on how much they prioritize research and the techniques used to create more buy-in will differ from situation to situation but there are some tried-and-true methods worth noting.

Get People to Watch Research Sessions

While there may be some truth in the obstacle of time, I don’t believe people are lacking interest. So it is crucial you give them an opportunity to watch research happening. I’ve heard of and seen several tactics practiced that are effective.

Invite all your stakeholders to the research sessions. Set up calendar invites, make posters, send out all-company emails, personally invite them; just get them in the room. Steve Krug believes, “try to get [them] to at least drop by; they’ll often become fascinated and stay longer than they planned.”

Set up live viewings in common areas. Alaska Airlines broadcasted research sessions in one of their cafes and found it a great way to evangelize research efforts happening on their website and mobile app.  

Make viewing interactive—this will keep observers engaged and can even help with analysis. Provide an observation worksheet or something like the Rainbow Spreadsheet to fill out or have them jot down observations related to key questions on sticky notes—perfect for affinity diagramming later. Their insights are valuable and can help with analysis and recommendations.

Again, with the right tools and equipment, your stakeholders and observers can watch live UX research via Zoom or even YouTube.

Once they’ve had the opportunity to observe participants experiencing frustration with an account verification code thought to be working perfectly or the joy of successfully using a voice command to activate an IOT device you’ll have them hooked.

Create Compelling/Interesting Research Deliverables

Research reports are often stereotyped as dry and boring but there’s no reason they have to be. The insights and findings uncovered in research can have profound impacts so do what you can to make engaging, interesting, and empathy-inducing deliverables.

I’m particularly fond of highlight reels. They are quickly viewed, can be easily shared, and can have a huge impact on those watching them. For a multi-part study focused on DIY soap makers I created this highlight reel of participants sharing how soap making has impacted their lives. The stories shared in this video helped to build excitement around the design and research work we were doing.  

When it comes to effectively communicating your work don’t be afraid to build on the successes of others. There are numerous ways to learn from others who’ve gone before us. I’m fortunate to work with tremendous researchers and designers who are eager to share the lessons for creating compelling work they’ve learned along the way. Seek out UXers around you for advice and wisdom. I’m found that most are willing and excited to help others in the UX industry. I’ve also found the podcast Mixed Methods and their Slack group to be very helpful. Additionally, Medium is full of articles like this one, with great tips on how to ensure your research isn’t boring and people will read it.

Try Something New

When I was completing my User-Centered Design Graduate Certificate I had two instructors, Rebecca Destello and Justin Marx, who had previously worked together at a start-up where they developed a unique solution to their organization’s lack of research buy-in. The need to “quickly instill a culture of user validation,” as well as other needs, prompted them to develop a strategy they called “Witness Wednesdays” or “usability sprints.”

Rebecca and Justin define usability sprints as “a series of rapid-fire user studies, emphasizing team collaboration and organizational buy-in.” These sprints would take five days to run and were often executed as a four-week series. The basic outline for one week looked like this:

Monday

• Researchers and designers collaborate on the week’s session guide and prototype.

• Observers are reminded of their upcoming obligation on Wednesday.

Tuesday

• Researcher finalize the session guide and ensure the lab/observation room are ready.

• Designers make any necessary tweaks to ensure the prototype matches the session guide.

Wednesday

• Researcher conducts five, one-hour sessions.

• Observers record issues, quotes, ideas, surprises, etc. on sticky notes.

• Team debriefs and does quick affinity diagramming after each session.

Thursday

• Researchers, designers, and stakeholders discuss the results and ideate on top-line issues.

• Researchers generate a brief email report of the findings for the whole organization (because everyone on the email was in the observation room, an email is all that is needed!).

Friday

• Discussion and ideation continues, if necessary

• Given time, researcher begin next week’s session guide, and designers begin iterations to the prototype.

While this system had its challenges it also produced numerous great results for their organization. According to Rebecca and Justin, the biggest outcome of “Witness Wednesdays” was that it broke “the stereotype that ‘research take too long!’” Check out their talk at Convey UX 2018 for more about the unique system they developed.

⬡ ⬡ ⬡

Pivotal to overcoming these obstacles and deploying these creative tactics is ensuring that your research can be observed. Whether it’s during a collaborative live-viewing or in a compelling highlight video it all starts with an effective UX lab. Don’t have a UX lab? That’s not a problem! You can see there are simple, elegant, and effective solutions out there ready to adapt to your needs and the creative solutions you develop to promote UX research.

For more specifics regarding building out your own adaptable and capable UX lab see my recommendations list.

---

What are the ways you’ve overcome these obstacle? Please share your UX tech tips, strategies, and creative solutions by leaving a comment below. Thanks!