Build trust with remote users to get qualitative feedback

Over the past decade, software developers and growth marketers have automated most qualitative user feedback and testing. And yet, what about testing with communities like patients or senior citizens who may be more challenging to reach?

It was 2:00 a.m. at the Marriott Hotel in Singapore and I just wanted to get to bed after a 16-hour flight. As co-founder of a digital health company, I was in the process of building a community of test patients. Because of security and privacy concerns, I had to approach this process unconventionally; manually recruiting prospective testers online through administered groups and forums.

One of our test users had placed two urgent calls to me. I immediately called her back.

“One of our group members needs a new doctor. She is not doing well and needs a better specialist. I know you have a doctor on staff and I know it’s not his job but…umm, but…”

I interjected immediately.

“Don’t worry. You don’t need to say anything. We’ll do everything we can.”

Immediately, I dropped everything and called our company’s Chief Medical Officer to start a referral process. For the next few days, we fired off introductions to new doctors and assisted even though these tasks were not at all related to our company’s product. We were engaging with a non-conventional community which sometimes required going above and beyond the call of duty.

In recent years, product managers have fundamentally altered and automated usability testing for new products. Employing distributed labor marketplaces like Rainforest QA, Usabilia and Juicy Studio, growth-minded product managers have accelerated UI, UX, and backend product testing to ship faster and faster.

One of the most important subsets of usability testing, qualitative user feedback, has also faced an onslaught of automation.

And yet, there are numerous organizations who operate in spaces like healthcare, politics, or even eldercare where obtaining qualitative feedback is not that easily automated. Often, these are fields where security, privacy, and other restrictions necessitate a manual recruiting strategy focused on partnership and community development. A good example of this is a digital health company looking to test the first iteration of its product with patients where protected health information may be shared. Yet another example is an application focused on First Amendment violations targeting journalists or other at-risk groups where identity disclosure may be prohibitive. One needs to look no further than recent news of Google’s Nightingale project with Ascension Health to underscore the importance of the right policies and controls in these spaces.

I’ve learned the lessons in this space first-hand. Over the past two years, I have built a user community of patients who suffer from cardiovascular disease. For no monetary compensation, they are testing our company’s digital health application because they believe in its potential to make a difference in their lives and those of others. The most remarkable and fulfilling experience of my professional career, I have learned that to test your product with non-conventional users, you have to approach the process non-conventionally as well. In the words of Y Combinator Founder Paul Graham, you are going to have to “do things that don’t scale” and not be afraid of digging right in.

Specifically, you have to look for and recruit users in unexpected places; some of which resist automated growth marketing efforts. Second, you need to understand the value of partnership as these groups tend to resist more transactional relationships. And finally, you need to ask for permission and be honest and forthright with your intentions as to the testing process and the eventual product that you hope will hit the market.

Recruit in unexpected places

If recruiting test users in a challenging space like healthcare, law and order, or even eldercare, you need to seek them out in non-traditional and unexpected places. While you may think that online discussion forums like that are centered around user testing is the first place to go, there are other channels in which you can find more engaged and eager communities.

In medicine, community platforms including Patients Like Me and Care Opinion provide a key outlet to reach potential participants in a constructive and open way. In the political space, sites like Democratic Gain and Hill Zoo act similarly. Uniquely, these platforms have built-in security and approval features that protect users’ identities and allow them to only enter into conversations with their express and full consent. This is a key consideration for sensitive groups.

Facebook Groups allows for even more long tail recruiting but with the obvious and attendant risks that recruiting on an open platform like Facebook carries. Due to the closed nature of the Groups product as well as many built-in security features, Facebook Groups has escaped many of the information integrity issues and as a result, is one of the healthy components of the platform. Start by searching for a group in your space. Proceed by asking the group’s administrator for permission to engage with members by explaining your purpose and focus honestly. Often, in areas like medicine or politics, group members are eager to participate in testing new products where they can offer feedback in real-time and make an impact.

Even more fascinating is Quora . Quora’s platform emphasizes long tail discussions on a range of topics that even Facebook Groups cannot be narrow enough to encapsulate or cover. So, if you are looking for users with an extremely narrow focus, say those who are interested in testing a mobile app for tourism in the historic center of Hvar, Croatia, Quora may be your best opportunity to connect with like-minded individuals. Quora’s discussions can go quite deep, drive substantial value, and be generative of new product features.

It’s a partnership, not a transaction