User Research and Its Inevitable Evolution
We are still in a time of tech layoffs and professional pressure: researchers have been laid off, UXers have lost their jobs, and entire swaths of staff have been removed from the corporate roster. The market is difficult for juniors (and has been for years), perhaps moreso now than it was three to five years ago.
Within my network — a large availability bias in that sample — the fact that researchers and UX people have been affected is readily apparent. Even experienced researchers have been laid off, some across multiple job roles during this period, and even for them, the market is difficult.
According to layoffs.fyi, since January of 2022, over 450,000 people working in the tech industry have been impacted.
Certainly, research is a less-established, less-integrated, and less-essential profession in the technology organization compared to engineering or design. It does not create an output directly visible to users. Its value for some kinds of organizations, especially those without a strong design perspective is less apparent. The pressure we're feeling in our profession is real.
I want to argue, though, that there is no structural fault in the research profession itself; no reason to believe that researchers have been dramatically misguided in developing our practice.
Along with higher interest rates, a corporate drive for operational efficiency, and aggressive staff reductions, there's no evidence that research-as-a-profession has a qualitatively lower standing with those who previously recognized its value, versus being a casualty of circumstance.
And even now, organizations that recognize the value of contextual insight and evidence-based product development continue to open and fill roles for researchers, as they should.
Perspectives and pathways of evolution
Let's look at where research is playing in the organization. The diagram below is an update to the view we established in a Spectrum of Perspectives. It gives us a lens to examine who, which teams, need research in the organization, in terms of their dominant functional perspective.
Our practices were traditionally rooted in design, and that's where the demand for research is the strongest. We're also moving into the product sphere, or at least, out of the UX world, and Peter Merholz notes some similar motion in "Whither UX Research?" and that research practices are attempting to embrace a more holistic scope.
At the same time, product teams are looking to solve a need for connection to customer context and evidence-based insights as a driver for the regular cadence of work.
These two sides, (1) design-rooted research looking more broadly and (2) product practices looking for more integrated insights, haven't quite figured out where to meet. I think some fit will emerge in the coming years, and first researchers will need to attune to product cycles and repeat activities, and product teams will need to realize that continuous discovery habits are not the best method for every situation.
The integration of these perspectives is one force that will grow the next phase of user research. And other factors are, and have been, substantially changing the landscape of our practice.
We've seen massive practice changes in the last 15-20 years, and the trends will only continue: at a similar, slow pace, but inevitably. If we're going to adapt to the future, we need to recognize the forces at play and anticipate how they'll unfold.
Four evolutionary forces at work
Compared to 15 years ago, it takes far fewer "researcher work hours" to accomplish the same outcome now. Research is remarkably more efficient, and that creates pressure when demand does not scale at the same pace.
So much of the practice still takes skills, attention, sensibility, and judgment, but these are no longer the cost of entry: previously it was access to users and interview moderation knowledge and coordination and logistics; now it's an email address and a credit card.
Let's take a look at some of the forces at play, so we can get a handle on their rate of change and consider what happens when they reach their inevitable, commoditized conclusions years or decades down the road.
#1 – Software efficiency: supplanting individual activities
Consider usability testing. The first Handbook of Usability Testing was published in 1998; Usertesting.com appeared in 2007; and I ran my first remote unmoderated usability test in 2015.
In 2024, we have User Testing, User Interviews, Lookback, Great Question and many more in the mix, each with their angle and varying degrees of desire to own end-to-end research processes.
This kind of product motion will move up the stack into our higher-order capabilities, on a similar timescale.
So many of the basic activities that were solely the realm of the researchers now have purpose-built vendors, products, or even platforms that perform this work. I'm talking about the work of
- finding the right kind of people to work with
- screening and inviting them to participate
- scheduling our sessions
- preparing and prompting tasks
- recording interview sessions
- getting transcripts
- conducting first-pass analysis
- clipping video highlights
- orchestrating synthesis and sensemaking
- and so on...
Evolution: All of these activities have evolved from custom-built for the organization, by the researcher and design team, to off-the-shelf products with expected behaviors.
Especially in the consumer space, vendor platforms can handle nearly all ofthe executional aspects of simple evaluative studies: in-place products, prototypes, and even low-fidelity or storyboard concepts.
Implication: The work is still essential — it just doesn’t take as much research capacity or even capability to execute as it used to. And as non-specialists with basic knowledge (or better yet training and coaching) can carry out the work, the line for “good enough” is moving as the cost-value tradeoff changes.
2. ResearchOps: dramatically reducing practice overhead
Even with basic tools and automation, teams of researchers would still operate more or less as individuals with their own style, techniques, tricks, and procedures for the basic operations above.
As researchers increase engagement with the organizational surface area, the amount of work to navigate it and the knowledge needed to do so also dramatically increase. That knowledge and corresponding practices navigating procedural, legal, logistical, and ethical concerns, would be fragmented across the research team depending on researchers' different scopes of work.
Inevitably, this knowledge and the larger set of practices came to be centralized as Research Operations. As early as 2018, Kate Towsey was speaking about ResearchOps, recognizing that not just researchers, but People Who Do Research ("PWDRs") will be making use of these services. Research operations is a formal, professional role now.
Evolution: Investment in research operations is an explicit recognition of the value of research practices. Administration, overhead, and procedural work are centralized and provided as a service: they are no longer custom-built by individual researchers. Over ~5 years, the isolation and formalization of these practices has led to a new set of professional roles.
Implication: The formalization of research operations amplifies the capabilities of individual researchers, further increasing efficiency. When an Ops team is in place, the barrier to entry for PWDRs to undertake this kind of work (depending on the attitude and orientation of the larger design-and-product group) is further reduced.
3. Expert knowledge: converging and proliferating
In 2010 if you wanted to learn about how to do user research, you would wade into academic literature, and then follow either the HCI → Usability line of thinking or the Design line.
On the Usability route, you might work with Usability Engineering (1993) by Jakob Nielsen, the Handbook of Usability Testing (1998) by Rubin & Chisnell, or follow Steve Krug's pragmatic guide Don't Make Me Think (2000).
On the Design route, perhaps you follow pieces of Alan Cooper's guide to interaction design, About Face (1995), work through Holtzblatt & Beyer's Contextual Design (1997), or much later, Kim Goodwin's marvelous Designing for the Digital Age (2009).
"User research" as we know it would later come to contain the practices of the Usability side, and isolate and extend certain components of the Design side. The shift from disparate expert knowledge into a professional place-to-play was accelerated in 2013 with two books: Steve Portigal's Interviewing Users and Erika Hall's Just Enough Research.
In the last 10 years, we've left the realm of pure academic and expert knowledge. We have industry-specific conferences, meetups, and communities. You can attend university courses, read other professionals' research manuals and handbooks, listen to podcasts, watch tutorials on youtube, read the latest editions of the above books, sign up for a boot camp, or invest in personal coaching.
Evolution: User research moved from chaotic (a bricolage of other practices' component methods) in the 1990s, to the domain of experts in the 2000s establishing and learning how to employ the practice, to the domain of professionals in the 2010s. The recent operationalization of our activities solidifies and reinforces a practice-knowledge that can be publicized, shared, understood, refined... and selectively built upon.
Implication: The craft of research is and will remain for some time a skill that requires a learning investment, practice, and attention to progress — it is a professional skill. But the genie won't go back in the bottle. The door is open for exploratory practitioners and emerging professions to take component parts from our own practices and begin to shape new ways of working.
4. Product teams: developing evidence-based practices
Given the prior three forces, it's no surprise that other practices and professions are starting to work with methods, activities, and capabilities that we once regarded as "our domain."
Product teams have always desired customer context and the ability to make well-informed and evidence-based decisions for their critical activities. And the practice of research, as rooted in Usability or Design was not necessarily amenable to product work cycles.
And so, unsurprisingly, product teams have also been looking to build new ways of informing decisions that fit with their own existing practices. Teresa Torres' Continuous Discovery Habits is a perfect example of the latest iteration of a new kind of product practice. To the deep consternation of researchers who believe that only researchers are allowed to speak with customers, product teams are already embracing, experimenting, and growing this practice that uses many component pieces of the larger research set.
Continuous discovery is itself not new: we have had rolling research and customer advisory boards and customer panels for quite some time. This unique product-led flavor is only possible now that recruiting, scheduling, recording, sharing, and analyzing customer conversations have evolved to an off-the-shelf productized state.
Evolution: Continuous discovery habits are still custom-built on a per-organization or per-team basis, moving from seeds of ideas 10-15 years ago into a recognizable set of practices with common features that can be learned, experimented with, and expanded into more robust ways of work in the next 5-10 years.
Implication: Teams work with continuous discovery and rolling research and customer advisory boards because it makes direct sense — customers are the source of insight. If we who are trained in user research see these methods merely as primitive forms of our own work, we take the stance of incumbents everywhere who fall into the trap of the innovator's dilemma.
Demand footprint and supply efficiency
The punchline of those first three forces is this: research isn't dying, demand for research capability is still growing, but the demand for research specialists to carry out the work does not necessarily track (we are more efficient.)
Those first three forces, combined with the fourth, mean that control of the work is less in the hands of the trained-and-titled researcher. Product teams want work that fits with their execution cycles.
Together, all of these forces point to where the future of research is evolving.
Watch slowly: user research is evolving
We're working with slow cycle changes, happening over decades-long scales.
While current economic conditions are amplifying the pressure, I argue that we have reached no point of catastrophe (save that for the unexpected singularity, or the next major evolution of AI?) but are witnessing the beginning of real changes in the practice. We also feel pressure from our ongoing convergence and standardization: course correction is inevitable.
This standardization is a step that enables the customer of research, our organizations and their teams, to understand our capabilities and adapt to or harvest from our converging practices, even as we adapt to theirs.
Research activities are moving toward commodity forms, which will take another 5 to 15 years. We can't predict exactly what new forms of value will be built with the productized, operationalized, and modular research components that are evolving.
What we can say, for the next evolution of this work, is that anyone worth their salt will understand and employ much of the researcher's knowledge and capabilities of today as a core component of their practice tomorrow.
This essay is part of my larger exploration into the evolution of research and its integration with product practices in the organization.
For behind-the-scenes updates, contextual commentary, and personal prognostication, try my newsletter.