Data-Informed Product Design

Use data from a variety of sources to inform your design—analytics, A/B tests, social media sentiment, customer service logs, sales data, surveys, interviews, usability tests, contextual research, and other studies.
Include numbers and context. Whether you call them quantitative and qualitative, studies and nonstudies, or Big Data and thick data, you need the numbers and the context to tell the real story.
Ensure that data is sensitive to the complexity of the human experience. Use averages sparingly, infer with caution, corroborate liberally.
Use data to track changes over time, explore new patterns, and dig deeper on problems

No process is one-size-fits-all, though. Depending on the goal, different combinations of data sources might be more actionable.
For acquisitions, you might want to pair analytics and competitive data from a source such as Alexa or SimilarWeb. To understand content strategy, combining specialized analytics from Chartbeat with intercepts might be the way to go. Understanding the recommendation cycle might require a combination of NPS scoring with interviews and social listening. The key is to create a multidimensional pictu

Bringing in more sources of data can reduce bias, but all data has some kind of bias.

There is no perfect data. You always must ask where data comes from, what methods were used to gather and analyze it, and what cognitive biases you might bring to its interpretation.

Big Data looks backward, A/B tests seem to focus on the small stuff, and analytics just skim the surface. There is some truth to all of these observations, but the core issue is not in the data itself; rather, it is in how it’s being used.

Big Data on its own is often not enough to drive innovation. It can lack context. And, more important, it can lack empathy—a strong tradition in the design community.

Data can reveal patterns and trends to drive innovation.
We can use data to incrementally improve the product experience.
Data can measure success, whether tracking across time, across versions, or against competitors.

UX is about more than just ease of use, of course. It is about motivations, attitudes, expectations, behavioral patterns, and constraints. It is about the types of interactions people have, how they feel about an experience, and what actions they expect to take. UX also comprehends more than just the few moments of a single site visit or one-time use of an application; it is about the cross-channel user journey, too. This is new territory for metrics.

Many experts will argue that Big Data, for all its power, can leave big questions unanswered, such as why people take action or why they don’t. Big Data is still mostly about the what, and less about the why, these same experts will argue. What they mean is that Big Data has a harder time understanding the reality of lived human experience in all its complicated glory, and understanding this reality of lived human experience is where the real insights lay.

If people are highly aware that they are being studied, the data is inherently different than if they do not.