See the slideshare on this topic that I gave at UXPA Boston 2017
Measurement is a part of your life. You may have been avoiding it for a while, but it’s here to stay and you need to figure out how to measure your UX. Our friend Dr. Nielsen has been measuring user experience (“usability” as we old timers call it) since the Web was born, so we have a strong foundation to lean on. His website and the Usability.gov are tried and true examples of how to measure the usability of a website or app.
I love usability metrics because they focus on the core of user experience – Can a person use this tool to do what they need to do. Can they accomplish their task? Are they happy with it? It’s the foundation of user success – accomplishment! But it’s this focus where I find the some big gaps…
What are the metrics that can tell me if a user experience is actually leading to ROI?
A crappy product can have a wonderful user experience or a wonderful product can have a wonderful user experience, but it does need a justification to exist. We know the UX solves problems…helps make everything better for our poor users out there in the world, but how? Makes our brand pop! Yeah but, specifically, how? What is the result of all that effort? Clear measurement is the language of that specificity. The UX has an impact and measurement is the method to show that impact.
Measurement justifies the UX existence for the long-term. Our designs can make users happy and successful, but only if there’s budget and a team to do so. Budgets get allocated and teams created only if there’s a financial reason to be there. Measurement shows how a design makes money – or a return on that “investment." The good news is that one does not have to design an e-commerce site to see how the UX contributes to ROI. Measuring ROI is not just simple shopping cart math. There are several models where we can find metrics to show how any UX does or does not generate that return.
In my experience, I have found one can easily tie the business results of a user experience together by understanding overall business goals. Then, we can use strategies metrics from marketing funnel/sales pipeline and customer experience methodologies to get to our UX metrics.
1. Know The Business Goals and Trickle Them Down to Your Work
All businesses have goals that they need to measure against and report on to their board, shareholders, and public. Generic examples of these goals are growth, profitability, customer service, retention, and efficiency and they come from the top down. These goals are what your company is investing in and needs to show a Return On that Investment. Words like these have cropped up in e-mails from your own company leadership around each quarter. Maybe they made sense to you or maybe then didn’t.
It’s time to pay attention to those words instead of deleting the email or making fun of corporate speak. Knowing what your company leadership cares about means you can determine the focus of your work. If you work for an agency, then you care about what your client’s business goals. If you don’t know, ask.
The business’ goals are what you need to care about in your day-to-day job and not just in your yearly performance review. Understand and focus on the company goals and you have the technique to give your team priority on the “What” to show ROI. And as a bonus, if the term “managing up” has ever been a mystery to you, it’s now solved. The "up" is your bosses' focus and the company focus. Focus your work on how it contributes to the company’s goals and you’ll never be out of work.
So today, do these things:
- Reread the most recent e-mail about your company’s (or client’s) business goals or, if you truly want to geek out, read the first few pages of the annual report
- Review your current work
- Ask yourself, “How does my current work impact those business goals?”
- Reprioritize your project list
If it doesn’t, then you need talk to your boss and ask for assistance or reprioritization. If you’re the boss, then reprioritize your team’s work to match up to what your company cares about. With that new list, you have the big picture of how your work aligns with overall corporate goals, and ultimately contributes to ROI.
2. Translate Business Goals into UX metrics to Show ROI
Let’s get down to the nitty-gritty of determining the ROI metrics for your specific UX. The chart below breaks down corporate goals into what they most likely mean for your department and team. They focus on two areas where UX is needed: marketing and product. Both areas are “customer-facing” which means the department’s work actually impacts real people. Plus, they have existing models and metrics we can use in our toolbox, ones that are common vocabulary to team members in those two groups.
So how do we determine what to measure? Here’s how…
ROI to UX Metrics Example - Uber
Uber CEO Travis Kalanick shared his business’s goal to Business Insider. “Our intention is to make Uber so efficient, cars so highly utilized that for most people it is cheaper than owning a car." His goal for Uber is about efficiency. With this one statement, marketing and product teams have a very clear goal, a focus for their efforts, and a clear indication of what to measure.
If I were the UX director or a product manager at Uber, I would do three things immediately after hearing this statement:
1. Look at my list of projects and prioritize all those that increase efficiency
2. Find data on or measure the following to set a benchmark:
- Efficiency --> Time to task completion
- Utilization --> Average Uber ride price/Ideal Uber ride price
- Profitability --> Number of screens to complete tasks
3. Start prototyping or measuring based on how well the designs facilitate efficiency, utilization, and profitability to show improvement from that benchmark
Then, I would challenge my team to think about how the UX could help a driver make quick decisions on who and what to pick up, how to facilitate voice UI instead of touch, and think of usage scenarios beyond passengers.
3. Measure – Benchmark and Monitor
We have our focus, the right metrics, and now what? Well, now we gather the data. The data serves two functions – show where we’re at by benchmarking and then show the change based on the UX. We need to take our metrics and gather the before and after. The "before" is a quantitative picture of what the UX is like for your users today. The "after" is where we’re heading with that new UX. Both are what we need to measure those an ROI for the UX.
Measure Only Key Audiences: And when I say “your users” what I really mean is your company’s target audience, or the “who” will be making driving accomplishment of all those corporate goals. For example, if the company goal is growth and the metrics are user acquisition, then you really care about measuring the actions and design the UX for new users in a target audience. If you focus on existing customers, your measurements and UX may not be helping any of that growth. It’s the key audience's actions and behaviors that matter most to measure.
Find The Benchmarks: For each target audience, to show any improvement, you need to show improvement from where you are at today. The improvement (or decline, which is not always bad) you measure will show two things: 1) whether or not your design contributes anything to the user’s experience and thus ROI and 2) whether or not there’s actual, quantitative results of the design effort.
Show Improvement: This is the meatiest part – gathering data of those metrics that actually show if a UX or design made any impact to ROI. To do this, run a test or gather data on the ROI metrics. Some naysayers will say, "But how do you know your design is the only thing that contributed to improvement? There are so many things going on that we can’t measure."
100% attribution or causation is very rare anywhere outside a science lab. You can do a few things to make sure as much noise as possible is drowned out. First, you’ve chosen a few key metrics to measure that directly relate to the business goals and UX strategy. That’s a start - you’re not measuring things that don’t matter (i.e. the noise). Also, it’s also about timing.
Gather the benchmark measurement 90-days prior to launch and measure for change 90 days after. You’ll have a clear idea of how much the UX improved on those key metrics for the user. That timeframe will show an "evening out" of any unusual spikes like an unexpected press release or event. If you’re in the prototyping phase, run two usability tests, one that’s a benchmark with current site or product and one that mimics real world tasks and usage on your prototype design.
These two tasks mean that you can now show true business results. It’s the last step in tying your UX to an ROI and one that requires digging and patience, but it’s what makes the UX (and your job) truly an asset vs. a liability.
Make Friends with the Analytics Team: Your analytics team is now your team’s best friend. Find them and take them out to lunch. Ask them to walk you through measurements they have set up for tracking your website and in-app metrics. Share with them what your thoughts are on the UX metrics to the business goal. Ask them, "What are other metrics or ways to collect benchmarks and measure improvement?" I guarantee that any data geek will love to start this conversation. For example, if you want to track Net Promoter Score (NPS), ask about the best way to do a one-question survey on the product or website to gather data on the most important metric for any company that has a customer experience goal.
If you’ve made it this far, congratulations! Measurement, while everywhere, is not for the shy. You need to measure what you do and it’s needs to be meaningful to the others in your organization. And don’t worry. I have faith in you. Start with these three things:
- What your company cares about, is what you care about
- ROI metrics are now your UX metrics
- Show a before and after
Now, get going!