How to research and test content and copy

This post is made up of material from the first few lessons of our Content Research and Testing course.

If you’re a UX writer, content strategist, developer, project manager, product manager, producer, or anyone interested in learning how to test and measure content, this course is for you.

Knowing how to test and measure your content is an essential skill

As UX writers, it falls on us to craft content that people want to engage with. We need to design content that results in sales, interactions, responses, or whatever success metric we may be measured against.

This is where many UX writers fall down. They’re able to craft the copy, and they’re able to explain why one phrase or flow might theoretically or anecdotally be better than another, but they don’t take the time to validate their work strategically using hard data.

Additionally, many UX writers lack the necessary skills to participate in user testing. In a business environment, you need to show executives and team leads not just that your copy converts, but why and how.

Thinking strategically is not a traditional writer’s first strength, but it must be yours as a UX writer or content strategist.

Strategic thinking about content research, measurement & data

When you have a research question that needs an answer, choose the method that will get you the answer you need.

As you first develop, and later optimize, your product experiences, you’ll begin to build a data-driven approach to answering content questions. The methods we introduce here align with standard UX research practices, but are adapted to helping you create the best content experience possible at every step of the user journey.

Keep in mind that you won’t use all of these methods for any one project or product. We want you to feel comfortable with many options so you can research and test your content as needed to ensure its success.

Establish process, goals & objectives

Good content is often seen as something that’s hard to measure. But actually content is easy to measure, you just have to know how.

~Rachel McConnell

Let’s start with process

You don’t want to feel haphazard when you test and measure content. If you follow a repeatable process, you’ll feel more organized—and you’ll be able to communicate the status of your testing to project managers and executives at a moment’s notice.

The “Analytics Process Model” is one famous process that is used as a bible for big data analytics experts. As a content strategist steeped in data, you should be familiar with it as well.

It looks like this:

(Source)

Let’s review these steps and then adapt this process to fit our content research and testing needs.

1. Identify the business problem

Although our goal is to identify content and design issues, never forget that user problems are always business problems. If users are unhappy or the product has issues, that will most definitely impact the business. When we identify our testing goals, we’ll want to align with our business goals as well.

2. Identify the data sources

This sounds like “big data” terminology. For us, we’ll identify our research or testing method.

3. Select the data

We won’t simply select data. Instead, we’ll identify the best “source” to answer the questions we have. Those sources might be internal stakeholders or internal subject matter experts (like customer support reps or product managers), but most often our source will be the customer.

4. Clean the data

“Cleaning” data means to remove any results from our data set that are not valid. Things like duplicate survey responses, usability testing subjects who were actually employees, or 20 app reviews that turned out to be from the same person.

5. Transform the data

“Transforming” data means to put it into a format where we can better understand it in the context of our work. That might mean transcribing handwritten notes into a searchable spreadsheet or mapping people’s experiences onto a user journey map.

6. Analyze the data

In this step, we soak up all that we’ve learned and draw conclusions and insights to inform our content decisions. We tabulate A/B test results, or consolidate survey feedback.

7. Interpret, evaluate, and deploy the model

Here’s where we implement changes based on what we learned through our research and testing.

For our purposes, we’ll condense these steps into our content testing process:

• Identify the research question and goal
• Choose the research method & audience
• Run the test and record results
• Clean and transform our findings into usable, shareable data
• Report insights, decisions, and implementation plans

The two types of research

Let’s take a look at a couple of terms you might be familiar with already: qualitative data and quantitative data. What is the difference between these two?

Qualitative (Qual) data is subjective information that can be hard to represent in numbers. It means measuring users’ feelings and opinions about the qualities of a product. Think about the quotes you gather through focus groups, interviews, and so on. That’s still “data”, it’s just uncovered in a different way, and expressed in different ways, too. Some tests help us to quantify this data to make its impact easier to measure.

Quantitative (Quant) data consists of metrics that can be counted and measured in numbers or values. For example, how many people clicked a button? How many people filled out a form? How many seconds did users stay on your homepage? Quantitative data is “hard” data, meaning it’s not based on opinions or feelings, but on measurable actions.

The Nielsen Norman Group (NN/g) puts it this way:

Understanding when, how, and why you use specific data at specific times is key to becoming a true content strategist.

For now, we’re going to focus on qualitative methods first.

Test and measure your content with detail

Many times, UX writers and content strategists forget that writing your UX copy is the last step in a longer process.

In fact, it’s often the last and easiest—because once you reach that point, you’ve already done the research that directs you regarding users’ language, wants, needs, and problems.

A lot of that hard work happens in the research stage. UX writers need to spend as long as possible talking with various experts along every customer touch point and pain point, document that entire process with journey maps and empathy maps, and use that to identify the overall key features and benefits that customers will experience.

Think about every action you take in the context of data

Remember, this course is about testing and measurement. The information and notes you take during the research stage are the first way to begin measuring.

As any scientist will tell you, you need a baseline in order to measure. The baseline is the standard against which you measure your experiments. The information you gather before writing is your baseline, and should be treated the same as any other type of data.

From there, voice and tone guidelines will help provide structure for the writing. But the research has done most of the work for you.

Clarify the purpose of your specific content before testing it

Any data-driven process needs benchmarks and success metrics. Define the scope of your copy and your goal so that you always have “success” and “failure” states in mind.

Finding the purpose of your content is the first question you should always begin with, no matter what type of project, its scope or the people involved.

Before testing, think about what is it that you are working on, and what you’re trying to accomplish.

Are you working on screens within an app, like onboarding? Or a particular feature?
If you are creating a marketing funnel, how is it going to be used, and why? To what metrics is this content connected?
Are you attempting to increase overall usability through content identified by specific user feedback?

Every piece of content has “work” that it performs, whether its job is to sell, to guide, to influence, or to prevent errors. Testing can answer whether the content in place performs that job effectively.

Create a repository for your data

Ever found yourself scratching out notes about customer language onto loose pieces of paper, only to find later that you either lost the notes or aren’t quite sure what on earth you were trying to jot down? If so, you’ll appreciate the idea of starting with some discipline and structure.

As a general best practice, you’ll first want to create a repository to collect all of your research and testing data. Ideally, that will be a folder in the cloud so you can share with your team and leadership as needed.

Whether it’s in Confluence, a Dropbox or Google Drive folder, or using a new product like Aurelius, be sure to create a central source for your research before you ever run your first test.

Include all of your notes and photos that you collect over time:

• Spreadsheets with summaries of issues and insights
• Participant photos
• Whiteboard photos
• Photos of user-completed worksheets and notes
• Your database or spreadsheet of interview quotes
• Details and notes from workshops
• Empathy maps
• Journey maps
• User testing videos
• Survey results

With this rich resource to back up all your work, you’ll be set to write content that withstands scrutiny.

Your goal as a data-driven UX writer or content strategist is to avoid subjective disputes about your content choices whenever possible.

Consider tagging your notes based on key questions, goals, or insights so they’re easier to find later. You can add a column to your spreadsheet or add a field to your database to hold tags like:

#pain-point
#pithy-quote
#strong-preference
#value-prop
#UI-guidance-issue
#feature-request
#confusing-language

It’s a beautiful thing to be able to pull up research at a moment’s notice to illustrate a point or expand on a test result.

Aligning research questions with business goals

There’s no point crafting tests if the research doesn’t answer a critical question that aligns with the goals of the business. To clarify your research questions, you’ll need to draw on as much knowledge from internal experts as possible before you start.

Get feedback from your team

When you decide to test content, get the buy-in of your design and product team members to make sure your outcomes will provide the qualitative and quantitative data you need.

Ask your team to brainstorm with you to identify content-related questions you need answered. In general, your research should seek to answer some (or all) of these questions:

Qualitative

• How do users feel about the brand promises?
• How do users feel about the product value proposition?
• How would users would describe the qualities of your feature, product, or brand?
• How would users describe your product voice? (Does it match what you intended?)
• Do users respond well to contextual terms used throughout the UI?
• Do users see themselves in the terms used to describe them?

Quantitative

• What percentage of users are successfully completing tasks?
• What percentage of users enter the sales funnel and make it all the way to a purchase?
• What percentage of user successfully make it past transition points, like clicking from a landing page to a checkout flow, or from your onboarding screens to your first task flow?

Ultimately, your testing goals should ladder up to your company’s larger goals. You might not know the larger sales goals or Key Product Indicators (KPIs) to aim for until you talk to people outside of your design team, like product leaders or executives.

Let’s take a look at a quick example. If the #1 goal for your company or your business unit is to become the market leader in your product space, target goals for your copy might be to:

1. Increase total number of signups (total number of users in your funnel)
2. Increase active users per day (a measure of maintained engagement)
3. Increase duration of first visit (an excellent measure of early engagement)
4. Decrease dropoff at any point (keep users moving forward and help them see value)

Your hypotheses, experiments, tests, and research should then focus on establishing direct results to inform you on the effectiveness of your copy in these areas.

Keen to learn more? Check out our Content Research and Testing course!