New Product Validation & Testing

A screenshot of how I collected research information in Airtable
287 surveys completed
20 customer interviews
28% increase in ARPU

When opportunity knocks you should answer.

This case study shows how I learned about a new product category and validated the business opportunity by speaking to customers.

I was talking to a customer about his marketing strategy and he told me that he had been stitching together 3 or 4 separate tools to create Resource Library functionality on his website. This homemade solution was time consuming to build and he was never 100% sure that his automations were working as intended.

I was convinced that there must already be a dedicated solution for this problem on the market. After a rudimentary search turned up very little I began to think that there may be a fairly large opportunity for us.

In this case study I share how I used Human-Centered Design techniques to validate a product hypothesis.

Project Skills

  • UX Design – user interviews, wireframes, usability tests
  • UI Design – mockups, prototypes
  • Front End Development – HTML, CSS

Research

Building a new product is a big commitment. Not only do you have to find time for design and development but you have to factor in the long-term marketing, maintainance and support.

Before I asked any other team members to commit their time I embarked on some comprehensive research of my own.

Existing Standards

It’s always tempting to start thinking about the solution right away. Even during those casual conversations with customers my brain was coming up with ideas for how I could solve their problem. It’s natural to get excited at this stage and jump straight into Figma but experience has taught me to take a step back and start at the very beginning.

I wanted to learn more about the finished product. What does a high performing resource library look like? How does it function? How is it built? What are the established standards in the world of resource libraries?

Learn From The Best

Companies that already invest in content marketing are likely to already have high quality resource libraries. And you know who invests in content marketing? Software companies.

My target audience is at the lower end of the market but I think it’s a good idea to see how the market leaders achieve their results. If I can bring some of that success to companies with smaller budgets then it’ll be a big win later for both our product and marketing teams.

I found a list of the top 100 software companies and I began visiting their websites to establish some basic criteria.

  • Do they have a resource library?
  • If so, what do they call it?
  • What are the basic features of the library?
  • Are they using a specific technology to build the resource library?

I recorded this research in a simple Airtable and I added some new fields whenever I found something interesting:

My analysis of resource libraries from the top 100 software companies

Now I knew for sure that:

  1. Resource Libraries are very common
  2. Most big companies build their resource libraries from scratch instead of using a dedicated tool

Based on this research my hypothesis was:

  1. Resource libraries are common becuase they are effective for generating leads
  2. Small companies can’t afford custom development to build their own resource libraries

Target Audience

At this stage I needed to speak to the small business owners, freelancers and marketing professionals who I hoped would become eventual customers for any new tool.

That’s a pretty big pool of people and finding good participants for an interview can be a real pain so I came up with an idea to reduce a lot of the leg work.

I took all my research from the top 100 software companies and wrote a blog post titled “What Makes a Good Resource Library?”. I know from my experience as a founder that this type of research-driven content performs well. I was confident that lots of people in my target audience would see my article so I included a call-to-action that linked to a pre-qualification questionnaire. This way I could let my target audience come to me instead of scouring the internet for interview subjects.

I used SurveyMonkey to create a simple questionnaire

The 5 questions I asked were:

  1. What is your email address?
  2. What role best describes you? (marketer, business owner etc)
  3. Do you already have a resource library?
  4. Is there anything you dislike or want to change about your current resource library solution?
  5. What matters MOST to you about a Resource Library?

My goal here was to get a good variety of people from different industries and I wanted some people who already feeling the pain of an inferior resource library solution.

The responses came in thick and fast so I knew were onto something. We got 287 completed surveys in just a few days so I was happy to call the blog post strategy a success!

Now for the interviews…

Jobs To Be Done

User interviews are only as good as the questions you ask. I like to use the Jobs To Be Done framework to give me some structure as I conduct the interview.

I began reaching out to survey participants and interviewing them one-by-one.

After each call I would write up some quick notes in Basecamp and attach the interview audio for both my own future reference and for my teammates. This is really a braindump of everything that was discussed on the call, there’s no analysis or categorisation at this stage.

Sample interview notes

I also created another Airtable so I could compare multiple interviews simultaneously and look for patterns:

In this table I wanted to clearly record the answers to some of the more pertinent findings from the interviews, including:

  • Details of their current solution (if they had one)
  • Their first thought (when did they know they needed a solution?)
  • Details of their struggle (how was life without a solution?)
  • Better life (how would their life look with the solution?)
  • Specific requirements of a new solution
  • Any dealbreakers

Job Story

I use my findings from the interviews to write a series of concise Job Stories to describe some of the ‘jobs’ my customers would be hiring any new product for:

Job Story 1

“When adding a new resource to my website I want to ensure that customers can find it so we can generate more leads.”

Job Story 2

“When adding a new resource to our website I don’t want to manually configure the opt-in funnel every time because it feels like repetitive busy work.”

Job Story 3

“When a resource is available on our website I want to be sure that it is connected to the appropriate marketing funnel.”

Job Story 4

“When reviewing our marketing resources I want to know which ones are performing well so I can use that information in my monthly reports.”

There was a bunch more of these and I would refer back to them throughout the visual design process to ensure that I stayed on track.

Wrapping Up The Research

Once my research was complete I had a really good understanding of the problems my target audience were experiencing. I had also documented my findings in a way that I could easily share with the rest of the team. Now I could start to think about solutions to these problems.

Design Overview

You can learn more about my design process here >