How to Select a Test to Get Market Validation for a New Product or Business Idea

To test assumptions for a new business or new product idea is among the foundations of the Lean Startup methodology.
By selecting the riskiest assumption, defining an hypothesis, testing it with customers and then analysing the results to decide the way forward, entrepreneurs can systematically reduce the risk of launching a new product.
This is extremely important, since 72% of all new products fail to generate the expected results in terms of positive impact on profit.

But how do we choose the right test to validate the riskiest assumption?
Any 3 pennies out of 4 spent during the early stage of a new business or product idea are very likely to be wasted, so it’s best to minimise by:

• using cheap and quick tests when uncertainty is high
• using more expensive and longer tests when the riskiest assumptions have been tested and validated already

In order to facilitate the difficult task of choosing the right test at the right time, I’ve collected a library of tests to get market validation, combining my personal experience with what the Strategyzer team have written in their Value Proposition Design bestseller book.

The tests are ordered below from the cheapest and quickest one to the longest and most expensive. One note though: this is just an overview, intended to be a catalogue to facilitate choice, not execution. You will find in the text links to other posts where I’ve collected more details on how to conduct specific actions required by each test.

CUSTOMER DISCOVERY INTERVIEWS BASED ON A SCRIPT

This is by far the cheapest and fastest thing to do when validating assumptions related to customer needs and target segment behind a new product or business idea.
Following Steve Blank’s mantra “get out of the building“, this test requires entrepreneurs to recruit target customers, to meet them and let them tell about their experiences when they complete the jobs-to-be-done that the new product is addressing.
When conducting these interviews, it’s critically important not to jump into “pitch mode” and not to sell the idea.
Instead, these interviews are a precious learning exercise that will allow entrepreneurs to understand more about their customers frustrations, pains, alternative services they use and channels they have been exposed to. Plus the interviews will also help identify the early adopters (see here how).
I’ve written a few resources on how to practically conduct customer discovery interviews: here about recruiting, here about running interviews, here about what to ask to have customers telling the truth and not what you hope to hear.
I usually prepare a script ahead of the interview, which helps me to be focused and to lead the conversation instead of being carried away. This doesn’t mean that you have to conduct an interview while reading the script, it’s just supposed to be a guide.
Finally, interviews always work best if conducted in situ, meaning in the actual place where customers feel the pain and find a solution. This will help A LOT to get customers in context and gather meaningful insights.

– Use this test to: validate assumptions about customer needs and target segments for a new product or business idea, and gather insights to define a competitive value proposition
— When to use: you have just an idea, nothing else
— Time and resources required to set up: from just a few days to a couple of weeks, depending on how well the initial assumptions are defined, and how easy is to get access to target customers. A few hundred pounds might be required in case you hand out incentives and rewards to people interviewed.
— Evidence level: weak. It’s just a chat after all, customers are not going to buy anything — especially because you are not going to sell. If well conducted, and focused on past behaviours instead of hypothetical future actions, Customer Discovery interviews are the easiest and fastest way to learn whether we are wrong or right with our idea and the underlying needs.

CUSTOMER VALIDATION INTERVIEWS TO GET COMMITMENT

This is the cheapest and fastest thing to do when a competitive value proposition has been crafted based on customer insights, we want to get an understanding on whether it addresses customers’ pain points and does it better then competitors. It will also help validate the early adopter target segment identified.
These interviews usually start from going through customers frustrations about the jobs-to-be-done in scope, and then go ahead demonstrating a quick and rough prototype that represents the value proposition. Anything will work as a demo, from sketches on paper to a mock up. Just try to be lean and creative, and to spend as little as possible to get maximum impact.
The objective of Customer Validation interviews is to get feedback and get some form of commitment. Commitment will come in one of these forms:

– Time (e.g. schedule next meeting with clear goals, use a trial of a product for a reasonably long time)

– Reputation (e.g. intro to peers and team, intro to decision maker, public testimonial or case study as alpha user)
— Money (e.g. a letter of intent, a pre-order or a deposit). The commitment requested has to be correlated to the kind of prototype presented. We can’t really ask for a deposit after showing a sketch on a napkin. (If you manage to succeed, please let me know!).

Whatever the commitment, if this interviews end with no clear next steps, that’s a bad sign: it means that either the product needs iteration or pivoting or you are just asking for their opinion about your idea and you are not explicitly asking for clear commitment and next steps.

One last note about asking for commitment: you might be tempted to avoid the scary question. But a rejection is a good learning! You won’t never know whether the lead is real until you’ve given them a chance to reject you. And if rejection comes, it’s time to go back to the whiteboard. Good news is: you would have spend almost nothing at this stage.

– Use this test to: validate a value proposition for a new product or business idea and decide the next steps
— When to use: you have already defined a competitive value proposition based on customer insights
— Time and resources required to set up: from just a few days to a couple of weeks time, depending on how easy is to get access to target customers and on the complexity of what you are using as a demo. A few hundred pounds might be required in case you are handing out incentives and rewards to people interviewed.
— Evidence level: medium. It really depends from the kind of commitment you are going to ask for, but if they do commit, it’s a good sign that you have something valuable.

CUSTOMER DISCOVERY INTERVIEWS WITH MOCK-UPS, STORYBOARDS AND CARD SORTING

When there is the desire — or the need — to dive deeper into customer habits and put them in context in order to gather meaningful insights and validate assumptions about pain points, this test might be the right choice. Instead of running an interview in form of a simple chat, you can spend more time to prepare a storyboard representing the situation, and then use it to take customer through the journey of how they complete the job-to-be-done. This will facilitate the conversation and gather feedback about their past behaviours. Card sorting is also a very helpful exercise to help customers prioritise their pains or to structure complex situations or competing needs.

Again, when conducting these interviews it’s critically important not to jump into “pitch mode” and not to sell the idea. Instead, the interviews are a precious learning exercise that will allow entrepreneurs to learn about their customers frustrations, pains, alternative services they use and channels they have been exposed to. Plus the interviews will also help identify the early adopters (see here how).
Finally, interviews always work best if conducted in situ, meaning in the actual place where customers feel the pain and find a solution. This will help A LOT to get customers in context and gather meaningful insights.

– Use this test to: validate assumptions about customer needs and target segments for a new product or business idea, and gather insights to define a competitive value proposition
— When to use: you have just an idea, nothing else. But you are addressing complex needs that have to be investigated thoroughly
— Time and resources required to set up: from just a few days to a couple of weeks time, depending on how well the initial assumptions are defined, how easy is to get access to target customers and the complexity of the material being prepared for the interview. A few hundred pounds might be required in case you are handing out incentives and rewards to people interviewed.
— Evidence level: weak, but slightly higher than interviews with just a script. Customers have more opportunities to get into the context through to the collaterals presented.

LANDING PAGE AND ADVERTISING CAMPAIGN

This a classic way to test a business or new product idea in a quick and fairly cheap way. You just have to buy a domain, put a landing page live showing off the value proposition, and run advertising campaigns to drive traffic. Then you measure the conversion rate and you tweak either the customer targeted, the channel/message used for driving traffic or the proposition until you get a decent conversion rate. If you want to be super lean and minimise expenses, you might opt for one of those services that allow non-coders to create landing pages with a professional look and feel (WordPress.com, Wix or Squarespace to name a few).

Whatever you use to put the webpage live, it is absolutely mandatory that the landing page has a call to action in it, such as a button with a “leave your email to pre-order” or a “leave your email to get updates” message. Without call to action this exercise is wasted time, as you won’t be able to measure customers’ commitment. Besides, all these emails collected are going to be useful in a way or another (i.e. you could contact these customers when you have something to sell).

The landing page is so much a powerful and multipurpose tool that it can be used not only to validate a value proposition at a very early stage, but even to recruit customers to run customer discovery interviews in case you are struggling to secure meetings with them.

There is one important thing I feel I have to recommend: before starting to design the landing page, you should invest 10 minutes of your time to compile a lean canvas of the business idea. I put together a few tips on how to compile a lean canvas and an example here.

Compiling a Lean Canvas will help enormously to clarify the value proposition message, the channels and the customers to be targeted. You are going to need these things anyway, so better to start with the right feet.

– Use this test to: validate a value proposition for a new product or business idea, or recruit customers for customer discovery interviews
— When to use: either you have just an idea, or you have defined a competitive value proposition based on customer insights and you want to measure customers response
— Time and resources required to set up: from just a few hours to up to a week, depending on the tool you are using, and how refined you want the landing page to be. A few hundred pounds will be required to run the AD campaigns — traffic doesn’t come for free, to buy a domain and to pay for the website, either because you hired a developer and a designer or you purchased a premium subscription to a service.
— Evidence level: medium. These are not words anymore, but actual customer actions. Customer pre-ordering or leaving their email have clicked on an advert, gone through the website and found it convincing enough to leave their email address. A pretty good validation.

CROWDFUNDING CAMPAIGN

This is a quite popular way to validate a value proposition and get funding for the next steps. If you have done your home works right, i.e. you have gone through the process from initial idea to a value proposition validated by customer commitment (read more here), there is a good chance that the campaign is going to be successful and will help you raise the money to launch a Minimum Viable Product or improve/grow the existing one.
However, a crowdfunding campaign is not an easy feat, and the effort required shouldn’t be underestimated. The idea that you will start a campaign, share it with your friends and money will come with a snowball effect it’s comparable to be convinced to get in an off licence and buy a winning a lottery ticket. In fact, succeeding at a crowdfunding campaign is damn hard, and you won’t get the money pledged unless you at least achieve the funding target.

You might need support from video makers, PR agencies and potentially specialised agencies that create the strategy and execute it (this one is quite good, for example).

All this will divert your energies from the exercise of understanding target customers needs and defining a competitive value proposition to get their commitment. So it’s really pointless to start a crowdfunding campaign before you have completed those tasks. Considering the effort, it’s certainly not the first thing to do when you have a business idea in mind.

There are much faster and more effective ways to gather market insights and de-risk the idea instead (see above).

– Use this test to: validate a value proposition for a new product or business idea, and get funding required to launch a MVP or grow
— When to use: you have a competitive value proposition already validated with customers
— Time and resources required to set up: this will take at least two/three months, and a few thousands pounds. There is a lot of competition out there, and you need to invest if you want to win.
— Evidence level: quite strong. Customers are opening their wallets and pledging their money on your idea and team. A pretty good validation.

PRE-SALES AND A/B TEST ON PRICING

This a more advanced version of the landing page test. The difference is that in this case you are not just gathering customers emails, you are actually asking them to pay for your product, before it is available. A giant leap of faith for your target customers, and a huge validation for your value proposition. You might also implement an A/B test infrastructure so that you can see to which price customers respond better and adjust the pricing strategy accordingly. This is quite useful especially if you have no references in the market.

However, this test is quite advanced and does not come for free. The payment and A/B testing things require more complex tech capabilities than a simple landing page, and you are expected to have done all the home works (see here the 7 steps to validate a business idea) in order to be able to convince customers to give you their precious money for something that does not even exist yet. Not for beginners then.

– Use this test to: validating a value proposition for a new product or business idea, and get funding required to launch a MVP
— When to use: you have a competitive value proposition already validated with customers
— Time and resources required to set up: depending on the complexity and tech background of the founders, this might take from half-day to a couple of weeks. You will need to invest cash to drive traffic to the landing page, to buy a domain and to pay for the website, either because you hired a developer and a designer or you purchased a premium subscription to a service.
— Evidence level: very strong. Customers are paying for something that not even exist. They definitely fit the description of early adopters: they are so crazy about what you are doing that are willing to commit before it makes rational sense to do it.

CUSTOMER VALIDATION INTERVIEWS WITH A WORKING PROTOTYPE

From this test onward, we start dealing with real products. So no more sketches on paper, scrappy prototypes or promises, here we test the real stuff. With all the cash and time investment that follow. This test is the pro version of Customer Validation Interviews to get commitment (see above). Same process and objectives (get commitment), but instead of showing sketches on paper or a mock ups, you will show a working prototype of the product.

It’s easy to see that a rejection at this stage might cost quite a lot, and it might hurt badly, simply because building real things does not come for free.

So this test is good in case either you have a working prototype ready for any reasons (e.g. it comes from a R&D or University Lab), the product requires multiple real iterations because of its intrinsic nature, or you are so much advanced in the validation process that you decided to invest in creating something real.

– Use this test to: validating a value proposition for a new product or business idea
— When to use: you have a working prototype ready for some reasons, the product requires multiple real iterations, or you have a competitive value proposition already validated with customers
— Time and resources required to set up: this might take from a few days in case of a 3D printed object to several years.
— Evidence level: medium. It really depends from the kind of commitment you are going to ask for, but if they do commit, it’s a good sign that you have something valuable.

SELL CONCIERGE SERVICE

With this test we enter the realm of “fake it before you make it“, and we are validating customers demand by selling a makeshift product that we pretend to be the real thing.
Just to clarify: this does not mean we are deceiving customers! They are going to get the exact service for which they are paying for, nothing less. However, what is in the background is quite Do-It-Yourself, as what would enable the service it’s so complex, expensive and full of uncertainty that instead of building it we will be just doing it…through the founders’ manual work. An example might help: a guy called Nick Swinmurn had an idea to sell shoes online back in 1999. He could have raised enough capital to buy a giant warehouse, put in place a complex value chain, stock million of dollars worth in shoes and send them to customers ordering online. All this provided that he had been able to find someone believing in his idea and giving him funds, and most importantly, that customers were willing to buy shoes online. Instead, he made a deal with a few shoes shops in his neighbourhood, took pictures of the shoes and published them on his landing page. In case a customer ordered, he would go at the shop, buy the shoes and send them by post.

As you can see, selling a concierge service requires a lot of work from the founders, and it’s not intended to be ready for scaling, or even to make a profit.

Instead it’s a learning exercise, it helps validate a value proposition, de-risk the new business idea, and get plenty of insights to put together a meaningful financial plan. And in case orders start piling up, you will have enough validation to meet with investors and convince them to give you the funds required to automate the most time consuming and customer critical operations, starting in this way the proper infrastructure able to deliver the value proposition.

The concierge service requires a landing page with an order form able to accept customers payments, and advertising campaigns to drive traffic.

Ah, in case you were wondering, Nick changed the name of his website into Zappos.com and sold it to Amazon for $1.2B 10 years later. Not bad.

– Use this test to: validate a value proposition for a new product or business idea, get insights for a financial plan, get traction either to convince investors or to decide to take the plunge
— When to use: you have a competitive value proposition already validated with customers in cheaper ways, and you want to try the real thing
— Time and resources required to set up: providing the actual concierge service might be extremely time consuming, depending on the service
— Evidence level: very high. Customers are buying, leaving you feedbacks. It’s like you are really live with your business idea, just at a very small and DIY scale.

SELL BY PRETENDING TO OWN OR RE-LABELLING

Again, with this test we are “faking it before we make it“, and we are validating customers demand by selling a makeshift product pretending to be the real thing.
Just to clarify again: this does not mean we are deceiving customers! They are going to get the exact service for which they are paying for, nothing less.
However, we are selling something we do not really own or built. An example might help: a film sound service testing the value proposition for an brand new advanced recording technique have two options. Either to get a loan and buy £100k worth of equipment, or to rent it from another supplier, and sell it to their customers barely covering the costs. The advantage of the second option is that if their customers won’t respond positively, they would have just wasted a few hundred pounds instead of having a £100k to pay back to the bank.

So this test is about borrowing or renting something needed for a business idea before investing big money to buy it. Businesses like HSS hire or Appear Here are largely serving these kind of needs: before rushing to buy a tool or sign a contract for renting a prime commercial space for a shop, they allow entrepreneurs to “pretend to own” something or “re-label” it, postponing a sizeable investment until they have enough validation for the value proposition.

One more time, this is not intended to be made for scaling, and not even to make a profit.

Instead it’s a learning exercise, it helps validate a value proposition, de-risk the a new business idea and get plenty of insights to put together a meaningful financial plan.

Also this test requires a landing page with an order form able to accept customers payments, and advertising campaigns to drive traffic.

– Use this test to: validate a value proposition for a new product or business idea, get insights for a financial plan, get traction either to convince investors or to decide to take the plunge
— When to use: you have a competitive value proposition already validated with customers in cheaper ways, and you want to try the real thing
— Time and resources required to set up: it depends from the actual costs for renting or re-labeling the service/product. One thing is sure here: expect to spend money, potentially in the thousands.
— Evidence level: very high. Customers are buying, leaving you feedbacks. It’s like you are really live with your business idea.

SELL ACTUAL MVP IN TEST MARKETS

These last two tests are the most resource intensive ones, and they require founders to build the actual product and launch it on the market. This means that a potentially considerable amount of cash is going to be invested.
The concept of a Minimum Viable Product, or MVP, is one of the key principles of the Lean Startup methodology. It dictates to build an actual product to get customers commitment with a number of features reduced to minimum.
Twitter, for example, started as a way for broadcast SMS among a group of people, and there were no @replies, #hashtags or retweets. Fin-tech startups like Monzo or Tide launched early versions of their app-only banks using prepaid card or not allowing customers to set standing orders.
A MVP it’s a minimalistic exercise to provide target customers with the right set of features that will make the product credible, competitive, loved and bought. In order to minimise the risk, this test requires to launch the actual product in a selected market where there is the highest concentration of early adopters.

For example, the Halal food delivery startup HalalEat launched first in East London, where the concentration of Muslim people and restaurants is higher. After having validated the concept and the value chain and achieved considerable traction, they have collected so much credibility to raise funds to expand broadly from an investor.

– Use this test to: launch the actual product in the market with the highest concentration of early adopters, learn and fine tune operations and value proposition, and gather credibility to expand in adjacent markets later
— When to use: you have a competitive value proposition already validated with customers in cheaper ways, and you have funds to build and launch the real thing
— Time and resources required to set up: it depends from the actual costs of building and launching the service. One thing is sure here: expect to spend thousands of pounds and consider to work on this full time.
— Evidence level: quite high. Customers are buying, leaving you feedbacks. It’s like you are really live with your business idea, but in a sort of “safe” market

SELL ACTUAL MVP BROADLY

As above, the concept of a Minimum Viable Product, or MVP, dictates to build an actual product to get customers commitment with a number of features reduced to minimum.
A MVP it’s a minimalistic exercise to provide target customers with the right set of features that will make the product credible, competitive, loved and bought. The only difference from the previous test is that you are not only targeting the market with the highest concentration of early adopters, but you are expanding to adjacent ones.

This is not really a test anymore, but an actual business starting to grow.

For example when Uber started to test the market to deliver their value proposition of simplifying cab hailing, they have launched in 2010 in a single location, San Francisco, targeting the high-end of the market (professionals using black limo kind of cars) that they had identified as their early adopters (read more about how they did it here).
This allowed them to iterate and validate the proposition and to be credible in front of investors, show traction, and raise additional funds to expand broadly. So far Uber has raised at least the staggering amount of $11B in investment.

– Use this test to: expand your business
— When to use: you have successfully launched in a test market and you have funds to expand
— Time and resources required to set up: it really depends from the actual costs of expanding the business. I’d bet you are working full time on the business already, and probably hired someone to help
— Evidence level: very high. This is not really a test anymore, you have a business to manage now

An extended version of this article is available on StudioZao.com.