[Discovery Prototyping Techniques]


In a directed hack day, there's a customer problem (for example,
something is really difficult to learn and use, or it takes too long to
do) or business objective we've been assigned (for example, "Reduce the
customer churn rate" or "Increase customer lifetime value"), and we ask
people from the product teams to self-organize and work on any ideas
they like that might address this objective.
The goal is for the self-organizing groups to explore their ideas and create some form of
prototype that can be evaluated, and if appropriate, tested on actual
users.
There are two major benefits to these directed hack days. The first is
practical, as the technique facilitates the inclusion of engineers at
ideation. I've mentioned several times in this book that many of the
best ideas come from the engineers on the team, and we need to ensure
this is happening. It should be happening on an ongoing basis, but this
technique will ensure it happens.
The second benefit is cultural. This is one of my favorite techniques
for building a team of missionaries rather than mercenaries. The
engineers, if they haven't already, are now diving much deeper into the
business context and playing a much larger role in terms of innovation.


This is one of my favorite techniques for building a team of
missionaries rather than mercenaries.


There are many variations of hack days, but in this chapter, I describe
one of my favorite techniques to quickly get a range of high-potential
ideas that are focused on solving a pressing business or customer
problem.
The two main types of hack days are directed and undirected. In an
undirected hack day, people can explore whatever product ideas they
like, so long as it's at least loosely related to the mission of the
company.


[CHAPTER 44][Hack Days]


With a public API, you are essentially saying to the developer
community, "These are the things we can do---perhaps you can leverage
these services to do something amazing that we couldn't anticipate
ourselves."
Facebook's platform strategy is a good example of this. They opened up
access to their social graph to discover the types of things that
developers might be able to do once they could leverage this asset.
I have been a long-time fan of public APIs as a part of a company's
product strategy. I consider developers to be one of the consistently
best sources of truly innovative product ideas. Developers are in the
best position  to see
what's just now possible, and so many innovations are powered by these
insights.


I consider developers to be one of the consistently best sources of
truly innovative product ideas.


The Power of Developer Misbehavior
While the eBay example was intended to be used by end users (buyers and sellers), this same
concept is what's behind the trend toward exposing some or all of a
product's services via programmatic interfaces (public APIs).


This third alternative is to allow, and even encourage, our customers to
use our products to solve problems other than what we planned for and
officially support.
Mike Fisher, a longtime friend of mine, wrote a book called The Power
of Customer Misbehavior. This book tells the eBay and Facebook stories
from a viral growth perspective, but there are several other very good
examples in there, too.
From its earliest days, eBay has always had an "Everything Else"
category. This is where people could buy and sell things that we at eBay
couldn't anticipate people might want to trade. And while we anticipated a lot (there
were and still are thousands of categories), some of the biggest
innovations and biggest surprises came from monitoring what customers
wanted to do.
We realized early on in the eBay situation that this was where much of
the best innovation was happening, and we did everything we could think
of to encourage and nurture customers using the eBay marketplace to be
able to buy and sell nearly anything.
While the marketplace may have been originally designed to facilitate
trading items like electronics and collectibles, soon people started
trading concert tickets, fine art, and even cars. Today, amazingly, eBay
is one of the largest used car companies in the world.
As you might imagine, there are some very significant differences
between safely buying and transporting a car and buying a ticket that's
good for one night and then worthless. But that work was only done after
the demand had been established by enabling customers to transact in
items and ways the team and the company didn't anticipate.
Some product people can get upset when they find customers using their
products for unintended use cases. This concern is usually tied to the
support obligations. I'm suggesting, however, that this special case
can be very strategic and well worth the investment to support. If you
find your customers using your product in ways you didn't predict, this
is potentially very valuable information. Dig in a little and learn what
problem they are trying to solve and why they believe your product might
provide the right foundation. Do this enough and you'll soon see
patterns and, potentially, some very big product opportunities.


This technique is to allow, and even encourage, our customers to use
our products to solve problems other than what we planned for and
officially support.


You can think of the first as following the market, and the second as
following the technology. Either way can get you to a winning product.
However, some of the most successful companies today have taken a third
approach, and while it's not appropriate for every company, I would
like to suggest that this is an extremely powerful technique that's
largely underutilized and underappreciated in our industry.


[Try to assess the market opportunities and pick potentially
lucrative areas where significant pain
exists.]
[Look at what the technology or data enables---what's just now
possible---and match that up with the significant
pain.]


Historically, the two main approaches used by good teams to come up with
product opportunities have been:


[CHAPTER 43][The Power of Customer Misbehavior]


This is similar, but not the same, as spending some time with your
customer service or customer success staff. That is also valuable, and
often a good source of product ideas as well, but that is helping
customers once they call with a problem.
A concierge test requires going out to the actual users and customers
and asking them to show you how they work so that you can learn how to do their job, and so that
you can work on providing them a much better solution.
If you are building a customer-enabling product, the users may be
employees of your company, but the technique is the same---you go to
these colleagues and ask them to teach you how they do their job.
Like the principle of shared learning, it is most valuable if the
product manager, the product designer, and one of the engineers does the
concierge test.


A concierge test requires going out to the actual users and customers
and asking them to show you how they work so that you can learn how to
do their job, and so that you can work on providing them a much better
solution.


A concierge test is one of my favorite techniques for quickly generating
high-quality product ideas and, at the same time, working on developing
the customer understanding and empathy that's so important for
motivating the team and delivering strong solutions.
A concierge test is a relatively new name to describe an old but
effective technique. The idea is that we do the customer's job for
them---manually and personally. Just as if you went to a hotel concierge
and asked if he could find you some theater tickets to a popular show.
You don't really know the details of what that concierge is doing for
you to get those tickets, but you do know that he is doing something.
With this technique, you become the concierge. You do what the user or
customer needs done for them. You may have to ask them to train you
first, but you are in their shoes doing the tasks they would do.


[CHAPTER 42][Concierge Test Technique]


This hour consistently yields a great return on your time.


I would argue that this hour consistently yields a great return on your time. It's
critical to learn the answers to these key questions. However, I am a
big fan of taking the opportunity of a customer interview to also try
out some of our product ideas. We do that after we've learned the
answers to these key questions, but it's such a great opportunity I
really like to take advantage of it.
Later, when we talk about testing usability and value, you will see the
techniques for this. But for now, just know that you don't have to
wrap  up after the
interview---you can follow it up with a user test of your latest product
ideas.


[Frequency. Establish a regular cadence of customer interviews.
This should not be a once-in-a-while thing. A bare minimum would be
two to three hours of customer interviews per week, every
week.]
[Purpose. You are not trying to prove anything during these
interviews, one way or the other. You're just trying to understand
and learn quickly. This mindset is critical and needs to be
sincere.]
[Recruiting users and customers. I talk much more about this
when we discuss the usability testing technique, but for now, be
sure to talk primarily to people in your intended target market.
You're looking for about an hour of their
time.]
[Location. It's always amazing to see customers in their native
habitat. There's so much to learn just by observing their
environment. But it's also fine to meet them somewhere convenient
or have them come to your office. If you need to do this over a
video call, that's not as good, but much better than not doing at
all.]
[Preparation. Be clear beforehand what problem it is you think
they have, and think about how you'll either confirm or contradict
that.]
[Who should attend. My favorite is to bring three people to
these interviews: the product manager, the product designer, and one
of the engineers from the team (we normally rotate among those that
want to attend). Usually, the designer drives (because they've
usually been trained how to do this well), the product manager takes
notes, and the developer observes.]
[Interview. Work to keep things natural and informal, ask
open-ended questions, and try to learn what they're doing today
(not so much what they wish they were doing, although that's also
interesting).]
[Afterward. Debrief with your colleagues to see if you've all
heard the same things and had the same learnings. If you made any
promises to the customer during that session, be sure you keep
them.]


Now there are lots of ways
to get these answers, and if you have access to a user researcher, you
would normally follow their lead. Here are some tips for getting the
most out of these learning opportunities:


[Are your customers who you think they are?]
[Do they really have the problems you think they
have?]
[How does the customer solve this problem
today?]
[What would be required for them to switch?]


But in every user or customer interaction, we always have the
opportunity to learn some valuable insights. Here's what I'm always
trying to understand:


This is one of the most powerful and important skills for any product
manager and very often the source or inspiration for many breakthrough
product ideas.


The customer interview is the most basic technique I'll discuss in this
book. I wish I didn't need to include it because I'd like to be able
to take it for granted that product managers already know how to do this
well and do it frequently.
However, the reality is that this is often not the case. Or, if customer
interviews are happening, the product manager is not present, so the
learnings are not understood viscerally or taken as seriously as they
need to be (see Discovery Principle #10 on shared learning).
But no question, this is one of the most powerful and important skills
for any product manager and very often the source or inspiration for
many breakthrough product ideas. Later, when we discuss techniques for
testing your product ideas qualitatively, these skills will be a
prerequisite.
There are many forms of customer interviews, so this is not really a
single technique. Some are informal and some are more formal. Some have
a user research methodology behind them (one of my favorites is the
contextual inquiry), and others are more about just getting out of the
building and learning what you don't know.


[CHAPTER 41][Customer Interviews]


I have some favorite techniques that consistently deliver to the team
very promising and very relevant product ideas.
One important caveat, though. If you use these techniques, I am fairly
certain you will get very excited by many of the ideas you discover. But
that doesn't mean you should just go ahead and build them. In most
cases, we  will still need
to test them to ensure they are valuable and usable for our customers,
are feasible for our engineers, and are viable for
our  business.


How do we generate the types of ideas that are likely to truly help us
solve the hard business problems that our leaders have asked us to focus
on right now?


Overview
There are, of course, any number of techniques for generating product
ideas. I really haven't met many ideation techniques I haven't liked.
But to me, the more relevant question is, "How do we generate the types
of ideas that are likely to truly help us solve the hard business
problems that our leaders have asked us to focus on right now?"
Remarkably, in the vast majority of companies (not the ones that are
good at product), the actual product teams don't do much ideation
themselves. This is because what's really going on is that the ideas
are already handed to the product teams in the form of prioritized
features on product roadmaps, where most of the items on those roadmaps
are coming either from requests from big customers (or prospective
customers), or from company stakeholders or execs. Unfortunately, these
are rarely the quality of ideas we're looking for.
In general, if the product team is given actual business problems to
solve rather than solutions, and the product team does their job and
interacts directly and frequently with actual users and customers, then
getting a sufficient quantity and quality of product ideas is not really
a problem.


[Discovery Ideation Techniques]


The release succeeded in fixing the perception problems, but more
important, it genuinely made the version dramatically better for the
Macintosh. It was a product the Mac team could be proud of and what the
team felt they should have delivered to market in the first place.
This is a good example of how hard it can be to do the right thing for
the customer, often in the face of massive pressures. But that's
exactly what strong product managers figure out how to do.
In subsequent years, not only did Microsoft once again decide to diverge
the code base, they completely separated the teams into different
buildings and business units and had them fully embrace all things Mac.
Strategically, it was a complete 180.
It's hard to estimate how
important this was to both Microsoft and Apple. Even today, more than 20
years later, many businesses and consumers consider Word and the rest of
Office absolutely essential to using their Mac for business and personal
use. What started then became a multibillion-dollar win for both Apple
and Microsoft. There are more than 1 billion Macs and PCs running
Office around the world.
Martina has gone on to have a remarkable career in both product
management and product marketing. From Microsoft she went on to
Netscape, where she was responsible for marketing of the Netscape
browser, and then Loudcloud. And now I'm happy to say she's been my
partner at SVPG for more than a decade, and she also teaches marketing
at University of California, Berkeley.
Let me also add that there are few things as powerful as a marketing
person who's also strong at product. The combination is amazing.


This is a good example of how hard it can be to do the right thing for
the customer, often in the face of massive pressures. But that's
exactly what strong product managers figure out how to do.


In 1993, Word 6.0 was the biggest release, feature-wise, Microsoft had
ever produced.
In addition to all the new features, the team had another very large
objective. Their code base had diverged, and it was extremely slow and
costly for Microsoft to implement Word separately for each platform:
Windows, DOS, and Mac. This code convergence effort was supposed to save
Microsoft substantial development time, and---they tried to convince
themselves---improve the offering since Word would have the same
features on every platform.
It also meant that there was great pressure to get the release out so
they could start to gain the efficiencies of a single code base.
At the time, Word for Mac was a relatively small market. It was only
$60 million, versus Windows, which at that point was more than a $1
billion market. If you remember, back then Windows machines absolutely
dominated, and the future of Apple was not a sure thing. However, the
Mac community was also very vocal---with passionate fans of their
platform---and this community had very little love for Microsoft.
PowerMacs were just hitting the market, which had significantly faster
chips and more memory. Most of the team members were using those new
computers because the Word 6.0 beta in its early days was just too slow
on regular Macs. Of course, most of the Mac user base was not on new
PowerMacs---they were on regular Macs. Hardware upgrade cycles were much
slower then.
So, when Microsoft released the most "full-featured word processor ever
for the Mac," it crawled on their Macs---we're talking two minutes
just to start it up.
The community immediately started posting in newsgroups that Microsoft
was trying to "kill the Mac." Hate mail started streaming in from
everywhere, including e-mails directly to Bill Gates, who would forward
them to the team with messages like "This is depressing MSFT's stock
price. Fix it."
Enter Martina Lauchengco, a young product manager recently out of Stanford, whose job
it was to help turn this around.
The team quickly learned that, while it may be a worthwhile objective to
get to a common code base, it's an empty victory if the product that
results is not good. Moreover, users choose their devices and platforms
because they value what's different, not what's the same. From the
customer's point of view, they would rather wait a little longer and
have a better platform-specific solution, than simultaneously ship a
generic product on all platforms.
The team ended up focusing hard on performance and taking advantage of
what the Mac could do. They looked at when and how to load fonts, since
Mac users tended to have so many more than Windows users, and ensuring
all Mac keyboard shortcuts still worked.
They focused on word count---which is used 10 times a day by every press
person---to make sure that it was lightning fast, as the press used the
feature as their performance barometer. They even made it faster than
the feature on Windows.
The result was that in a couple of months they produced a 6.1 release
that was sent to every registered user with an apology letter---signed
by Martina---along with a discount coupon for future purchases.


[CHAPTER 40][Profile: Martina Lauchengco of Microsoft]


There are many ways of defining this critically important concept of
product/market fit. Unfortunately, most of them are largely subjective.
It is true that product/market fit is one of those things that "you know
it when you see it." It certainly shows up in terms of happier
customers, lower churn rates, shortened sales cycles, and rapid organic
growth. But the threshold for any of these can be tough to define.
Companies often spend countless hours debating what product/market fit
should mean for them and whether they have achieved it.
One of the most common techniques for assessing product/market fit is
known as the Sean Ellis test. This involves surveying your users (those
in your target market that have used the product recently, at least a
couple times, and you know from the analytics that they've at least
made it through to the core value of the product) and asking them how
they'd feel if they could no longer use this product. (The choices are
"very disappointed," "somewhat disappointed," "don't care," and "no
longer relevant because I no longer use."). The general rule of thumb is
that if more than 40 percent of the users would be "very disappointed,"
then there's a good chance you're at product/market fit.
While useful, as you might imagine there are lots of caveats here, depending on the type of
product and the size of the sample. I like this test for consumer
products and services, but for products for businesses, one of the
reasons I like this customer discovery program so much is that I
consider this a very practical and very effective definition of
product/market fit.
If we can get to the point where we have six reference customers in a
specific target market, we will typically declare product/market fit for
that market.
Remember product/market fit does not mean that you are done with working
on that product. Not even close. We will continue to improve that
product continuously for years. However, once we have those six
reference customers, we can aggressively and effectively sell that
product to other customers in that market.
So, each reference customer is a truly significant milestone. But, for
example, getting six reference customers in a given target market for a
B2B company is perhaps the most significant, meaningful milestone
business result for a product organization and something truly worth
celebrating.


Product/market fit shows up in terms of happier customers, lower churn
rates, shortened sales cycles, and rapid organic growth.


The core concept is the same for all four variations, but there are some
differences. I'll describe the variation for businesses first and then
describe the differences for each of the other uses.
I also need to point out that you would not do this program for small
efforts like features or minor projects. This is for larger efforts.
Good examples would be creating a new product or business, taking an
existing product to a new market or new geography or a redesign of a
product.
The basic driver behind this technique is that, with a significant new
product, the most common objection is that prospective customers want to
see that other companies, like themselves, are already successfully
using the product. They want to see the reference customers. In general,
the more reference customers the better but too few, and the prospective
customer is worried that the product is a special and only works for
those one or two customers.
For products and services aimed at businesses, I was taught years ago
that the key number is six reference customers. This is not meant to be
statistically significant---it is meant to instill confidence---and I
have found that number has held up over time. Again, more than six would
be even better, but we shoot for six because each one is a lot of work.
Single Target Market
Now these are not just any six customers. We are looking to develop six reference
customers in our specific target market or segment, so, the idea is to
find six similar customers. If you end up targeting two or three
customers from two or three different markets, this program will not
give you the focus you want and need.
In the earlier chapters on product vision and strategy, we talked about
the product strategy of pursuing a product vision by tackling one
vertical market after another. For example, first develop six references
for the financial services industry, then six for the manufacturing
industry, and so on. Or you can expand geographically in this same
manner (for example, first develop six references for the United States,
six for Germany, and then six for Brazil, and so on).
I do my best to persuade teams to not launch a product in the
marketplace until after they have those six reference customers. We
don't want to turn on the sales or marketing machine until we have
evidence that we can help them be successful, and the reference
customers are our best evidence.
The concept behind this technique is to focus on developing this set of
reference customers for a specific target market, which then makes it
easy for sales to go after those specific types of customers. Once we
have those reference customers for that initial target market, we can
move on to expanding the product to meet the needs of the next target
market.
Recruiting the Prospective Reference Customers
We want to end with six reference customers, so we'll typically recruit
between six and eight in case one or two turn out to be not a match or
unavailable. We need them to be from the specific target market we are
going after. They may be from your existing customer base, prospects, or
a blend.
We are looking for prospective customers that truly feel the pain and
are near desperate for the solution we want to build. If they could find
a solution that worked for them elsewhere, they would have already
bought it.
However, it's also important we screen out technologists. These people
are mainly interested because of the technology, not because they
desperately need the business value.
We need them to have
people and time willing to work closely with us. They need to be willing
to spend time with the product team, testing out early prototypes and
helping the team ensure the product works well for them. If possible, we
would like them to be well-recognized marquee names, because that will
be of the most value to the sales and marketing staff.
Coming up with the right set is normally something the product manager
does in tight collaboration with the product marketing manager.
The Relationship
The benefit to the prospective customer is that they get real input, not
lip service, to the solution---and, most important, they get a solution
that truly works for them.
The benefit to the product team is that you get ready access to a set of
users and customers that you can go deep with and figure out a solution
that will work for them. They've provided you access to their users.
They have agreed to test early versions. And, what's really important,
they have agreed to buy the product and serve as a public reference if
the resulting product works for them.
It's critical to explain to each prospective member of the program that
your job is to come up with a general product---something your company
can successfully sell to a large number of customers. You're not trying
to build a custom solution that only works for that one company (and
they wouldn't want that in any case as they would be left with
unsupported, dead-end software). You are, however, deeply committed to
coming up with a product that works extremely well for them and just a
handful of other companies.
Further, your job as product manager is not to put in the features that
all six companies request. While that would be much easier, that would
yield an awful product. Your job is to dive deep with each of the six
customers and identify a single solution that works well for all six
customers.
There are a number of important points to consider with this technique.
Not everyone agrees with
me on this, but I don't personally like the customer to pay in advance
to participate in this program. That makes this a different type of
relationship. You want a partner in coming up with the product. You do
not want to build a custom solution just for them, and you're not a
custom project shop. You can take their money after you deliver them a
product they love. That said, if you're a very early stage startup with
little cash, you may have to bend this rule just a bit. A compromise is
to have them put the money into escrow.
If you are working on an important and difficult problem, you will
likely be overwhelmed with customers that want to participate. It really
is a good deal, and customers know this. If you have a sales
organization, they'll try to use this as a bargaining chip, and the
result is that you'll be leaned on to include many more customers than
you can handle. This will take finesse at times, but it's important
that the members of the customer discovery program be the right set, and
no more than eight. However, it's no problem also having an early
release program that is essentially unlimited for those customers that
want the software early, but you determine aren't right for the
customer discovery program.
Note that, in many cases, you'll get people who say they are extremely
interested in this product, but they first want to see your references.
When you explain you're looking to work with them to become one of
those references, they will probably say they are just too busy, but to
come back once you have the references. That's fine. They're a useful
lead. But we are looking for those customers that are so hungry and
desperate for a solution that they will absolutely make time for this.
Every market has this segment.
That said, if you find you are having real trouble recruiting even four
or five prospective customers for this effort, then it's very possible
you're chasing a problem that isn't that important, and you will
almost certainly have a very hard time selling this product. This is one
of the very first reality checks (aka demand validation) to make sure
you're spending your time on something worthwhile. If customers aren't
interested in this problem, you may want to rethink your plans.
You need to make sure your customers are truly from your target market
and not more than one target market. A big benefit of this program is
focus, and that means the customers are from a single target market.
You will want to work  with
your product marketing manager to ensure that the prospective customer
has permission from their marketing organization to serve as a public
reference. You will also want to keep your product marketing partner
continuously involved in this program as she can help turn your
reference customer into some great sales tools and collateral. But,
remember, it is your job to develop those actual reference
customers---so be sure you deliver a product they love.
Think of these early prospective customers as development partners.
You're in this together. You need to treat them as colleagues---open
the kimono, you are helping each other. You'll find that the
relationships you create can last for many years.
You will be interacting with these people throughout the
effort---you'll be showing them prototypes and testing with their
users, you'll be asking many detailed questions, and you'll be testing
early versions in their environment.
Make sure you release the delivered product to these people before the
general release, and make sure they are live and happy before the
release. When you launch, they'll be ready to stand up for you.
Now let's consider the common variations of this program for different
types of products.
Platform/API Products
For developer products, the program is very much like the one for
businesses, but the main difference is that we work with the development
teams (engineers and product managers) that will use our APIs to get
them successfully using our product. The result of the program is a set
of reference applications rather than reference customers. We focus on
the successful applications created with our APIs.
Customer-Enabling Tools
For customer-enabling tools, such as a new dashboard for your customer service agents, we
pick six to eight well-respected, influential internal
users/employees---the individuals that the other agents look up to as
thought leaders---and we work closely with them to discover the
necessary product. Obviously, they are not customers and not paying
anything, but instead we ask them to work closely with us through
product discovery to make this tool great. Once they believe that the
product is ready, we ask them to tell their colleagues how much they
love the new tool.
Consumer Products
For consumer products, the same general concept applies. But, rather
than focusing on six businesses to work closely with (where we have
access to many different users at each customer), we instead focus on a
somewhat larger number of consumers (on the order of 10--50) that we
engage with to get them to the point that they are loving our product.
It's important to emphasize that, for consumer products, we will need
to supplement this program with much broader testing of our product
ideas---typically with people that have never been exposed to the
product. But it is often very helpful to have a smaller group of
prospective users that we can go back to over time, and that's what
this is for.
In terms of marketing, when a consumer decides to buy or use a product,
they may not look at reference customers as a business purchaser would.
But they are affected by social media, the press, and other influencers,
and when the press does write a story about your product, the first
thing they'll look for is real users.
Summary
While you can see this may be a lot of effort, especially for the
product manager, this powerful technique help ensures that you're
building a product that customers love.
Remember that this technique is not designed to discover the necessary product---that
comes next. Rather, it is designed to give you direct access to the
target customers where you'll find the product ideas necessary to
generate reference customers.
Defining Product/Market Fit


[Building products for businesses]
[Building platform products (e.g., public
APIs)]
[Building customer-enabling tools used by employees of your
company]
[Building products for consumers]


The reason I love the customer discovery program technique so much is
because it is designed to produce these reference customers.
We are discovering and developing a set of reference customers in
parallel with discovering and developing the actual product.
I will warn you that this technique takes substantial effort, primarily
on the part of the product manager. I wish it were easier. But I will
also say that if you do this technique, I consider it the single best
leading indicator of future product success.
I will also say that this technique is not new, although every few years
some influential person  in
the product world rediscovers its power and it gets attention once
again. It also goes by multiple names. In any case, I'm convinced that
everyone would do the technique if it didn't require so much actual
work.
There are four main variations of this technique for four different
situations:


We are discovering and developing a set of reference customers in
parallel with discovering and developing the actual product.


Let's be clear about what
it means to be a reference customer: This is a real customer (not
friends or family), who is running your product in production (not a
trial or prototype), who has paid real money for the product (it
wasn't given away to entice them to use it), and, most important, who
is willing to tell others how much they love your product (voluntarily
and sincerely).
Please believe me when I say that there are few things more powerful to
a product organization than reference customers. It is the single best
sales tool you can provide to your sales and marketing organization, and
it completely changes the dynamics between the product organization and
the rest of the company.
Ask any good salesperson the single best tool you can provide to help
her do her job, and she'll say, "happy reference customers."
If you find that you're constantly frustrated by having to react to
sales and the latest big-deal prospect they've managed to bring in,
this is how you turn the situation around.
Without reference customers, it's very hard for the sales team to know
where the real product/market fit is. And remember---they have a quota
and are paid by commission. So, without good examples, they will sell
however and whatever they can. Without reference customers, this
situation is not their fault---it's your fault.


There are few things more powerful to a product organization than
reference customers.


Our job in the product organization is to create products that can
sustain a business. Make no mistake about it: Everything depends on
strong products.
Without strong products, our marketing programs require customer
acquisition costs that are too high; our sales organization is forced to
get "creative," which drives up cost of sales, lengthens the sales
cycle, and puts downward pressure on price; and our customer success
organization is forced to take it on the chin every day with frustrated
customers.
The downward spiral continues because the sales organization loses a lot
of deals when they try to compete with a weak product. So, what do they
do? They start yelling at you about all the features you don't have,
and the competitor they lost to who does, which typically just makes the
bad situation even worse. And then you start complaining about working
at a sales-driven company.
Many of you may be thinking I've just described your company. Sadly, I
find this to be the state of affairs in far too many companies,
especially those with either a direct sales organization or an
advertising sales organization.
This entire book, in one way or another, is intended to prevent or
correct this situation. However, in this chapter, I talk about what I
consider one of the most powerful techniques we have to ensure and prove
we have a strong, viable product and prevent the situation I've just
described.
The Power of Reference Customers
First, we need to talk about the nearly magical power of a happy
reference customer.


[CHAPTER 39][Customer Discovery Program Technique]


These are two-dimensional maps, in which major user activities are
arrayed along the horizontal dimension, loosely ordered by time from
left to right. So, if there are a dozen major user activities, they
would be along the top from left to right, generally in the order you
would do them---or at least, if you were describing the overall system
to someone else, the order in which you'd describe them.
Along the vertical dimension, we have a progressive level of detail. As
we flesh out each major activity into sets of user tasks, we add stories
for each of those tasks. The critical tasks are higher vertically than
the optional tasks.
If you lay out your system this way, you can, at a glance, get the
holistic view and consider where to draw the line in terms of different
releases and their associated objectives.
Now each story has context. The entire team can see how it fits in with
the other stories. And not just as a snapshot in time. The team can see
how the system is expected to grow over time.
We can use this story map to frame our prototypes, and then as we get feedback on our
prototypes and learn how people interact with our product ideas, we can
easily update the story map to serve as a living reflection of the
prototypes. As we finalize our discovery work and progress into
delivery, the stories from the map move right into the product backlog.
Many teams I know consider a high-fidelity user prototype and a story
map as their go-to techniques.
Another must-read book for product managers: User Story Mapping:
Discover the Whole Story, Build the Right Product, by Jeff Patton
(O'Reilly Media, 2014).


Many teams I know consider a high-fidelity user prototype and a story
map as their go-to techniques.


Story maps are one of the most generally useful techniques I know. They
are essentially a framing and planning technique, but they are just as
useful for ideation. They are also used as a design technique when
working on prototypes, and they are great for communicating with your
team and stakeholders. They also play a very practical role in managing
and organizing your work. Further, a story map is helpful throughout
product discovery and delivery.
I think you'll agree that's a lot of benefits. But the best part is
how simple it is.
The origin of story maps came from frustration with the typical flat
backlog of user stories. There's no context, just a prioritized list of
stories. How can the team know how one story fits in with the big
picture? What does it mean to even prioritize at that granularity with
so little context? And what set of stories constitutes a meaningful
milestone or a release?
Jeff Patton, one of the early Agile thinkers, was frustrated by this, so
he leveraged some proven UX design techniques, and adapted them to Agile
concepts and introduced user story maps.


[CHAPTER 38][Story Map Technique]


Overview
Now that we've framed our discovery work, we're ready to jump in and
start figuring out solutions. For complicated product efforts, it often
helps to have some way to scope out and plan your discovery efforts.
In this section, I describe two of my favorite discovery-planning
techniques. One is simple (story maps), and the other is fairly
complicated (customer discovery program), but they are both remarkably
powerful and effective.
I don't want to scare you away from a technique just because it's a
lot of work. I often tell product teams that if they could only pick a
single technique, the one I'd recommend is the customer discovery program. Yes, it's a
lot of time and effort---especially on the shoulders of the product
manager---but it's my favorite leading indicator of future success. I
attribute much of the success in my own career to this technique.


[Discovery Planning Techniques]


I think this is at least partly because risk is subjective and hard to
quantify. So, depending on your perspective, you may think some risk is
secondary when I think it is primary.
Mostly, however, I think the major reason is that it's human nature for
people to focus more on those areas they feel they can control and are
knowledgeable about.
So, let's say your startup founder is someone who comes from a business
background, probably trained as an MBA. He or she is likely acutely
aware of the risks associated with coming up with a good business model.
They're often focused  on
unique value proposition, pricing, channels, and costs. These are all
real risks, which are part of assessing business viability.
But, I will often have to sit these people down and explain that, while
these are real risks, they are largely academic at this stage. And then
I try to point them at what, in my experience, is the biggest reason
that startups and new products fail.
You're probably thinking that I'm speaking of market risk---that the
new product is focused on solving a problem that customers just don't
care enough about. This is a very real risk, and one that's responsible
for its share of failed efforts, but I argue this is not usually the
most significant risk.
I need to mention a couple caveats here.
First, I have to say that the vast majority of the teams I meet are not
solving truly new problems. They are working on long-standing problems
with long-proven markets. What's different about the startup or product
is their approach to solving the problem (their solution), most
often---and increasingly---because they are leveraging newly available
technology to solve the problem in an innovative way.
Second, if the market is indeed new, then today the techniques we have
for validating demand have never been better. If you don't use these
techniques, you proceed at your own peril. This is an especially
egregious mistake because the techniques are not expensive in terms of
money and time, so there's just no excuse not to do this.
I believe the major risk facing most efforts is value risk. On a
startup canvas, this shows up under solution risk---discovering a
compelling solution to customers. A solution that your customers will
choose to buy and use.
This is generally hard enough but realize that to get someone to switch
to our new product, it's not enough that it's comparable (sometimes
referred to as feature parity), it must be demonstrably and
substantially better. This is a high bar.
However, if you've created a canvas before, you know that there's precious little in there
about the solution. The official rationale for that is that it's far
too easy to fall in love with your particular approach and lock yourself
in prematurely. In fairness, this is a very real issue with teams. I see
this behavior frequently. But a consequence of this meager
representation of the solution in a canvas is that it plays to the
tendency of many to focus on those risks they feel more comfortable with
and leave the solution as "an exercise for the engineers."
Rather than delegating or deferring figuring out the solution, we need
to embrace product discovery as the most important core competency of
the startup.
Look, if you can discover a solution that your customers love, then you
can tackle the risks of monetization and scale. However, without that
solution, the rest of your work is very likely going to be wasted. So,
whether your constrained resource is cash or management's patience, you
need to make sure you primarily use your time to discover a winning
solution. Get that risk resolved first and then you can focus on the
other risks.
The point is that you don't need to spend your time doing pricing
optimization testing, sales tools, marketing programs, and cutting
costs, until and unless you have discovered a truly valuable product.


It's human nature for people to focus more on those areas they feel
they can control and are knowledgeable about.


The Biggest Risk
One of the things I like about a startup canvas is that it helps to
quickly highlight the key assumptions and major risks facing a startup
or a significant new product in an existing business. This is a good
thing. The idea is to tackle the biggest risks first. At least that's
the theory.
In practice, I keep running into entrepreneurs and product leaders who
are focused on secondary risks rather than primary risks.


For decades, people would create thick business plans to try to
highlight these topics and how they intended to tackle them. But many
people, including me, have written about the many reasons those old
business plans were often more harmful than helpful.
A startup canvas, its close cousins the business model canvas, and the
lean canvas are intended to be lightweight tools to call out these risks
early and encourage the team to tackle them up front.
I much prefer the startup canvas to old-style business plans, but I have
also observed that many startup teams still spend too much time on the
canvas and keep postponing that pesky little problem of discovering a
solution that people want to buy (see the box "The Biggest Risk").
You can use a canvas for any product change, no matter the size, but you would likely quickly
find that, once you have an existing product and business, the majority
of the canvas doesn't change and is only duplicated. You already have a
sales or distribution model. You already have a monetization strategy.
You have a well-defined cost structure. You are mainly trying to create
more value in your solution. In that case, it probably makes sense for
you to look at one of the earlier framing techniques.
That said, you can use the startup canvas for simpler work, especially
if you have a new product manager. The startup canvas can help that new
product manager get a good holistic understanding of her product and
understand the key areas of the affected business.


You're not being asked to improve an existing product, you're being
asked to invent an entirely new product.


So far, we've explored techniques for typical-sized, smaller efforts
like adding a new feature, or medium to large-sized efforts like a
redesign. Those cover most of what product teams actually work on.
However, another especially difficult situation requires a more
comprehensive framing technique. This is an early stage startup, where
you are trying to figure out a new product that can power a new
business, or, for those that work at an enterprise size company, when
you're asked to tackle an all-new business opportunity for the company.
In other words, you're not being asked to improve an existing product,
you're being asked to invent an entirely new product.
In this situation, you have a much broader set of risks, including
validating your value proposition, figuring out how you intend to make
money, how you plan to get this product out to your customers and sell
to them, how much it will cost to produce and sell this product, and
what you will measure to track your progress---not to mention
determining whether the market is large enough to sustain a business.


[CHAPTER 37][Startup Canvas Technique]


The idea is that the product manager frames the work ahead of the team
by writing an imagined press release of what it would be like once this
product launches. How does it improve the life of our customers? What
are the real benefits to them? You've all read a press release
before---the only difference is that this is entirely imagined. It is
describing a future state we want to create.
It's so tempting for product teams to immediately slip into an enumeration of all the features
they plan to build, with little real thought into the actual benefits
for our customers. This technique is intended to counter that and to
keep the team focused on the outcome, not the output.
The actual reader of this press release is the product team, related or
impacted teams, and leadership. It's a terrific evangelism
technique---if people don't see the value after reading the press
release, then the product manager has more work to do, or perhaps should
reconsider the effort.
Some people also consider this a demand-validation technique (if you
can't get your team excited, maybe it's not worth doing). It's only
validating demand or value with your colleagues rather than with real
customers, however, so I think of it primarily as a framing technique.
In any case, Walker Lockhart, a former long-time Amazonian who joined
Nordstrom a couple of years ago, shared with me a variation of this
technique that was developed and refined at Nordstrom.
The idea is that rather than communicate the benefits in a press release
format, you describe them in the format of a customer letter written
from the hypothetical perspective of one of your product's well-defined
user or customer personas.
The letter---sent to the CEO from a very happy and impressed
customer---explains why he or she is so happy and grateful for the new
product or redesign. The customer describes how it has changed or
improved his or her life. The letter also includes an imagined
congratulatory response from the CEO to the product team explaining how
this has helped the business.
I hope you can see that this customer letter variation is very similar
to Amazon's imagined press release, and it is intended to drive much
the same type of thinking. A press release version includes a customer
quote as well.
I like this customer letter variation even better than the press release
style for a couple of reasons. First, the press release format is a bit
dated. The press release doesn't play the role it used to in our
industry, so it's not something that everyone is familiar with. Second,
I think the customer letter does an even better job of creating empathy
for the customer's current pain and more clearly emphasizes to the team
how their efforts can help the lives of these customers.
I will also admit that I love actual customer letters. I find them to be extremely
motivating. And it's worth noting that even when a customer letter is
critical of the product, it helps the team to understand the problem
viscerally, and  they often
feel compelled to find a way to help.


When embarking on a somewhat larger effort, there may in fact be
multiple reasons, several customer problems to be solved, or business
objectives to be tackled.


For smaller and more typically sized efforts, the opportunity assessment
is usually sufficient. But when embarking on a somewhat larger effort,
there may in fact be multiple reasons, several customer problems to be
solved, or business objectives to be tackled. To communicate the value
effectively, it may take more than the four questions listed in the
previous chapter.
A typical example of an effort of this size would be a redesign. There
are likely several objectives in the redesign, and maybe it is intended
to both improve the experience for current customers and perform better
for new customers.
One of my favorite technology-powered product companies is Amazon. They
have consistently innovated---including several truly disruptive
innovations---and have shown they can continue to do this at scale. In
my view, there are many reasons for this ongoing product success, from
leadership, to talent, to culture, and especially to their sincere
passion for taking care of customers. But there are a few techniques
that are central to how Amazon builds product, and one of them is
referred to as the working backward process, where you start the
effort with a pretend press release.


[CHAPTER 36][Customer Letter Technique]


Everything we do is, of course, intended to benefit our own company in
some way or we wouldn't do it. But we want to keep the focus on our
customers, and this question will clearly articulate the problem we want
to solve for our customers. We occasionally do something to help
internal users, so if that's the case we can call that out here. But
even then, we try to tie it back to the benefits to our end customers.
Target Market
So much product work fails because it tries to please everyone and ends
up pleasing no one. This question is intended to make it very clear to
the product team who the primary intended beneficiary of this work is.
Normally, this is a particular type of user or customer. It might be
described as a user or customer persona, a specific target market, or a
specific job to be done.
There are other factors you may want to consider when assessing an
opportunity, depending on the nature of the opportunity, but I consider
these four questions to be the bare minimum. You need to ensure that
every member of your product team knows and understands the answers to
these four questions before you jump into your product discovery work.
Answering these questions is the responsibility of the product manager,
and it normally takes a few minutes to prepare these answers. But then
the product manager needs to share them with her product team and with
key stakeholders to ensure you are on the same page.
One important caveat: Sometimes the CEO or another senior leader will explain that there's
something beyond normal product work that needs to be done. Realize that
there are sometimes strategic reasons for doing specific product work,
such as support a partnership. If it happens a lot, then that's a
different issue, but it's usually infrequent. If that's the case,
don't stress over  it.
Just give the team as much context as you can---these four questions may
still be relevant.


We want to keep the focus on our customers.


Business Objective
The first question should map to one or more of your team's assigned
objectives. For example, if you've been asked to focus on the problem
of growth, to reduce the time it takes for a new customer to onboard, or
to reduce the percentage of customers that churn each month, then we
want to be clear that this work will address at least one of our
assigned problems.
Key Results
We want to know at the outset what the measure of success is. For example, if we're trying to
reduce churn, would a 1 percent improvement be considered excellent or
would be it be considered a waste of time? The second question should
map to at least one of the key results assigned to our product team.
Customer Problem


[What business objective is this work intended to address?
(Objective)]
[How will you know if you've succeeded? (Key
results)]
[What problem will this solve for our customers? (Customer
problem)]
[What type of customer are we focused on? (Target
market)]


An opportunity assessment is an extremely simple technique but can save
you a lot of time and grief.
The idea is to answer four key questions about the discovery work you
are about to undertake:


[CHAPTER 35][Opportunity Assessment Technique]


This problem famously applies to startup founders. Founders will often
stew on a potential solution for month, if not years, before they get
the funding and the nerve to pursue it.
But one of the most important lessons in our industry is to fall in love with the
problem, not the solution.
Why is this so important? Because, more often than not, our initial
solutions don't solve the problem---at least not in a way that can
power a successful business. It usually takes trying out several
different approaches to a solution before we find one that solves the
underlying problem.
This is another reason why typical product roadmaps are so problematic.
They're lists of features and projects where each feature or project is
a possible solution. Somebody believes that feature will solve the
problem or it wouldn't be on the roadmap, but it's all too possible
they are wrong. It's not their fault---there's just no way to know at
the stage it's put on the roadmap.
However, there very likely is a legitimate problem behind that potential
solution, and it's our job in the product organization to tease out the
underlying problem and ensure that whatever solution we deliver solves
that underlying problem.
A small amount of time up front framing the problem to be solved---and
communicating this framing---can make a dramatic difference in the
results.


More often than not, our initial solutions don't solve the
problem---at least not in a way that can power a successful business.


Problems versus Solutions
There is an underlying theme you'll see in all framing techniques, and
the reason is that it's just human nature for people to think and talk
in terms of solutions rather than the underlying problems. This
applies especially to users and customers but also applies to
stakeholders in our business, other company execs, and if we're honest
with ourselves, it very often applies to us as well.


Note that these techniques are not mutually exclusive. You may find it
useful to do both an opportunity assessment and a customer letter, for
example.


[An opportunity assessment is designed for the vast majority of
product work, which ranges from a simple optimization to a feature
to a medium-sized project.]
[A customer letter is designed for larger projects or initiatives
that often have multiple goals and a more complicated desired
outcome.]
[A startup canvas for those times you're creating an entirely new
product line or a new business.]


For many things, we won't have concerns along these dimensions, but,
when we do, it's something that we must tackle aggressively.
If the product manager, designer, and tech lead do not feel there's a
significant risk in any of these areas, then normally we would just
proceed to delivery---fully realizing there's a chance the team will
occasionally be proved wrong. This, however, is preferable to the
alternative of having the team be extremely conservative and test every
assumption.
We like to use our discovery time and validation techniques for those
situations in which we know there's a significant risk, or where
members of the team disagree.
There are many ways to assess an opportunity. Some companies require
significant rigor and analysis, and others just leave it to the product
team's judgment.
In this section, I describe three of my favorite techniques, each for different-sized
efforts:


[Financial risk---can we afford this
solution?]
[Business development risk---does this solution work for our
partners?]
[Marketing risk---is this solution consistent with our
brand?]
[Sales risk---is this solution something our sales staff is equipped
to sell?]
[Legal risk---is this something we can do from a legal or compliance
perspective?]
[Ethical risk---is this solution something we should
do?]


Two examples I often find are teams that immediately proceed to tackling
technology risks---especially performance or scale---and teams that zero
in on usability risks. They know this change involves a complex
workflow, and they're nervous about that, so they want to dive in right
there.
Those are both legitimate risks, but in my experience, they are
generally the easier risks to tackle.
We must also consider value risk---do the customers want this
particular problem solved and is our proposed solution good enough to
get people to switch from what they have now?
And then there's the often-messy business risk, where we have to make sure that the solution
we come up with in discovery works for the different parts of our
company. Here are some common examples of that:


Ensure the team is all on the same page in terms of clarity of purpose
and alignment.


[The first is to ensure the team is all on the same page in terms of
clarity of purpose and alignment. In particular, we need to agree on
the business objective we're focused on, the specific problem we
are intending to solve for our customers, which user or customers
you're solving that problem for, and how you will know if you've
succeeded. These should align directly to your product team's
objectives and key results.]
[The second purpose is to identify the big risks that will need to
be tackled during the discovery work. I find that most teams tend to
gravitate toward a particular type of risk that they are most
comfortable with.]


[Discovery Framing Techniques]
Overview
Much of our product discovery work doesn't require a lot of framing or
planning. We need to come up with a solution to a particular problem,
and often this is straightforward, and we can proceed directly to
delivery work.
But for many efforts, this is decidedly not the case, and some framing
and true problem solving becomes critically important. Big
projects---and, especially, initiatives (projects spanning multiple
teams)---are common examples.
In this section, I consider how we frame our discovery work to ensure
alignment and to identify key risks.
There are really two goals here:


So, as you can see, we need quite a range of techniques. Some of the
techniques are quantitative, and some are qualitative. Some of the
techniques are designed to collect proof (or at least statistically
significant results), and some are designed to collect evidence. All
are designed to help us learn quickly.
To be clear, I am sharing the techniques here that I believe are
essential for any modern product team. Over the course of a year or two,
you will probably use each of the techniques at least several times.
There are, as you might imagine, many other useful techniques based on
specific types of products or situations, and new techniques are always
emerging. But, these are your go-to techniques.


I am sharing the techniques here that I believe are essential for any
modern product team.


There is no one perfect taxonomy for discovery techniques because
several of the techniques are helpful for multiple different situations.
Regardless, following are the key techniques in the framework that I
personally use and find helpful.
Discovery Framing Techniques
Framing techniques help us to quickly identify the underlying issues
that must be tackled during product discovery. If we're handed a
potential solution, we need to clarify the underlying problem to be
solved. We need to tease out the risks and determine where it makes
sense to focus our time. We also need to ensure that we understand how
our work fits in with the work of other teams.
Discovery Planning Techniques
There are a few techniques that are useful throughout the product
discovery effort and help with identifying the bigger challenges and
planning how you'll attack this work. We'll discuss these here.
Discovery Ideation Techniques
There are, of course, any number of ways to come up with ideas. But some sources are better
than others in their potential for keeping us focused on the most
important problems. Ideation techniques are designed to provide the
product team with a wealth of promising solutions aimed at the problems
we're focused on now.
Discovery Prototyping Techniques
Our go-to tool for product discovery is typically a prototype. We'll
discuss the four main types of prototypes and highlight what each type
is best suited for.
Discovery Testing Techniques
Product discovery is mostly about quickly trying out an idea. We are
essentially trying to separate the good ideas from the bad. Here we are
defining a good idea as one that solves the underlying problem in a way
that customers will buy, they can figure out how to use, we have the
time and skills and technology on the team to build, and that works for
the various aspects of our business.
It's important to recognize that many ideas don't have that much risk
associated with them. They may be very straightforward. Or they might
just have one area that's a risk, such as our legal department's
concern about a potential privacy issue.
Occasionally, however, we need to tackle much tougher problems, and we
may in fact have significant risks in most or even all these areas.
So, the way to think about discovery is that we only validate what we
need to, and then we pick the right technique based on the particular
situation.
Testing Feasibility
These techniques are designed for the engineers to address areas where
they identify concerns. The solution being tested might require some
piece of technology that the team has no experience with. There may be significant scale
or performance challenges. Or there might be third-party components that
need to be evaluated.
Testing Usability
These techniques are designed for the product designers to address areas
where they have identified concerns. Many of our products have complex
workflows and the designers need to ensure their interaction designs
make sense to the user and potential sources of confusion are identified
and pre-empted.
Testing Value
Much of our time in product discovery is spent validating value or
working to increase the perceived value. If it's a new product, we need
to ensure that customers will buy it, at the price we need to charge,
and that they'll switch from whatever they're using today. If it's an
existing product, and we are improving that product (such as with a new
feature or a new design), where the customer has already bought the
product, we need to ensure the customers will choose to use the new
feature or new design.
Testing Business Viability
Sadly, it's not enough to create a product or solution that our
customers love, that is usable, and that our engineers can deliver. The
product also must work for our business. This is what it means to be
viable. This means that we can afford the cost of building and
provisioning the product and the costs to market and sell the product.
It needs to be something our sales force is capable of selling. It means
that the solution needs to also work for our business development
partners. It needs to work for our legal colleagues. It needs to be
consistent with our company's brand promise. These techniques are about
validating these types of risks.
Transformation Techniques
When working to migrate your organization from working the way you do today to working
the way you believe you need to, there are a set of techniques that have
proved to be helpful for transforming how you work.


[CHAPTER 34][Discovery Techniques Overview]


Also, realize that  many
iterations never make it beyond just you, your designer, and your tech
lead. The very act of creating a prototype often exposes problems that
cause you to change your mind. As a rule of thumb, an iteration in
discovery should be at least an order of magnitude less time and
effort than an iteration in delivery.


To set your expectations, teams competent in modern discovery
techniques can generally test on the order of 10--20 iterations per
week.


You may think this is a matter of doing something illegal, but in the
vast majority of cases where ethics is an issue, it's not usually a
matter of law. Rather, just because we have the technology to build
something, and even if it otherwise works to accomplish the specific
business objective, this does not necessarily mean that we should
build it.
More commonly, the  issue
is that our technology and design skills are such that we might come up
with a solution that meets our business objectives (for example, around
engagement, growth, or monetization) but can end up with a side effect
of causing harm to users or the environment.
So, I encourage product teams to consider the ethical implications of
their solutions, too. When a significant ethical risk is identified, see
if you can't find alternative solutions that solve the problem in a way
that doesn't have negative consequences.
I have one final, but critically important, note about raising ethics
issues with senior management. You absolutely need to have a strong
understanding of your business, especially how you make money. You need
to use good judgment and be sensitive in your discussion. You are not
there to try to police the organization but, rather, to identify issues
and bring potential solutions.
Discovery Iterations
Most product teams normally think of an iteration as a delivery
activity. If you release weekly, you think in terms of one-week
iterations.
But we also have the concept of an iteration in discovery. We loosely
define an iteration in discovery as trying out at least one new idea
or approach. It's true that ideas come in all shapes and sizes, and
some are much riskier than others, but the purpose of discovery is to do
this much faster and cheaper than we can do in delivery.
To set your expectations, teams competent in modern discovery techniques
can generally test on the order of 10--20 iterations per week. This
may sound like a lot to you, but you'll soon see that's not so hard at
all with modern discovery techniques.


I encourage product teams to consider the ethical implications of their
solutions, too.


Ethics: Should We Build It?
In general, product discovery is about tackling risks around value,
usability, feasibility, and business viability. However, in some
cases, there's an additional risk: ethics.
I know this is a sensitive topic, and I don't want to sound like I'm
preaching or condescending in the least, but I personally encourage the
teams I work with to also consider the question, "Should we build it?"


[It's about shared learning.]
One of the keys to having a team of missionaries rather than a team
of mercenaries is that the team has learned together. They have seen
the customer's pain together, they have watched together as some
ideas failed and others worked, and they all understand the context
for why this is important and what needs to be done.
Everything that follows is based on these core principles.


[We need to validate the business viability of our ideas during
discovery, not after.]
Similarly, it is absolutely critical to ensure that the solution we
build will meet the needs of our business---before we take the
time and expense to build out that product. Business viability
includes financial
considerations, marketing (both brand and go-to-market
considerations), sales, legal, business development, and senior
executives. Few things destroy morale or confidence in the product
manager more than finding out after a product has been built that
the product manager did not understand some essential aspect of the
business.


[We need to validate the feasibility of our ideas during discovery,
not after.]
If the first time your developers see an idea is at sprint planning,
you have failed. We need to ensure the feasibility before we decide
to build, not after. Not only does this end up saving a lot of
wasted time, but it turns out that getting the engineer's
perspective earlier also tends to improve the solution itself, and
it's critical for shared learning.


[Our goal in discovery is to validate our ideas the fastest,
cheapest way possible.]
Discovery is about the need for speed. This lets us try out many
ideas, and for the promising ideas, try out multiple approaches.
There are many different types of ideas, many different types of
products, and a variety of different risks that we need to address
(value risk, usability risk, feasibility risk, and business risk).
So, we have a wide range of techniques, each suitable to different
situations.


[We must validate our ideas on real users and
customers.]
One of the most common traps in product is to believe that we can
anticipate our customer's actual response to our products. We might
be basing that on actual customer research or on our own
experiences, but in any case, we know today that we must validate
our actual ideas on real users and customers. We need to do this
before we spend the time and expense to build an actual product, and
not after.


To quote Marc Andreessen, "The most important thing is to know what
you can't know," and we can't know in advance which of our ideas
will work with customers and which won't. So, we approach
discovery  with the
mindset that many, if not most, of our ideas won't work out. The
most common reason for this is value, but sometimes the design is
too complicated, and sometimes it would take far too long to build,
and sometimes there turn out to be legal or privacy issues. The
point is we need to be open to solving the underlying problem in
different ways if necessary.


"The most important thing is to know what you can't know."


[We expect that many of our ideas won't work out, and the ones that
do will require several iterations.]


[Functionality, design, and technology are inherently
intertwined.]
In the old waterfall model, the market drove the functionality (aka
the requirements), which drove the design, which drove the
implementation.
Today, we know that the technology drives (and enables) the
functionality as much as the other way around. We know that
technology drives (and enables) design. We know that design drives
(and enables) functionality. You don't have to look further than
your own phone to see numerous examples of both. The point is that
all three of these are completely intertwined. This is the single
biggest reason we push so hard for the product manager, product
designer, and tech lead to sit physically adjacent to each other.


[As hard and important as the engineering is, coming up with a good
user experience is usually even harder, and more critical to
success.]
While every product team has engineers, not every team has the
necessary product design skills, and even when they do, are they
being used the way we need to use them?


[The most important thing is to establish compelling
value.]
It's all hard, but the hardest part of all is creating the
necessary value so that customers ultimately choose to buy or to
use. We can survive for a while with usability issues or performance
issues, but without the core value, we really have nothing. As a
result, this is generally where we'll need to spend most of our
discovery time.


Customers don't know what's possible, and with technology
products, none of us know what we really want until we actually see
it.


[We know we can't count on our customers (or our executives or
stakeholders) to tell us what to build.]
Customers don't know what's possible, and with technology
products, none of us know what we really want until we actually see
it. It's not that customers or our executives are necessarily
wrong; it's just that it's our job to make sure the solution we
deliver solves the underlying problem. This is probably the most
fundamental principle in all of modern product. Historically, in the
vast majority of innovations in our industry, the customers had no
idea that what they now love was even a possibility. This is only
becoming truer with time.


And it's not enough that it's just the product manager's opinion on
these questions. We need to collect evidence.
When it comes to how we do product discovery, there are a set of core
principles that drive how we work. If you understand these, you will
understand not only how to work well today but also how to easily
incorporate new techniques as they emerge in the future.


[Will the customer buy this, or choose to use it? (Value
risk)]
[Can the user figure out how to use it? (Usability
risk)]
[Can we build it? (Feasibility risk)]
[Does this solution work for our business? (Business viability
risk)]


The purpose of product discovery is to address these critical risks:


[CHAPTER 33][Principles of Product Discovery]


You might also recognize this problem in another guise. Many teams get
into a lot of grief with the concept of a minimum viable product (MVP)
because on the one hand we are very motivated to get this out in front
of customers fast to get feedback and learn. And, on the other hand,
when we do get out there fast, people feel like this so-called product
is an embarrassment to the brand and the company. How could we possibly
consider launching this?
In this section, I clarify how strong teams work to meet these dual and
simultaneous objectives of rapid learning in discovery, yet building
stable and solid releases in delivery.
In general, I find that most product teams have a much better sense of
how to accomplish the second goal of delivering solid software than how
to accomplish the first goal of rapid experimentation and discovery.
Continuous delivery is a good example of an advanced delivery technique
I find in teams that understand the importance of a series of small,
incremental changes to a complex system.
Part of what causes confusion is a dilution of what is really meant when
we call something a "product" or "product-quality" or "productized" or
"live in production."
I always try hard to reserve the term product to describe the state at
which we can run a business on it. Specifically, it is scalable and
performant to the degree necessary. It has a strong suite of automated
regression tests. It is instrumented to collect the necessary analytics.
It has been internationalized and localized where appropriate. It is
maintainable. It is consistent with the brand promise. And, most
important, it is something the team can release with confidence.
This is not easy---it's where most of the time goes when our engineers
are building. As such, we try very hard not to waste this effort.
Doing all this work when the product manager isn't even sure this is
the solution the customer wants or needs is a recipe for product failure
and big waste. So, the purpose of product discovery is to make sure we
have some evidence that when we ask the engineers to build a
production-quality product, it won't be a wasted effort. And, this is
why we have so many different techniques in product discovery.
We've got techniques  for
getting a much deeper understanding of our users and customers, and for
validating product ideas both qualitatively and quantitatively. And, in
fact, most of the techniques don't require the developer's time (which
is important, because we appreciate how much time and effort needs to go
into creating production-quality software in delivery).
Much of the key to effective product discovery is getting access to our
customers without trying to push our quick experiments into production.
If you are an early stage startup and you have no customers, then of
course this is not really an issue (and it's very likely premature to
even be creating production-quality software).
But, for most of us, we have real customers and real revenue, so we do
have to care about this. Later in this section, we'll talk about the
techniques that allow for rapid experimentation in a responsible way in larger, enterprise
companies.
But here's the key. If you want to discover great products, it really
is essential that you get your ideas in front of real users and
customers early and often.
If you want to deliver great products, you want to use best practices
for engineering and try not to override the engineers' concerns.


We need to learn fast, yet also release with confidence.


Overview
Most of us are working on solving some pretty hard problems, and it
usually ends up taking some fairly complex systems to power these
solutions. For most teams, there are two very significant challenges to
tackle.
First, discovering in detail what the customer solution needs to be.
That includes everything from making sure there are enough customers
that even need this solution (the demand) and then coming up with a
solution that works for our customers and for our business.
Even harder, we need to make sure we come up with a single solution
that works for many customers, and not a series of specials. To do
this, we need to be able to test out many ideas, and we need to do this
quickly and inexpensively.
Second, we need to ensure we deliver a robust and scalable
implementation that our customers can depend on for consistently
reliable value. Your team needs to be able to release with confidence.
While you'll never have 100 percent confidence, you should not have to
release and pray.
So, we need to learn fast, yet also release with confidence.
It's understandable that many people might naturally view these two
difficult goals as at odds with each other. We are in a big hurry to
push something out to learn what works and what doesn't. Yet, we don't
want to release something that's not ready for prime time and risk
hurting our customers and damaging our brand.
I spend a lot of my time visiting with product teams. I have on occasion
been called out for pushing hard one minute for the team to be much more
aggressive in getting out to customers and getting early feedback on
their ideas, and then just minutes later pushing that same team hard not
to  compromise their
standards on releasing scalable, fault-tolerant, reliable,
high-performance, secure software.


[Product Discovery]


We've explored product teams in Part Two, and we
described how to decide what each team needs to focus on in Part
Three. In Part Four, I explain how product
teams do their job. We'll work through the techniques, activities, and
best practices used to repeatedly discover and deliver successful
products.
Even though this part is titled "The Right Process," I hope you'll soon
realize that the right process is not any single process. Rather,
it's more accurately described as a combination of techniques, mindset,
and culture.
I mostly emphasize discovery techniques, as our focus is on product
managers, and that is their primary responsibility.
The bulk of the product manager's time needs to be focused on working
with her product team, with her key stakeholders, and with her customers
to discover solutions that her customers love and that work for the
business.
Keep in mind, however, that the product manager and product designer do need to ensure
that they're available to answer questions from the engineers that
arise during delivery activities. Normally, answering these delivery
questions is on the order of half an hour to an hour of time per day.


[PART IV] [The Right Process]


This is not just a story about applying technology to solve problems;
it's also a story about the power of force of will. With large enterprise companies, it's
never easy to drive substantial change, but this is exactly what strong
product managers figure out how to do.
Alex went on from the BBC
to have a terrific career at several tech and media companies and now is
a product leader in New York.


With large enterprise companies, it's never easy to drive substantial
change, but this is exactly what strong product managers figure out how
to do.


I have to admit I have a soft spot for the BBC. They've been around for
nearly 100 years, but they embraced technology and the Internet
relatively early. I've seen so many amazing product people come out of
the BBC, and many are now spread across Europe and beyond.
Back in 2003---a full four years before the debut of the iPhone---a
young product manager at the BBC, Alex Pressland, had just finished
leading a product effort that enabled the BBC to be one of the first
media companies in the world to syndicate content. Most people at the
BBC had no idea why this was important or even desirable, but Alex
understood that this enabling technology could be used in new and
unanticipated ways to increase the BBC's reach, a major part of the
institution's mission.
Because Alex understood the potential for IP-based syndicated content
technology, she started searching for new and useful ways to put this
technology to use. She began by looking at people in the United Kingdom
who were not being reached by the BBC's conventional broadcast media
(TVs and radios in homes and cars).
One early use she identified was large electronic billboard screens in
many city center venues that were capable of displaying video. But she
observed that these venues were just playing the same thing you could
watch on your television at home, even though the context and audience
was very different.
So, Alex proposed a series of experiments in which she would have
editorial teams assemble specific tailored content suitable for specific
venues and audiences, and then she would measure the audience reach and
engagement.
While this might sound obvious today, at the time this was a very
foreign concept to the BBC's broadcast journalism culture. There was
a long list of obstacles in trying to push the BBC in this direction,
not the least of which was editorial and legal.
Editorial wasn't used to the model in which content would be created
and then delivered in different contexts. This gets to the heart of the
BBC editorial culture and required considerable persuasion to show why
this was a very good thing for both the BBC and for the audience.
Legal wasn't used to distribution via IP-enabled devices. Imagine the
stack of content-licensing agreements that would need to be updated or
renegotiated.
The results of Alex's experiments and early successes, however, gave
her the confidence to propose to BBC leadership a new product vision and
strategy which she called "BBC Out of Home."
It's important to note that she did this as an individual contributor
product manager.
This work ended up fueling a dramatic shift at the BBC---from broadcast
content to content distribution---and this work dramatically affected
reach and soon became the basis for BBC's mobile efforts. Today, more
than 50 million people around the world depend on BBC's mobile
offerings every week.


[CHAPTER 32][Profile: Alex Pressland of the BBC]


If your company is midsize to large, then it's normal to have product
marketing play the role of evangelist with your customers and your sales
force. You still may be called on to help out on the big deals and
partnerships, but you'll need to focus your evangelism on your team
because the best thing you
can do for your customers is to provide them with a great product.


[Spend time with your team. If you're not spending significant
face time with your designer and every engineer on your team, then
they can't see the enthusiasm in your eyes. If your team is not
co-located, you'll need to make a special effort to travel there
and do this at least every couple months. Spending some personal
time with every last person on the team pays off big in their level
of motivation and, as a result, in the velocity of the team. It's
worth your time.]


[Learn to show some enthusiasm. Assuming you're genuinely
excited, it's amazing to me how many product managers are so bad or
so uncomfortable  at
showing enthusiasm. This matters---a lot. Absolutely be sincere,
but let people see you're genuinely excited. Enthusiasm really is
contagious.]


Absolutely be sincere, but let people see you're genuinely
excited. Enthusiasm really is contagious.


[Be genuinely excited. If you're not excited about your
product, you should probably fix that---either by changing what you
work on or by changing your role.]


[Do your homework. Your team and your stakeholders will all be
much more likely to follow you if they believe you know what you're
talking about. Be the undisputed expert on your users and customers.
And be the undisputed expert on your market, including your
competitors and the relevant trends.]


[Learn how to give a great demo. This is an especially important
skill to use with customers and key execs. We're not trying to
teach them how to operate the product, and we're not trying to do a
user test on them. We're trying to show them the value of what
we're building. A demo is not training, and it's not a test. It's
a persuasive tool. Get really, really good at
it.]


[Share credit generously. Make sure the team views it as their
product, not just your product. However, when things don't go
well, step forward and take responsibility for the miss and show the
team you're learning from the mistakes as well. They'll respect
you for it.]


[Share learnings generously. After every user test or customer
visit, share your learnings---not just the things that went well,
but share the problems, too. Give your team the information they
need to help come up with the solution.]


[Share the vision. Make sure you have a very clear understanding
of your product vision, product strategy, and product principles.
Show how your work contributes to this vision and is true to the
principles.]


[Share the pain. Show the team the customer pain you are addressing. This is why I
love to bring engineers along for customer visits and meetings. For
many people, they have to see (or experience) the pain themselves to
get it.]


[Use a prototype. For many people, it's way too hard to see the
forest through the trees. When all you have is a bunch of user
stories, it can be difficult to see the big picture and how things
hang together (or even if they hang together). A prototype lets
them clearly see the forest and the
trees.]


Product evangelism is, as Guy Kawasaki put it years ago, "selling the
dream." It's helping people imagine the future and inspiring them to
help create that future.
If you're a startup founder, a CEO, or a head of product, this is a
very big part of your job, and you'll have a hard time assembling a
strong team if you don't get good at it.
If you're a product manager---especially at a large company---and
you're not good at evangelism, there's a very strong chance that your
product efforts will get derailed before they see the light of day. And
even if product does manage to ship, it will likely go the way of
thousands of other large company efforts and wither on the vine.
We've talked about how important it is to have a team of missionaries,
not mercenaries, and evangelism is a key responsibility to make this
happen. The responsibility for this falls primarily on the product
manager.
There are several techniques to help communicate the value of what
you're proposing to your team, colleagues, stakeholders, executives,
and investors. Here are my top-10 pieces of advice for product managers
to sell the dream:


[CHAPTER 31][Product Evangelism]


In summary, when using
OKRs at scale, there's a larger burden on leadership and management to
ensure that the organization is truly aligned, that each and every
product team understands how they fit into the mix, and what they are
there to contribute.


[In many enterprise scale organizations, there are essentially
multiple business units, and in this case, we would expect that
there are corporate level OKRs, but there would also be business
unit\endash level OKRs, and the product teams would roll up into
those.]


When using OKRs at scale, there's a larger burden on leadership
and management to ensure that the organization is truly aligned,
that each and every product team understands how they fit into the
mix, and that they are there to contribute.


[The larger the organization, the longer the
list  of high-integrity
commitments that are needed, and the more actively they need to be
managed and tracked. Delivery managers play a key role in tracking
and managing these dependencies and our
commitments.]


[At scale, it's much harder to know what product teams are working
on which objectives and the progress they are making. There are now
a variety of online tools that help organizations make the
objectives transparent to the organization. But even with these
tools, we lean on management to help connect the dots between
teams.]


[Once you have your objectives, there is a very critical
reconciliation process in which the leadership team looks at the
proposed key results from the product teams and identifies gaps and
then looks to what might be adjusted to cover those gaps (for
example, enlisting the help of additional teams or reviewing the
priority of the work).]


[Moreover, at scale, it is very common to have some significant
number of product teams that are there in support of the other
product teams. These are often called platform product teams, or
shared services product teams. They are very high leverage, but
they are a little different in that they generally don't directly
serve customers. They serve customers indirectly, usually through
the higher-level, solution-focused product teams. These platform
teams will get requests from most or even all the higher-level
product teams, and they are there to help them succeed. But, again,
leadership will need to help coordinate the objectives for these
teams and make sure we coordinate the dependencies and align the
interests.]


[With startups or small organizations, when everyone essentially
knows what everyone else is doing and why, it's normal for each
product team to propose their objectives and key results. There's
some amount of give and take, and then people get to work. With
larger organizations, product teams need more
help.]
The first help they need is a very clear understanding of the organization-level
objectives. Let's say that the top two objectives for the company
are to improve customer lifetime value and expand globally. Let's
also say you have on the order of 25 product teams. All the product
teams likely have thoughts on both of these organizational
objectives, but, clearly, the company will need to be smart about
which teams pursue each objective. Some teams might focus on only
one, others might contribute to both, and yet other teams may be
tackling critical work beyond those two objectives.
Leadership (especially the head of product, head of technology, and
head of design) will need to discuss the company objectives and
which teams are best suited to pursue each objective.


The OKR system is very scalable. I would argue some sort of tool for
managing and aligning work is critical to scaling effectively, but it's
also true that many companies do struggle with scaling their use of
OKRs.
In this chapter, I shine a light on what needs to change as you use the
OKR system at scale. Remember that I'm just talking about the product
and technology organization here (product management, user experience
design, and engineering), and while you can use the techniques I'm
about to describe at any scale, I am focused here on growth stage or
enterprise organizations.


[CHAPTER 30][Product Objectives @ Scale]


Overview
We have thus far discussed product vision, strategy, and business
objectives. In truth, as an early stage startup, you can survive without
any of these for a while. It's amazing how far you can get by focusing
on meeting the needs of some early customers.
However, the need for this vision and business-objective context becomes
truly serious at scale.
Keeping a small number of teams and their engineers doing useful stuff
is not very hard, but getting good results out of a medium---or,
especially, large---organization can be truly challenging.
Realize also that by the time the company has scaled, the original
co-founders may have moved on, so there may very well be a void. Teams
need this context. It is nearly impossible for them to make good
decisions and do good work without it.
The main ways these issues show up is diminished morale, lack of
innovation, and reduced velocity.


What all too often happens in this case is that the people on the
product teams are conflicted as to where they should be spending their
time. This results in confusion, frustration, and disappointing results
from leadership and individual contributors alike.
But this is easily avoided.
If you deploy OKRs for your product organization, the key is to focus
your OKRs at the product team level.
This means don't let functional team or individual person OKRs confuse
the issue.
Focus the attention of the
individuals on their product team objectives. If different functional
organizations (such as design, engineering, or quality assurance) have
larger objectives (such as responsive design, technical debt, and test
automation), they should be discussed and prioritized at the leadership
team level along with the other business objectives, and then
incorporated into the relevant product team's objectives.
Note that it's not a problem for managers of the functional areas to
have individual objectives relating to their organization. This is
because these people aren't conflicted, as they're not normally
serving on a product team.
For example, the head of UX design might be responsible for a strategy
for migrating to a responsive design; the head of engineering might be
responsible for delivering a strategy around managing technical debt;
the head of product management might be responsible for delivering a
product vision; or the head of QA might be responsible for selecting a
test automation tool.
It's also not normally a big problem if individual contributors (such
as a particular engineer, designer, or product manager) has a small
number of personal growth-related objectives (such as improving their
knowledge of a particular technology). This assumes the individual
isn't committing to a burden that will interfere with their ability to
contribute their part to their product team, which of course is their
primary responsibility.
The key is that the cascading of OKRs in a product organization needs to
be up from the cross-functional product teams to the company or
business-unit level.


If you deploy OKRs for your product organization, the key is to focus
your OKRs at the product team level.