Note 35 | Rather Than Hiding Innovation, Draw a Sandbox for It

These are Robin Shen’s original reading notes and commentary, written in his own words after reading Eric Ries’s The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. They are not the original book, excerpts from it, or an official publication. Methods and concepts from the book are attributed to Eric Ries; other sources are identified where stated. For reference only; not investment advice.

English translation of Robin’s Chinese reading notes and commentary. Passages are paraphrased, not verified verbatim quotations from the original English edition.

Illustration for the reading notes on The Lean Startup: Rather Than Hiding Innovation, Draw a Sandbox for It

An innovation team inside a company is often said to need protection, lest it be strangled by the parent company’s bureaucracy. In Chapter 12, Eric Ries turns the view around: what may truly need protection is the parent company, and the best form of protection is not to hide the innovation, but to put it into a sandbox that is open, transparent, and bounded in scope.

He first tells a consulting case at an unnamed large company. Senior executives met to decide the features of the next version. The company believed it valued data and had also run a pricing experiment, yet the meeting could not reach a consensus on what the data meant. The material only showed total sales by quarter and by customer group at different price points; no one could say clearly which customers had taken part in the experiment. Different teams had changed different parts at different times over several months, and the designer had already transferred. As Ries diagnoses it, the problems included vanity metrics, a cycle that was too long, batches that were too large, unclear hypotheses, a design that was not rigorous, and no one being accountable. The result was that each department picked the reading of the numbers that favored itself, and in the end it came down to who sounded more reasonable. Even running an experiment became a term of abuse.

Ries also admits that reasonable worries sat behind the quarrel. The company had two kinds of customer: enterprise customers contributed most of the revenue but growth was slowing; individual consumers had large potential but no progress. Existing pricing set a high list price and then gave large customers deep discounts, which was too expensive for individuals. The consumer team wanted to try a lower price; the enterprise team worried about cannibalizing revenue, or about large customers discovering that individuals were in fact buying more cheaply. Ries points out that there is more than one solution: for example, tiered features like seats on a flight, or launching different products under different brands. But the company delayed again and again for fear of endangering the existing business. He stresses that a mature company does have something to protect. A decline in core revenue may cost people their jobs.

Is hiding the innovation team feasible, then? Ries’s view is that it is often the wrong approach. He mentions IBM’s personal-computer business in Boca Raton, Florida, completely separated from the main business, and regards such a success as hard to copy, more like a cautionary tale: managers left in the dark feel betrayed, and office politics grow. Ries comments that although IBM opened a new market, it failed to maintain the original culture of innovation. He also cites the dilemma described by Clayton Christensen: a company that will not experiment flexibly may see profits climb year after year, and then suddenly collapse.

The alternative Ries proposes is an innovation sandbox. I put it in my own words: any team can start a genuine split test, but only of a portion of the product inside the sandbox, or of a particular customer group or region; the team is responsible for the experiment from beginning to end; the experiment has a time limit, and the share of affected users also has a limit; every test uses the same standard report, containing five to ten key metrics; the team has to watch the metrics and customer responses, and stop if a serious problem appears. The sandbox at the company above at first covered only the pricing page.

My reading is that the power of the sandbox lies in limiting the scope and at the same time opening the method. Key metrics either change or they do not, so success and failure are easy to judge. Everyone looks at the same set of metrics each time, and the whole company gradually builds a shared language. Even people who want to object first have to understand the metrics and the learning goal before they can take part in the discussion. Rather than deciding who is right by political struggle, let a small number of real customers answer.

I think this is also a reminder for investors: when a portfolio company begins a second curve of growth, do not first rush to ask whether the new team is independent enough. Ask whether the worries of the old business have been laid out and discussed. The conclusion is that transparency is not a constraint on innovation. It is what lets innovation be trusted. First draw the range of impact small, then let the team work freely inside the boundary.

Three suggestions

Before starting a new experiment, invite the departments that may be affected to write down together the customer group affected, the time limit of the experiment, the upper limit on the share of users, and the conditions for stopping.

List the old business’s worries about the innovation one by one, and for each find a mitigating design, such as tiered features or a separate brand, rather than answering with delay.

Limit the experiment report to five to ten fixed, actionable metrics, publish it in one page to the colleagues involved, and keep success or failure from depending on each person’s own reading.

#TheLeanStartup | Source: Chapter 12, “Innovate”

These are reading notes and reflections based on my personal understanding. I recommend reading them alongside the original book for the full context.

Comments

What do you think? Share your experience or questions.

Loading comments…