Take the world’s biggest media company, with a commanding presence in cable TV, broadband Internet, telephony, and movies. Add a dot-com startup struggling to retool for a new era of Internet and mobile applications. Result: Comcast Silicon Valley Innovation Center, formed in 2011 to align the Philadelphia-based cable provider with the forces of technological disruption. Comcast, founded in 1963 and now 12,000 employees strong, acquired Plaxo, a cloud-based address book, in 2008. Preston Smalley joined two years later.
After reorienting the division toward agile software development and demonstrating the power of lean startup methods, he helped found the Innovation Center. Today, he and his 250 colleagues are cranking out new products at the nexus of mobile, social, and cable networks.
He took time to speak with us about lean startup experimentation.
Tell us about Comcast Silicon Valley Innovation Center and what you do there.
Comcast set up the Innovation Center a few years ago. The company realized that innovation around TV, home automation, and Internet connectivity would be driven by technology and software development. It acquired Plaxo, an online address book, to turn those activities into core competencies, and that became the basis of the Innovation Center’s engineering, product, and design talent. I came here to be the general manager of Plaxo, and that’s where I cut my teeth on how to do lean startup methods. It was clear to Comcast that what we were doing made sense, so they talked me into repositioning my team to work on more Comcast-focused products. Now I run product management, working on mobile apps and next-generation TV applications, sometimes in partnership with other Silicon Valley companies.
How did you learn about the lean startup method?
I went through an experience at Plaxo that really brought me to my knees. At the time I joined, Plaxo was running flat. They’d built an online address book and tried to pivot to being a social network, hoping to compete with Facebook and LinkedIn until it became clear that it was a winner-take-all situation. So we talked to customers, trying to figure out what to do next. A lot of people said, “My address book is a mess. It’s completely out of date. I have different people on my phone and a computer, I have duplicates, and everything
is out of sync. Please help me.”
So we doubled down on our original value proposition and set an audacious goal of building a machine-learning system that would automagically figure out the most up-todate info for all your contacts based on the wisdom of crowds and a host of data sources.
But what we thought would be a six-month project turned into 18 months. When we finally showed it to the same customers, they were like, “It’s nice, but I’m not willing to pay $10 a month for it.”
The company continued to go sideways, and we had wasted most of the team’s time for a year and a half. At around that time, I picked up Eric Ries’ book, The Lean Startup, and it hit me like a ton of bricks: _We never tested whether people would pay for the product!_
We could have tested our solution hypothesis long before we worked on heavy lifting around technology. I could have had a person in India updating peoples’ dress books manually and I would have learned immediately that this isn’t something people are willing to pay for. Many of us have a messy closet, but we don’t stampede the closetorganizing store to get the stuff to fix it. We manage through it.
What’s your personal approach to innovation?
My background is in computer engineering; I spent seven years running design at Ebay, and along the way I got a business degree. For me, innovation happens when I’m seeing overlapping trends, I see a customer problem, then I hear about interesting technology, and I see a business opportunity arise out of that.
Here’s an example. Right now, people want personalization in their entertainment experience.
They want Comcast to know what movies and shows they’re interested in and to make recommendations. But we don’t know who’s watching, because no one wants to log in and out of their TV. Meanwhile, I’m seeing Bluetooth Low Energy come forward as a lightweight way of cookie-ing and tracking people. I’m thinking, if we could layer those two things onto each other, we’d have a platform that told us who was watching what, we’d be able to personalize the experience for them, and that would be worth a lot of money as a business — not only as an experience, but also as an advertising platform. Seeing those
three worlds of business, tech, and the user overlap to spark an innovation.
How do your projects move through the experimental process?
We think about our projects like a VC would. We go through a seed stage where we evaluate a project. If it looks good, we seed fund it, which means we put a couple of people on it for a month or so. If they’re able to prove their hypothesis, we give them “Series A” funding to let them build what we call a minimal testable product, not quite viable, but testable enough that we can learn something from it. If that goes well, we might do a “Series B” in order to build an MVP. All the while we’re looking for an exit. We don’t want to be in the business of having to run all these things, so we look for another team within the company that we can hand off to. We’ll give them a successful MVP that they can continue to scale.
What percentages of your experimental efforts are devoted to research?
Roughly 10 to 15 percent of our effort consists of user interviews and other research, 15 percent pitching, 15 percent concierging, and 55 percent prototyping.
How do you find prospective customers?
It depends on who we’re trying to reach. If we’re bringing people in locally, we’ve had some luck with Craigslist. If we want to deploy something in the field, we have a trial team that was set up to try new set top boxes in the field. They can hand us existing Comcast customers who say, “I’m interested in doing studies.” They work at a much larger scale — like, one third of the Bay Area would get a new set top box — but we have used them in smaller-scale settings. We also recruit through advertising on Facebook. We did that for a
product called BirthdayGram that addressed a market adjacent to Plaxo’s. The idea was to have all a person’s friends record little video messages, and we would stitch them together into a single video and post it on the person’s Facebook wall on their birthday. We thought that our early adopters would be teenagers and college students, and we also wanted to find people who would be ready to put themselves on camera. So we did a Facebook ad campaign for girls who were in the right age bracket and liked to dance.
What role do observation and re-enactment play in your experimental process?
We do in-home ethnographic research. We did some observation around family communication.
We wanted to understand how families, especially families with teenage kids,
communicate. How do they talk about their schedule? How do they get in touch with each other during the day? We followed families around and watched them interact. That led to an MVP app called Family Connect that lets families to share messages and calendars with one another. They can say, “I need to be picked up,” and post a picture of the location.
They can push a button and enter a party line; it calls everyone in the family so
they’re all on a conference call together. That concept came from user observation.
What tips would you offer for successful customer interactions?
Trying to get people in their natural environment where you can see what they do rather than what they say they do. I know a guy who used to work on Tide at Procter & Gamble. Whenever he asked people how they washed their clothes, they always explained how their mother taught them: “I separate the whites and colors and do each load separately.”
But when he watched them, they just dumped it all together. If you didn’t watch them wash their clothes, you wouldn’t realize it, so Procter & Gamble had to make a detergent that worked well in a mixed environment like that. It’s the same problem we had at Plaxo. People say, “I love to have a clean address book,” but they don’t really care.
What form do your pitches usually take? Do you use landing pages, or what?
They start verbally. They could be sketches or screenshots that we’ve mocked up, like a paper prototype. Those are the primary ways until you get into concierge or Wizard of Oz prototypes.
What tokens of customer interest have you collected, such as email addresses or credit card numbers?
One very helpful thing we’ve done is ask people to choose between keeping the product we’d given them or taking a gift card. For instance, in the early stages of the music video app project, we asked, “For $5, which would you rather have? This app that lets you tune to music videos on your TV or a $5 iTunes gift card? It was easier than deciding whether to buy a service for $5 per month, because most teenagers don’t sign up for monthly services;
that’s the parent’s role. Either I’m going to have this service or get this gift card. It gave us a way to determine what the service was worth to a teenager.
Under what circumstances would you build a prototype?
You build a prototype either because you’re not sure how the customer will respond or you’re not sure whether the technology is feasible or how useful it would be. The Bluetooth Low Energy prototype was like that. We wanted to see whether we could tell whether users were in their living room, kitchen, or bedroom, since those locations would imply different things about how they would use their TV. The prototype worked well and now we’re going to try it with customers.
What’s your approach to building prototypes?
We do it three ways. One is to hack together a solution out of other vendors’ products.
For instance, we recently tested a concept for a music video application. People love watching music videos on the TV but it’s hard to look for specific ones. We were trying to place that activity in a group context to see how multiple teenagers would interact together in a jukebox / after-school scenario. So we hooked up an Apple TV with the Vevo video app running on it and brought in 10 teenagers. They had never used that technology, so for all they knew, we had built it.
Sometimes we bring in people to use technology we’ve developed on our own. And sometimes we might deploy a functional product to a limited number of customers, say 20 to 30. For the music video project, we built a TV remote that ran an app similar to Spotify and delivered it in the field. We tried it with 20 teenagers in the Midwest. We realized that they liked it, but they wouldn’t use it enough. The data taught us that if we wanted to go that route, we needed to either go full bore into music or make the music video functionality a feature of a broader remote control setup.
Can you offer tips for a successful prototyping process?
I just had this conversation with my team yesterday. You shouldn’t prototype something that’s obvious or unsurprising, something you know you could do, how it would work, and how the customer would respond. You want to test the riskiest elements of your plan.
Do you use innovation accounting in conjunction with concierging and prototyping?
I’m looking to use it more. We’re keeping track of how much we’ve spent on a given innovation, how many man-weeks we’ve spent on a given project, to understand what our outlay is and weigh it against other opportunities. We’re also looking at the eventual revenue opportunities for the company. That might not be straight revenue that improved retention or improved acquisition of customers.
How do you insulate the enterprise against brand risk from minimum viable products?
We do three things. First, we launch products under a separate brand. BirthdayGram, while it was covered under a broad set of Comcast policies, was positioned as a separate brand and you’d have to dig to understand that it was Comcast. Second, we launch under a brand called Comcast Labs and tell the customer explicitly that these things are experimental technologies that we may get rid of or incorporate, and this is your opportunity to play with them. Third, we have a variety of ways of trialing and a/b testing with customers.
We can do it in small samples, certain regions, special contexts where we can see how it plays out before it gets scaled to tens of millions of customers.
How do you decide which approach to use?
One heuristic is how much brand risk the product poses. The greater the brand risk, the more likely we’ll go with a separate brand. Another is how likely the new product is to be folded into a mainstream product. If it’s likely to go mainstream, we’re more likely to use Comcast Labs or possibly trial it under the proper Comcast brand. The sooner you want to roll out that feature, the closer you are to the Comcast brand, and vice-versa.
Can you tell us about a time when a prototype led to a pivot?
Recently we set up a partnership with Twitter to launch a nationwide social TV service called SEEit. Twitter’s audience likes to discuss TV, yet Twitter is disconnected from how people watch TV. We have many ways to control a TV remotely over the Internet, but it’s separate from the conversation on Twitter. So we worked with Twitter to develop an app that, when people tweet about TV, a button appears that will record the show, tune a TV that’s in front of you, or let you watch on an iPad. That service evolved in a lean-startup way from a concept to an initial prototype, but we pivoted before releasing the MVP about a month ago.
How did you pivot?
Initially the product was focused on cable TV. When users clicked the button, they needed to identify who their TV provider was, and then we worked with the TV provider to create an experience that was tailored to them. Time Warner would power one experience, we’d power another, DirecTV would power another, and so on. As we saw how many clicks it took to go through that process, we recognized that we needed to create a provider-agnostic experience that didn’t ask those questions up front. We also realized that we needed to pull in the programmers who created the content — ABC, NBC, HBO, or whoever — and integrate their apps deeply into the experience as well. Their apps may or may not already be on the user’s device, so we didn’t want to create a barrier at the very beginning of the process. That required us to change our business development posture in terms of how we reached out to TV providers versus programmers.
What did you learn from the experience?
That experience taught me that you need not only great innovators but also great business development. There’s an interesting synergy between my team in Silicon Valley, Comcast’s business development minds on the East coast, and great partners like Twitter. Together, we’ve tackled challenges that nobody else has been able to crack.
Preston Smalley, Executive Director, Product Management