Tag Archives: programming

Why Pilot Projects Can Be Catastrophic

Many companies think they are staying nimble during product innovation by setting up pilot projects to validate concepts before they’re rolled out at scale. But pilots aren’t the answer, either, at least not on their own.

Once something gets anointed as a “pilot,” it’s no longer an option—it’s the destination. There are typically no graceful ways to kill a pilot, and even course corrections are too hard to make. Systems such as software have all been done at the production level, with the assumption that the pilot will work and will need to be quickly rolled out at scale. Changes are seen as a sign of defeat, and digging into production code can be complicated.

Besides, problems at the pilot stage often get hidden. A pilot is very public, and some senior people have a strong interest in success, so they may work behind the scenes and use their connections to make it successful.

I once watched a client be all over a pilot in a single state, so thoroughly covering the pilot with senior-management attention that the client learned little before initiating a national roll-out. The executives knew what they were doing, but they couldn’t help themselves. They were so invested in the success of the pilot.

The solution is to rephrase the issue. There needs to be less planning and more testing. The only way to accomplish that is to defer the pilot stage and stay in the prototyping phase much longer than most companies do.

The difference between a prototype and a pilot is that there’s no possibility or expectation that a prototype will turn into the final version of the product or service. Prototypes are just tests to explore key questions, such as whether the technology will work, whether the product concept will meet customer needs or whether customers will prefer it over the competitive alternatives.

The early prototypes should be all chewing gum and baling wire. They shouldn’t have hardened processes or the people required to go live. Yet they should provide real insight that informs further development. Each stage of prototyping should minimize costs and maximize flexibility. To borrow a term from computer programming, new products and services should be explored using “late binding”; they should take final form as late as possible, based on the most up-to-date learnings that can be generated.

Pixar has made a religion of prototyping through what the company calls “story reels.” The company doesn’t just write a script; it creates storyboards that provide a sort of comic-book version of a prospective movie, then adds dialogue and music. The story reels cost almost nothing, compared with the fully animated versions of Pixar’s movies, yet provide a great sense of how a story will flow and allow for problems to be spotted. The story reels can also be changed easily.

Here’s a fascinating video in which the creators of Toy Story describe their storyboarding process:

https://www.youtube.com/watch?v=QOeaC8kcxH0#action=share

Every regular review of progress on the prototypes should begin with a demo, much like what Pixar does with its storyboards. My old friend Gordon Bell, who designed the first minicomputer while at Digital Equipment, likes to say that “one demo is worth a thousand pages of a business plan,” and that notion applies to every stage of prototyping. It’s easy to get lost in talk of value propositions, competencies and market segments. A demo makes an idea tangible in a way that no business plan ever will.

At Charles Schwab, in the lead-in to the company’s great, early successes with the Internet, executives talked about a hamster on a wheel. Schwab would test potential services by having people working behind the scenes answering questions, looking up information, and so on, running just as fast as their little (metaphorical) legs could go. Anything that didn’t work or didn’t resonate with customers was easily set aside. Only once Schwab had a sense of what customers truly wanted would it start building the capabilities into software.

Prototypes and demos are part of what has made Apple products so successful. Steve Jobs always used prototypes of products to drive his thinking. For example, early in the process of figuring out the right screen size for the iPad, Jobs had Jonathan Ive make 20 models in slightly varying sizes. These were laid out on a table in Ive’s design studio, and the two men and their fellow designers would play with the models. “That’s how we nailed what the screen size was,” Ive told Walter Isaacson in his biography of Jobs.

Admittedly, it helps when you have a genius like Jobs playing with the devices, but even he couldn’t envision everything. He needed many alternatives of something tangible. As Isaacson quoted him as saying, “You have to show me some stuff, and I’ll know it when I see it.”

If Steve Jobs thought it was critical to prototype, shouldn’t you?

The Science (and Art) of Data, Part 1

Most insurers are inundated with data and have difficulty figuring out what to do with all of it. The key is not just having more data, more number-crunching analysts and more theoretical models, but instead identifying the right data. The best way to do this is via business-savvy analysts who can ask the right strategic questions and develop smart models that combine insights from raw data, behavioral science and unstructured data (from the web, emails, call center recordings, video footage, social media sites, economic reports and so on). In essence, business intelligence needs to transcend data, structure and process and be not just a precise science but also a well-integrated art.

The practitioners of this art are an emerging (and rare) breed: data scientists. A data scientist has extensive and well-integrated insights into human behavior, finance, economics, technology and, of course, sophisticated analytics. As if finding this combination of skills wasn’t difficult enough, a data scientist also needs to have strong communication skills. First and foremost, he must ask the right questions of people and about things to extract the insights that provide leads for where to dig, and then present the resulting insights in a manner that makes sense to a variety of key business audiences. Accordingly, if an organization can find a good data scientist, then it can gain insights that positively shape its strategy and tactics – and gain them more quickly than less-well-prepared competitors.

What it takes to be an effective data scientist

The following table highlights the five key competencies and related skills of a qualified data scientist.

Competencies

Key Skills

Business Impact

1. Business or Domain Expertise

   Deep understanding of:

  • Industry domain, including macro-economic effects and cycles, and key drivers;
  • All aspects of the business (marketing, sales, distribution, operations, pricing, products, finance, risk, etc.).
  • Help determine which questions need answering to make the most appropriate decisions;
  • Effectively articulate insights to help business leadership answer relevant questions in a timely manner.

2. Statistics

  • Expertise in statistical techniques (e.g., regression analysis, cluster analysis and optimization) and the tools and languages used to run the analysis (e.g., SAS or R);
  • Identification and application of relevant statistical techniques for addressing different problems;
  • Mathematical and strategic interpretation of results.
  • Generate insights in such a way that the businesses can clearly understand the quantifiable value;
  • Enable the business to make clear trade-offs between and among choices, with a reasonable view into the most likely outcomes of each.

3. Programming

  • Background in computer science and comfortable in programming in a variety of languages, including Java, Python, C++ or C#;
  • Ability to determine the appropriate software packages or modules to run, and how easily they can be modified.
  • Build a forward-looking perspective on trends, using constantly evolving new computational techniques to solve increasingly complex business problems (e.g., machine learning, natural language processing, graph/social network analysis, neural nets, and simulation modelling);
  • Ability to discern what can be built, bought or obtained free from open source and determine business implications of each.

4. Database Technology Expertise

  Thorough understanding of:

  • External and internal data sources;
  • Data gathering, storing and retrieval methods (Extract-Transform-Load);
  • Accessing data from external sources (through screen scraping and data transfer protocols);
  • Manipulating large big data stores (like Hadoop, Hive, Mahoot and a wide range of emerging big data technologies).
  • Combine the disparate data sources to generate very unique market, industry and customer insights;
  • Understand emerging latent customer needs and provide inputs for high-impact offerings and services;
  • Develop insightful, meaningful connections with customers based on a deep understanding of their needs and wants.

5. Visualization and Communications Expertise

Comfort with visual art and design to:

  • Turn statistical and computational analysis into user-friendly graphs, charts and animation;
  • Create insightful data visualizations (e.g., motion charts, word maps) that highlight trends that may otherwise go unnoticed;
  • Use visual media to deliver key message (e.g., reports, screens – from mobile screens to laptop/desktop screens to HD large visualization walls, interactive programs and, perhaps soon, augmented reality glasses).
  • Enable those who aren’t professional data analysts to effectively interpret data;
  • Engage with senior management by speaking their language and translating data-driven insights into decisions and actions;
  • Develop powerful, convincing messages for key stakeholders that positively influence their course of action.

While it may seem unrealistic to find a single individual with all the skills we've listed, there are some data scientists who do, in fact, fit the profile. They may not be equally skilled in all areas but often have the ability to round out their skills over time. They typically tend to be in high-tech sectors, where they have had the opportunities to develop these abilities as a matter of necessity.

However, because of the increasing demand for data scientists and their scarcity, insurers (and companies in other industries) should consider if they want to build, rent or buy them. Although buying or renting capabilities can be viable options – and do offer the promise of immediate benefits – we believe that building a data science function is the best long-term approach. Moreover, and as we will address in our next post, in light of the shortage of data scientists, a viable approach is creating a data science office of individuals who collectively possess the core competencies of the ideal data scientist.