Sections

When Good Is Good Enough: Guiding Business Users On BI Practices

Ina Felsheim

Image_part2-300x200In Part One of this blog series, I talked about changing your IT culture to better support self-service BI and data discovery. Absolutely essential. However, your work is not done!

Self-service BI and data discovery will drive the number of users using the BI solutions to rapidly expand. Yet all of these more casual users will not be well versed in BI and visualization best practices.

When your user base rapidly expands to more casual users, you need to help educate them on what is important. For example, one IT manager told me that his casual BI users were making visualizations with very difficult-to-read charts and customizing color palettes to incredible degrees.

I had a similar experience when I was a technical writer. One of our lead writers was so concerned with readability of every sentence that he was going through the 300+ page manuals (yes, they were printed then) and manually adjusting all of the line breaks and page breaks. (!) Yes, readability was incrementally improved. But now any number of changes–technical capabilities, edits, inserting larger graphics—required re-adjusting all of those manual “optimizations.” The time it took just to do the additional optimization was incredible, much less the maintenance of these optimizations! Meanwhile, the technical writing team was falling behind on new deliverables.

The same scenario applies to your new casual BI users. This new group needs guidance to help them focus on the highest value practices:

  • Customization of color and appearance of visualizations: When is this customization necessary for a management deliverable, versus indulging an OCD tendency? I too have to stop myself from obsessing about the font, line spacing, and that a certain blue is just a bit different than another shade of blue. Yes, these options do matter. But help these casual users determine when that time is well spent.
  • Proper visualizations: When is a spinning 3D pie chart necessary to grab someone’s attention? BI professionals would firmly say “NEVER!” But these casual users do not have a lot of depth on BI best practices. Give them a few simple guidelines as to when “flash” needs to subsume understanding. Consider offering a monthly one-hour Lunch and Learn that shows them how to create impactful, polished visuals. Understanding if their visualizations are going to be viewed casually on the way to a meeting, or dissected at a laptop, also helps determine how much time to spend optimizing a visualization. No, you can’t just mandate that they all read Tufte.
  • Predictive: Provide advanced analytics capabilities like forecasting and regression directly in their casual BI tools. Using these capabilities will really help them wow their audience with substance instead of flash.
  • Feature requests: Make sure you understand the motivation and business value behind some of the casual users’ requests. These casual users are less likely to understand the implications of supporting specific requests across an enterprise, so make sure you are collaborating on use cases and priorities for substantive requests.

By working with your casual BI users on the above points, you will be able to collectively understand when the absolute exact request is critical (and supports good visualization practices), and when it is an “optimization” that may impact productivity. In many cases, “good” is good enough for the fast turnaround of data discovery.

Next week, I’ll wrap this series up with hints on getting your casual users to embrace the “we” not “me” mentality.

Read Part One of this series: Changing The IT Culture For Self-Service BI Success.

Follow me on Twitter: @InaSAP

Comments

Data Analysts And Scientists More Important Than Ever For The Enterprise

Daniel Newman

The business world is now firmly in the age of data. Not that data wasn’t relevant before; it was just nowhere close to the speed and volume that’s available to us today. Businesses are buckling under the deluge of petabytes, exabytes, and zettabytes. Within these bytes lie valuable information on customer behavior, key business insights, and revenue generation. However, all that data is practically useless for businesses without the ability to identify the right data. Plus, if they don’t have the talent and resources to capture the right data, organize it, dissect it, draw actionable insights from it and, finally, deliver those insights in a meaningful way, their data initiatives will fail.

Rise of the CDO

Companies of all sizes can easily find themselves drowning in data generated from websites, landing pages, social streams, emails, text messages, and many other sources. Additionally, there is data in their own repositories. With so much data at their disposal, companies are under mounting pressure to utilize it to generate insights. These insights are critical because they can (and should) drive the overall business strategy and help companies make better business decisions. To leverage the power of data analytics, businesses need more “top-management muscle” specialized in the field of data science. This specialized field has lead to the creation of roles like Chief Data Officer (CDO).

In addition, with more companies undertaking digital transformations, there’s greater impetus for the C-suite to make data-driven decisions. The CDO helps make data-driven decisions and also develops a digital business strategy around those decisions. As data grows at an unstoppable rate, becoming an inseparable part of key business functions, we will see the CDO act as a bridge between other C-suite execs.

Data skills an emerging business necessity

So far, only large enterprises with bigger data mining and management needs maintain in-house solutions. These in-house teams and technologies handle the growing sets of diverse and dispersed data. Others work with third-party service providers to develop and execute their big data strategies.

As the amount of data grows, the need to mine it for insights becomes a key business requirement. For both large and small businesses, data-centric roles will experience endless upward mobility. These roles include data anlysts and scientists. There is going to be a huge opportunity for critical thinkers to turn their analytical skills into rapidly growing roles in the field of data science. In fact, data skills are now a prized qualification for titles like IT project managers and computer systems analysts.

Forbes cited the McKinsey Global Institute’s prediction that by 2018 there could be a massive shortage of data-skilled professionals. This indicates a disruption at the demand-supply level with the needs for data skills at an all-time high. With an increasing number of companies adopting big data strategies, salaries for data jobs are going through the roof. This is turning the position into a highly coveted one.

According to Harvard Professor Gary King, “There is a big data revolution. The big data revolution is that now we can do something with the data.” The big problem is that most enterprises don’t know what to do with data. Data professionals are helping businesses figure that out. So if you’re casting about for where to apply your skills and want to take advantage of one of the best career paths in the job market today, focus on data science.

I’m compensated by University of Phoenix for this blog. As always, all thoughts and opinions are my own.

For more insight on our increasingly connected future, see The $19 Trillion Question: Are You Undervaluing The Internet Of Things?

The post Data Analysts and Scientists More Important Than Ever For the Enterprise appeared first on Millennial CEO.

Comments

About Daniel Newman

Daniel Newman serves as the Co-Founder and CEO of EC3, a quickly growing hosted IT and Communication service provider. Prior to this role Daniel has held several prominent leadership roles including serving as CEO of United Visual. Parent company to United Visual Systems, United Visual Productions, and United GlobalComm; a family of companies focused on Visual Communications and Audio Visual Technologies. Daniel is also widely published and active in the Social Media Community. He is the Author of Amazon Best Selling Business Book "The Millennial CEO." Daniel also Co-Founded the Global online Community 12 Most and was recognized by the Huffington Post as one of the 100 Business and Leadership Accounts to Follow on Twitter. Newman is an Adjunct Professor of Management at North Central College. He attained his undergraduate degree in Marketing at Northern Illinois University and an Executive MBA from North Central College in Naperville, IL. Newman currently resides in Aurora, Illinois with his wife (Lisa) and his two daughters (Hailey 9, Avery 5). A Chicago native all of his life, Newman is an avid golfer, a fitness fan, and a classically trained pianist

The Value Of BI Competency Centers

Simran Kohli

A business intelligence competency center (BICC) or center of excellence has always been a key component of BI success.

“It has been one of two major recommendations for an effective BI strategy, the other being an executive business sponsor,” says Pat Saporito, global COE for BI and SAP’s BI strategy program leader. However, with the growing demand for business user self-service fueled by Big Data and the Internet of Things, organizations are either creating new BICCs or retooling existing more technically oriented BICCs that were more shared services or project management offices.  “We found that BICCs and information governance were the two hot topics companies wanted more information on as a follow-on to a BI strategy discussion or workshop,” added Saporito.

To address this renewed interest and gauge the value of BI competency centers, SAP used the survey-based performance benchmarking approach and created a BI Competency Center survey. Below are the aggregate findings to date from the assessment:

  • More than 40% organizations currently have a BI competency center in place, and approximately 60% of them are less than 5 years old.
  • 33% organizations follow a best-practices-driven model for managing their BI competency centers.
  • Only 12% organizations have more than 50% business ownership or engagement in their BI competency centers.
  • Almost 85% organizations find “increased BI usage/adoption” as one of the top 5 benefits they have attained or seek to gain through BI competency centers.
  • 57% of BI competency centers are funded from corporate or IT and have CFO as an executive business sponsor other than the CIO.

These findings substantiate the fact that many BI and analytics leaders now are looking at redefining the BICC to support more impactful use cases aimed at improving business performance. It may be an intuitive step, but it underscores the reality that developing a BICC strategy holistically across an entire organization is a critical first step in operationalizing your BI strategy, and is the key to business alignment and BI success.

To learn more about BI Competency Centers, visit our BI Success website and download the BICC eGuide.

Comments

Primed: Prompting Customers to Buy

Volker Hildebrand, Sam Yen, and Fawn Fitter

When it comes to buying things—even big-ticket items—the way we make decisions makes no sense. One person makes an impulsive offer on a house because of the way the light comes in through the kitchen windows. Another gleefully drives a high-end sports car off the lot even though it will probably never approach the limits it was designed to push.

We can (and usually do) rationalize these decisions after the fact by talking about needing more closet space or wanting to out-accelerate an 18-wheeler as we merge onto the highway, but years of study have arrived at a clear conclusion:

When it comes to the customer experience, human beings are fundamentally irrational.

In the brick-and-mortar past, companies could leverage that irrationality in time-tested ways. They relied heavily on physical context, such as an inviting retail space, to make products and services as psychologically appealing as possible. They used well-trained salespeople and employees to maximize positive interactions and rescue negative ones. They carefully sequenced customer experiences, such as having a captain’s dinner on the final night of a cruise, to play on our hard-wired craving to end experiences on a high note.

Today, though, customer interactions are increasingly moving online. Fortune reports that on 2016’s Black Friday, the day after Thanksgiving that is so crucial to holiday retail results, 108.5 million Americans shopped online, while only 99.1 million visited brick-and-mortar stores. The 9.4% gap between the two was a dramatic change from just one year prior, when on- and offline Black Friday shopping were more or less equal.

When people browse in a store for a few minutes, an astute salesperson can read the telltale signs that they’re losing interest and heading for the exit. The salesperson can then intervene, answering questions and closing the sale.

Replicating that in a digital environment isn’t as easy, however. Despite all the investments companies have made to counteract e-shopping cart abandonment, they lack the data that would let them anticipate when a shopper is on the verge of opting out of a transaction, and the actions they take to lure someone back afterwards can easily come across as less helpful than intrusive.

In a digital environment, companies need to figure out how to use Big Data analysis and digital design to compensate for the absence of persuasive human communication and physical sights, sounds, and sensations. What’s more, a 2014 Gartner survey found that 89% of marketers expected customer experience to be their primary differentiator by 2016, and we’re already well into 2017.

As transactions continue to shift toward the digital and omnichannel, companies need to figure out new ways to gently push customers along the customer journey—and to do so without frustrating, offending, or otherwise alienating them.

The quest to understand online customers better in order to influence them more effectively is built on a decades-old foundation: behavioral psychology, the study of the connections between what people believe and what they actually do. All of marketing and advertising is based on changing people’s thoughts in order to influence their actions. However, it wasn’t until 2001 that a now-famous article in the Harvard Business Review formally introduced the idea of applying behavioral psychology to customer service in particular.

The article’s authors, Richard B. Chase and Sriram Dasu, respectively a professor and assistant professor at the University of Southern California’s Marshall School of Business, describe how companies could apply fundamental tenets of behavioral psychology research to “optimize those extraordinarily important moments when the company touches its customers—for better and for worse.” Their five main points were simple but have proven effective across multiple industries:

  1. Finish strong. People evaluate experiences after the fact based on their high points and their endings, so the way a transaction ends is more important than how it begins.
  2. Front-load the negatives. To ensure a strong positive finish, get bad experiences out of the way early.
  3. Spread out the positives. Break up the pleasurable experiences into segments so they seem to last longer.
  4. Provide choices. People don’t like to be shoved toward an outcome; they prefer to feel in control. Giving them options within the boundaries of your ability to deliver builds their commitment.
  5. Be consistent. People like routine and predictability.

For example, McKinsey cites a major health insurance company that experimented with this framework in 2009 as part of its health management program. A test group of patients received regular coaching phone calls from nurses to help them meet health goals.

The front-loaded negative was inherent: the patients knew they had health problems that needed ongoing intervention, such as weight control or consistent use of medication. Nurses called each patient on a frequent, regular schedule to check their progress (consistency and spread-out positives), suggested next steps to keep them on track (choices), and cheered on their improvements (a strong finish).

McKinsey reports the patients in the test group were more satisfied with the health management program by seven percentage points, more satisfied with the insurance company by eight percentage points, and more likely to say the program motivated them to change their behavior by five percentage points.

The nurses who worked with the test group also reported increased job satisfaction. And these improvements all appeared in the first two weeks of the pilot program, without significantly affecting the company’s costs or tweaking key metrics, like the number and length of the calls.

Indeed, an ongoing body of research shows that positive reinforcements and indirect suggestions influence our decisions better and more subtly than blatant demands. This concept hit popular culture in 2008 with the bestselling book Nudge.

Written by University of Chicago economics professor Richard H. Thaler and Harvard Law School professor Cass R. Sunstein, Nudge first explains this principle, then explores it as a way to help people make decisions in their best interests, such as encouraging people to eat healthier by displaying fruits and vegetables at eye level or combatting credit card debt by placing a prominent notice on every credit card statement informing cardholders how much more they’ll spend over a year if they make only the minimum payment.

Whether they’re altruistic or commercial, nudges work because our decision-making is irrational in a predictable way. The question is how to apply that awareness to the digital economy.

In its early days, digital marketing assumed that online shopping would be purely rational, a tool that customers would use to help them zero in on the best product at the best price. The assumption was logical, but customer behavior remained irrational.

Our society is overloaded with information and short on time, says Brad Berens, Senior Fellow at the Center for the Digital Future at the University of Southern California, Annenberg, so it’s no surprise that the speed of the digital economy exacerbates our desire to make a fast decision rather than a perfect one, as well as increasing our tendency to make choices based on impulse rather than logic.

Buyers want what they want, but they don’t necessarily understand or care why they want it. They just want to get it and move on, with minimal friction, to the next thing. “Most of our decisions aren’t very important, and we only have so much time to interrogate and analyze them,” Berens points out.

But limited time and mental capacity for decision-making is only half the issue. The other half is that while our brains are both logical and emotional, the emotional side—also known as the limbic system or, more casually, the primitive lizard brain—is far older and more developed. It’s strong enough to override logic and drive our decisions, leaving rational thought to, well, rationalize our choices after the fact.

This is as true in the B2B realm as it is for consumers. The business purchasing process, governed as it is by requests for proposals, structured procurement processes, and permission gating, is designed to ensure that the people with spending authority make the most sensible deals possible. However, research shows that even in this supposedly rational process, the relationship with the seller is still more influential than product quality in driving customer commitment and loyalty.

Baba Shiv, a professor of marketing at Stanford University’s Graduate School of Business, studies how the emotional brain shapes decisions and experiences. In a popular TED Talk, he says that people in the process of making decisions fall into one of two mindsets: Type 1, which is stressed and wants to feel comforted and safe, and Type 2, which is bored or eager and wants to explore and take action.

People can move between these two mindsets, he says, but in both cases, the emotional brain is in control. Influencing it means first delivering a message that soothes or motivates, depending on the mindset the person happens to be in at the moment and only then presenting the logical argument to help rationalize the action.

In the digital economy, working with those tendencies means designing digital experiences with the full awareness that people will not evaluate them objectively, says Ravi Dhar, director of the Center for Customer Insights at the Yale School of Management. Since any experience’s greatest subjective impact in retrospect depends on what happens at the beginning, the end, and the peaks in between, companies need to design digital experiences to optimize those moments—to rationally design experiences for limited rationality.

This often involves making multiple small changes in the way options are presented well before the final nudge into making a purchase. A paper that Dhar co-authored for McKinsey offers the example of a media company that puts most of its content behind a paywall but offers free access to a limited number of articles a month as an incentive to drive subscriptions.

Many nonsubscribers reached their limit of free articles in the morning, but they were least likely to respond to a subscription offer generated by the paywall at that hour, because they were reading just before rushing out the door for the day. When the company delayed offers until later in the day, when readers were less distracted, successful subscription conversions increased.

Pre-selecting default options for necessary choices is another way companies can design digital experiences to follow customers’ preference for the path of least resistance. “We know from a decade of research that…defaults are a de facto nudge,” Dhar says.

For example, many online retailers set a default shipping option because customers have to choose a way to receive their packages and are more likely to passively allow the default option than actively choose another one. Similarly, he says, customers are more likely to enroll in a program when the default choice is set to accept it rather than to opt out.

Another intriguing possibility lies in the way customers react differently to on-screen information based on how that information is presented. Even minor tweaks can have a disproportionate impact on the choices people make, as explained in depth by University of California, Los Angeles, behavioral economist Shlomo Benartzi in his 2015 book, The Smarter Screen.

A few of the conclusions Benartzi reached: items at the center of a laptop screen draw more attention than those at the edges. Those on the upper left of a screen split into quadrants attract more attention than those on the lower left. And intriguingly, demographics are important variables.

Benartzi cites research showing that people over 40 prefer more visually complicated, text-heavy screens than younger people, who are drawn to saturated colors and large images. Women like screens that use a lot of different colors, including pastels, while men prefer primary colors on a grey or white background. People in Malaysia like lots of color; people in Germany don’t.

This suggests companies need to design their online experiences very differently for middle-aged women than they do for teenage boys. And, as Benartzi writes, “it’s easy to imagine a future in which each Internet user has his or her own ‘aesthetic algorithm,’ customizing the appearance of every site they see.”

Applying behavioral psychology to the digital experience in more sophisticated ways will require additional formal research into recommendation algorithms, predictions, and other applications of customer data science, says Jim Guszcza, PhD, chief U.S. data scientist for Deloitte Consulting.

In fact, given customers’ tendency to make the fastest decisions, Guszcza believes that in some cases, companies may want to consider making choice environments more difficult to navigate— a process he calls “disfluencing”—in high-stakes situations, like making an important medical decision or an irreversible big-ticket purchase. Choosing a harder-to-read font and a layout that requires more time to navigate forces customers to work harder to process the information, sending a subtle signal that it deserves their close attention.

That said, a company can’t apply behavioral psychology to deliver a digital experience if customers don’t engage with its site or mobile app in the first place. Addressing this often means making the process as convenient as possible, itself a behavioral nudge.

A digital solution that’s easy to use and search, offers a variety of choices pre-screened for relevance, and provides a friction-free transaction process is the equivalent of putting a product at eye level—and that applies far beyond retail. Consider the Global Entry program, which streamlines border crossings into the U.S. for pre-approved international travelers. Members can skip long passport control lines in favor of scanning their passports and answering a few questions at a touchscreen kiosk. To date, 1.8 million people have decided this convenience far outweighs the slow pace of approvals.

The basics of influencing irrational customers are essentially the same whether they’re taking place in a store or on a screen. A business still needs to know who its customers are, understand their needs and motivations, and give them a reason to buy.

And despite the accelerating shift to digital commerce, we still live in a physical world. “There’s no divide between old-style analog retail and new-style digital retail,” Berens says. “Increasingly, the two are overlapping. One of the things we’ve seen for years is that people go into a store with their phones, shop for a better price, and buy online. Or vice versa: they shop online and then go to a store to negotiate for a better deal.”

Still, digital increases the number of touchpoints from which the business can gather, cluster, and filter more types of data to make great suggestions that delight and surprise customers. That’s why the hottest word in marketing today is omnichannel. Bringing behavioral psychology to bear on the right person in the right place in the right way at the right time requires companies to design customer experiences that bridge multiple channels, on- and offline.

Amazon, for example, is known for its friction-free online purchasing. The company’s pilot store in Seattle has no lines or checkout counters, extending the brand experience into the physical world in a way that aligns with what customers already expect of it, Dhar says.

Omnichannel helps counter some people’s tendency to believe their purchasing decision isn’t truly well informed unless they can see, touch, hear, and in some cases taste and smell a product. Until we have ubiquitous access to virtual reality systems with full haptic feedback, the best way to address these concerns is by providing personalized, timely, relevant information and feedback in the moment through whatever channel is appropriate. That could be an automated call center that answers frequently asked questions, a video that shows a product from every angle, or a demonstration wizard built into the product. Any of these channels could also suggest the customer visit the nearest store to receive help from a human.

The omnichannel approach gives businesses plenty of opportunities to apply subtle nudges across physical and digital channels. For example, a supermarket chain could use store-club card data to push personalized offers to customers’ smartphones while they shop. “If the data tells them that your goal is to feed a family while balancing nutrition and cost, they could send you an e-coupon offering a discount on a brand of breakfast cereal that tastes like what you usually buy but contains half the sugar,” Guszcza says.

Similarly, a car insurance company could provide periodic feedback to policyholders through an app or even the digital screens in their cars, he suggests. “Getting a warning that you’re more aggressive than 90% of comparable drivers and three tips to avoid risk and lower your rates would not only incentivize the driver to be more careful for financial reasons but reduce claims and make the road safer for everyone.”

Digital channels can also show shoppers what similar people or organizations are buying, let them solicit feedback from colleagues or friends, and read reviews from other people who have made the same purchases. This leverages one of the most familiar forms of behavioral psychology—reinforcement from peers—and reassures buyers with Shiv’s Type 1 mindset that they’re making a choice that meets their needs or encourages those with the Type 2 mindset to move forward with the purchase. The rational mind only has to ask at the end of the process “Am I getting the best deal?” And as Guszcza points out, “If you can create solutions that use behavioral design and digital technology to turn my personal data into insight to reach my goals, you’ve increased the value of your engagement with me so much that I might even be willing to pay you more.”

Many transactions take place through corporate procurement systems that allow a company to leverage not just its own purchasing patterns but all the data in a marketplace specifically designed to facilitate enterprise purchasing. Machine learning can leverage this vast database of information to provide the necessary nudge to optimize purchasing patterns, when to buy, how best to negotiate, and more. To some extent, this is an attempt to eliminate psychology and make choices more rational.

B2B spending is tied into financial systems and processes, logistics systems, transportation systems, and other operational requirements in a way no consumer spending can be. A B2B decision is less about making a purchase that satisfies a desire than it is about making a purchase that keeps the company functioning.

That said, the decision still isn’t entirely rational, Berens says. When organizations have to choose among vendors offering relatively similar products and services, they generally opt for the vendor whose salespeople they like the best.

This means B2B companies have to make sure they meet or exceed parity with competitors on product quality, pricing, and time to delivery to satisfy all the rational requirements of the decision process. Only then can they bring behavioral psychology to bear by delivering consistently superior customer service, starting as soon as the customer hits their app or website and spreading out positive interactions all the way through post-purchase support. Finishing strong with a satisfied customer reinforces the relationship with a business customer just as much as it does with a consumer.

The best nudges make the customer relationship easy and enjoyable by providing experiences that are effortless and fun to choose, on- or offline, Dhar says. What sets the digital nudge apart in accommodating irrational customers is its ability to turn data about them and their journey into more effective, personalized persuasion even in the absence of the human touch.

Yet the subtle art of influencing customers isn’t just about making a sale, and it certainly shouldn’t be about persuading people to act against their own best interests, as Nudge co-author Thaler reminds audiences by exhorting them to “nudge for good.”

Guszcza, who talks about influencing people to make the choices they would make if only they had unlimited rationality, says companies that leverage behavioral psychology in their digital experiences should do so with an eye to creating positive impact for the customer, the company, and, where appropriate, the society.

In keeping with that ethos, any customer experience designed along behavioral lines has to include the option of letting the customer make a different choice, such as presenting a confirmation screen at the end of the purchase process with the cold, hard numbers and letting them opt out of the transaction altogether.

“A nudge is directing people in a certain direction,” Dhar says. “But for an ethical vendor, the only right direction to nudge is the right direction as judged by the customers themselves.” D!

Read more thought provoking articles in the latest issue of the Digitalist Magazine, Executive Quarterly.


About the Authors:

Volker Hildebrand is Global Vice President for SAP Hybris solutions.

Sam Yen is Chief Design Officer and Managing Director at SAP.

Fawn Fitter is a freelance writer specializing in business and technology.

Comments

Tags:

The Big (Data) Problem With Machine Learning

Dan Wellers

Historically, most of the data businesses have analyzed for decision-making has been of the structured variety—easily entered, stored, and queried. In the digital age, that universe of potentially valuable data keeps expanding exponentially. Most of it is unstructured data, coming from a wide variety of sources, from websites to wearable devices. As a recent McKinsey Global Institute report noted: “Much of this newly available data is in the form of clicks, images, text, or signals of various sorts, which is very different than the structured data that can be cleanly placed in rows and columns.”

At the same time, we have entered an era when machine learning can theoretically find patterns in vast amounts of data to enable enterprises to uncover insights that may not have been visible before. Machine learning trains itself on data, and for a time, that data was scarce. Today it is abundant. By 2025, the world will create 180 zettabytes of data per year (up from 4.4 zettabytes in 2013), according to IDC.

Big Data and machine learning would seem to be a perfect match, coming together at just the right time. But it’s not that simple.

The connected world is ever-widening, enabling the capture and storage of more—and more diverse—data sets than ever before. Nearly 5,000 devices are being connected to the Internet every minute today; within ten years, there will be 80 billion devices collecting and transmitting data in the world. Voice, facial recognition, chemical, biological, and 3D-imaging sensors are rapidly advancing. And the computing muscle that will be required to churn through all this data is more readily available today. There’s been a one trillion-fold increase in computing power over the past 60 years.

The importance of data prep

But having vast amounts of data and computing power isn’t enough. For machine learning tools to work, they need to be fed high-quality data, and they must also be guided by highly skilled humans.

It’s the age-old computing axiom writ large: garbage in, garbage out. Data must be clean, scrubbed of anomalies, and free of bias. In addition, it must be structured appropriately for the particular machine-learning tool being used as the required format varies by platform. Preparing data is likely the least sexy but most important part of a data scientist’s job—one that accounts for as much as 50 percent of his or her time, according to some estimates. It’s the unglamorous heavy lifting of advanced analytics, and it takes experience and skill to do it—qualities that are, and will continue to be, in short supply even as demand for data scientists is predicted to grow at double-digit rates for the foreseeable future.

It took one bank 150 people and two years of painstaking work to address all the data quality questions necessary to build an enterprise-wide data lake from which advanced analytics tools might drink. That’s the kind of data wrangling that has to be done before companies can even begin to test the value of machine-learning capabilities.

More data, more problems

There’s also the misperception that having access to all this new data will necessarily lead to greater insight. There’s great enthusiasm around data-driven decision-making and the promise of Big Data and machine learning in boardrooms and executive suites around the world. But in reality, says UC Berkeley professor and machine learning expert Michael I. Jordan, more data increases the likelihood of making spurious connections. “It’s like having billions of monkeys typing. One of them will write Shakespeare,” said Jordan, who noted that Big Data analysis can deliver inferences at certain levels of quality. But, he said, “we have to be clear about what levels of quality. We have to have error bars around all our predictions. That is something that’s missing in much of the current machine learning literature.”

Again, this is where the expertise of the data scientist is of critical value: deciding what questions machine learning might be able to answer, with what data and at what level of quality.

These problems are not insurmountable. Tools are being developed to help businesses deal with some of the data management blocking and tackling that stands in the way of advanced analytics. One company, for example, has developed a machine-learning tool for real estate and finance companies that it says can extract unstructured data in 20 different languages from contracts and other legal documents and transform it into a structured, query-ready format.

What is clear is that the business of combining Big Data and big computing power for new insight is harder than it looks. The benefits almost certainly will be huge. But companies are still at the early stages of experimenting with new data types and emerging machine-learning tools and discovering the drawbacks and complications we will need to work through over time.

This blog is the fifth in a six-part series on machine learning.

Comments

About Dan Wellers

Dan Wellers is the Global Lead of Digital Futures at SAP, which explores how organizations can anticipate the future impact of exponential technologies. Dan has extensive experience in technology marketing and business strategy, plus management, consulting, and sales.