Sections

#GartnerBI and Big Data: Fear, Loathing, and Business Breakthroughs

Timo Elliott

big data

At the Gartner Business Intelligence and Analytics Summit in Barcelona, there was near universal agreement about three things to do with “big data”

  1. It’s an awful term (but we’re stuck with it)
  2. Whatever it means, it’s a big deal, and requires big changes to traditional information infrastructures
  3. It will result in big new business opportunities

“Big Data” is a terrible term

Gartner analyst Doug Laney first coined the term “big data” over over 12 years ago (at least in its current form – people have been complaining about “information overload” since Roman times). But the term’s meaning is still far from clear and it was nominated the #1 “tech buzzword that everyone uses but don’t quite understand” (followed closely by “cloud”).

When using the term, Gartner usually keeps the quote marks in place (i.e. it’s “big data”, not big data). Here’s the definition provided by analyst Ted Friedman to “de-hype” the term during the summit keynote:

“Big Data” are high volume, velocity and variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision-making”

Analyst Donald Feinberg warned people that “talking only about big data can lead to self-delusion” and urged people not to “surrender to the hype-ocracy.” He left left no doubt over where he stood on the use of the term: “Big data doesn’t mean MapReduce or Hadoop. Big data doesn’t exist, it’s meaningless, it’s ridiculous…” The audience started applauding, to which he replied: “Why are you clapping?! Why do you all fall for it? Why do the vendors do it?!

As SAP’s Jason Rose? put it “how can we demystify this? …easy, drop the ‘big’. Data has always been the key challenge in BI”.

For what it’s worth, here’s my tongue-in-cheek definition, and it’s worth noting that big data is quickly becoming the default term for what we used to call analytics or business intelligence.

But Big Data is a Big Deal

Despite the problems, Doug Laney noted that big data the most-searched-for term on Gartner.com.  Why is it so popular? Maybe because it’s so nebulous that people want to check if they have understood it. Or maybe because there’s no other more precise term to indicate the new analytic opportunities. And maybe because “hype has a value” as Ted Friedman put it: big data has proved to be a new opportunity to talk to business people about the power of analytics, and because everybody’s searching for it, vendors would be crazy not to include it in their marketing.

Conference attendees generally believed that the biggest opportunity for big data analysis was new insights from “dark data” that lies unused within organizations today. Gartner highlighted the dangers of implementing shiny new big data technology, separate from existing analytics infrastructures: “Do not make your big data implementations siloed. Make them part of the overall strategy for BI.” said Ted Friedman. “Link to stuff you are already doing. Don’t make big data a standalone thing. And don’t feel like you’ve got to go out and buy a whole new technology stack.”

Analyst Rita Sallam, in a session on data variety, gave some examples of the new opportunities:

Some of Gartner’s public predictions related to big data:

  • By 2015, 65 percent of packaged analytic applications with advanced analytics will come embedded with Hadoop.
  • By 2016, 70 percent of leading BI vendors will have incorporated natural-language and spoken-word capabilities.
  • By 2015, more than 30 percent of analytics projects will deliver insights based on structured and unstructured data.

But What Does It Mean For The Business?

Donald Fienberg: “Realize that big data is not about doing ‘more’ of the same thing – it’s about doing things differently”. “The major opportunities for big data are around ways to transform the business and disrupt the industry” said Doug Laney. These included radically changing existing business processes, introducing new, more-personalized products and services, and “answering chewy questions that weren’t possible before.” Some examples:

  • NetFlix did deep analysis of their viewers’ preferences, and used that to craft the new “House of Cards” TV series – a $100M investment
  • New financial lenders are using big data to find untapped banking opportunities – including lending scores based on what you say on social media
  • Passur uses big data to provides real-time monitoring of air traffic, to potentially save millions of dollars per year. Today, pilots’ estimated times of arrival are off by more than ten minutes ten percent of the time, and five minutes 30% of the time – knowing exactly when the planes will arrive means more automation, better operating efficiencies, improved security, etc.
  • Enologix analyzes the chemical composition of new wines to predict wine spectator score, and offer advice on how to improve the score
  • Dollar General, Kroger and other retailers provide data to partners to analyze, for “free strategic advice”
  • Insurance companies are using text mining on previously-unexamined “dark data” on claims forms to sniff out indicators of fraud
Mats-Olov Eriksson of King.com, a Sweden-based casual gaming site (card games, social games, etc), gave an entertaining presentation called “Beyond Big and Data”, hosted by recently-returned Gartner analyst Frank Buytendijk. It was illustrated with more cute kittens, puppies, and otters than I’ve ever seen before in a technology conference!

kingcom

Mat’s title is “Spiritual Leader, Data Warehouse”, and he explained how the company collects massive amounts of log data about user activities, then uses a combination of Hadoop, traditional data warehousing, and BI cubes to optimize the player experience – and the company’s profits.

The company works with Facebook on games like Candy Crush Saga. The company’s main revenue comes from in-app purchases: for example, you might get five free lives in a game, and if you run out you can either wait for a period, or pay a small sum to play immediately. To try to figure out what options generate the most revenue, and keep players in the game, the company currently processes 60,000 log files per day. The files contain around three billion “events” per day, and the company expects that to rise to six billion by the end of this year. Mats explained that big data is best compared to a software development project, requiring hard-to-find skilled personnel. He characterized hadoop as a “powerful but immature five year old” and explained that they used a MySQL data mart for low-latency access: “HIVE is anything but low latency – it takes fifteen seconds before you even get an error message!”.

One of the recent challenges the company has faced is information governance – a term they hadn’t even used until six months ago. Mats noted that the European privacy laws were by far the “harshest” they deal with.

In a separate presentation, that I unfortunately didn’t attend, Tom Fastner of eBay gave an overview of his company’s analytic systems, explaining that they process more than 100 Petabytes of data I/Os every day, and a “singularity” keyvalue store on Teradata manages 36 Petabytes of data. The largest table is 3 Petabytes / 3 trillion rows, and only takes 32 seconds to scan. Here’s a presentation from 2011 that talks about how eBay combines structured, semi-structured, and unstructured data.

The inimitable Steve Lucas says of big data: “when you see it, you know it”. MKI, a bioscience company is using big data to analyze genome data, using a combination of hadoop, open-source “R” algorithms, and SAP’s in-memory platform, SAP HANA (registration required).

image

image

image

Here’s a summary of the MKI project:

In Conclusion

Big data is a lousy term, but offers big opportunities in return for some big information infrastructure changes. What does the future hold? Let’s hope less of the hype, and more of the business change…

 

Comments

About Timo Elliott

Timo Elliott is an Innovation Evangelist for SAP and a passionate advocate of innovation, digital business, analytics, and artificial intelligence. He was the eighth employee of BusinessObjects and for the last 25 years he has worked closely with SAP customers around the world on new technology directions and their impact on real-world organizations. His articles have appeared in articles such as Harvard Business Review, Forbes, ZDNet, The Guardian, and Digitalist Magazine. He has worked in the UK, Hong Kong, New Zealand, and Silicon Valley, and currently lives in Paris, France. He has a degree in Econometrics and a patent in mobile analytics. 

Letting Transparency And Insights Lead The Way

Mohammed Karzoun

Excellence in governance is hinged on exceeding service delivery expectations of citizens while balancing fiscal health and overall efficiencies. A transparent and connected government that uses insights to guide decision-making fosters trust and helps governments achieve efficiencies across financial, social, and economic aspects of governance. 

Today’s information age is no longer about sole proprietorship of information and data. The more information is shared, the more value it adds. Sharing information represents transparency and encourages trust by offering ample opportunity for citizens to scrutinize and understand the rationale behind public service offerings.

Further, analyzing data yields insights, helping the government make informed policy decisions and step up against corruption, fraud, and lacunae in functioning. To achieve this effectively, I feel data is key to driving excellence in governance. Data, captured in real time and shared, enables a transparent and accountable environment and delivers significant insights for improved governance. 

Insights help governments deliver the customer promise

Citizens today are very demanding of governments, expecting open access to information, transparency of transactions, seamless interactions, and fulfillment of needs. The last decade has witnessed a greater focus on reforms in the public sector aimed at improving service delivery and enhancing the customer promise. A critical result of these reforms has been the necessity to treat citizens like customers and place their needs at the core of every decision, from formulating strategies to designing policies and execution.

Governments need to re-engineer their processes and use data-driven governance to enable this customer-centricity. And for this, technology can be a key enabler because the development of customer-centric models calls for customer insights—an evaluation of customers’ wants and needs, improving the customer experience and minimizing risks associated with service delivery.

Business forensics can help optimize government resource planning

Customer feedback can generate significant insights to help governments deliver on the customer promise. By understanding the needs of its citizens and putting in place mechanisms to address those needs or problems, a government sets the tone for achieving excellence.

Insights are not only necessary to deliver efficiencies to citizens. To achieve a fiscal balance and internal efficiencies, the governments’ interaction with its extended customer base of employees, suppliers, and partners also needs to be managed and resources optimized. Business forensics also deliver these capabilities to governments by helping generate insights needed to optimize the governments’ financial resources and react to economic changes.

I believe that SAP’s transformative technologies that leverage real-time analytics provide governments with real-time data and flexible reporting tools to gain deeper insights into areas of improvement across departments. This helps identify and minimize risk and ensure compliance.

Transparency drives accountability as silos give way to a connected government

Transparency syncs governmental objectives with project objectives. A transparent and clear understanding of governance strategy helps create this sync while balancing resource priorities to ensure that outcomes are timely and effective. This also helps the governments see the big picture and drill down into details for instant decision-making while having single version of the truth.

Transparency requires that department silos give way to a connected government. An integrated and coordinated governance model not only avoids duplication of efforts and resources but also eliminates redundancies in storing information and utilizing resources.

A connected government boasts better communication and coordination across departments, increases ministry-wide transparency, and enhances accountability across a decentralized environment. It holds to reason that government entities and departments that are accountable tend to be more agile, flexible, and efficient.

I feel an important enabler for this transparency and accountability can be setting service standards. By putting in place specific mechanisms to measure KPI’s and evaluate service levels, governments can become more responsive and achieve citizen happiness.

Keeping it real: Why real-time information is the cornerstone of transparency and insights

People no longer want to be told about what happened in the past and how; they seek real-time information. Real-time information and real-time analytics enrich both the government and its citizens with data, which leads to transparency from the perspective of citizens and converts into insights from the government’s point of view.

This open government partnership implies that the decision-makers can anticipate the needs of citizens and the needs of the economy on a real-time basis and make decisions on policies accordingly and on the fly. Such decisions increase reliability and consistency of financial and operational information.

Real-time governance enabled by technology provides the governments entities and their leadership with unprecedented visibility into the workings of the various departments, increasing accountability and enhancing opportunities for collaboration across functions. It allows governments to answer key questions in the moment, such as: What is happening now? Why is it happening? What will happen? Governments can move to the era of predictive analytics by collecting and relating their objectives in real time to make informative decisions for better future planning.

In conclusion, changes in citizen needs and expectations are a continuous process, and a radical one-time reform cannot suffice. Governments need to leverage technology to implement a mechanism that continually monitors insights and develops an institutional culture of improvement. At the same time, encouraging a data-driven transparent service delivery model can position governments well to achieve excellence by delivering on the customer promise.

For more on the role of technology in government, see A Duty Of Care In The Digital Government Era.

Comments

Mohammed Karzoun

About Mohammed Karzoun

Mohammed Karzoun is the Industry Leader for Public Sector at SAP. He manages government, smart cities, healthcare, public security, defense, higher education, and postal services sectors across the United Arab Emirates and Oman. With 20 years of experience in primarily public sector transformation, Mohammed has been engaged with multiple government entities to help drive their strategies and digital transformation initiatives.

How Analytics Fits Into Your Next Digital Transformation

Eamon Ida

We hear a lot about digital transformation these days and the groundbreaking technologies that will lead to these transformations, such as blockchain, IoT (Internet of Things), ML (machine learning), AI (artificial intelligence), the cloud, and analytics, to name just a few. All it takes is a look at advertisements from various tech companies (and these days every company is a tech company) in Harvard Business Review, Wired, or the Wall Street Journal to see the examples in action.

Much of the message revolves around leveraging these new technologies to disrupt existing models and create new value. But many business leaders who are new to these technologies—though  not lacking in curiosity about their potential—wonder how they can leverage these different technologies. Can they be connected and surfaced together to create true digital innovation systems? By showcasing scenarios, we’ve shown how an analytics cloud platform helps solve these questions and connect technology together to make smarter decisions faster and at the speed of market changes.

Digital transformation and machine learning

Machine learning is one example. In scores of today’s business scenarios, algorithms are instantly suggesting intelligent decisions for use in more involved decision making or where many variables are present. But in many cases, we’re still at the point where humans need to approve the machine (or robot) decision by hand.

However, scheduling a predictive maintenance repair to one or more your vehicles in a fleet before damage occurs, changing the route of a vehicle in the supply chain to avoid traffic that will delay delivery, or detecting patterns in transactions before a fraud occurs—all of these are examples of intelligent decisions suggested by machine learning. (Read Trenitalia’s IoT-Enabled Dynamic Maintenance.)

These decision suggestions or approvals are best located in an analytics platform, where we can surface up not only the recommended decisions, but also contextual data from other sources (such as geographic or IoT information) to improve the quality of the decision.

Digital transformation and IoT

When we consider IoT data, the value of analytics becomes even clearer—sensor technology, for example, can produce data every second or, in many instances, milliseconds. With analytics, we’re able to surface up this sensor data, along with data from other sources, to a real-time dashboard that provides both real-time and actionable insights into sensor data and activity.

Digital transformation and cloud platforms

With the democratization of data and the inevitable transparency of business in today’s connected world, we need to consider investments in technology that allow us to provide these platforms to employees, customers, and vendors all around the world. This is where the beauty of a cloud platform, powered by an in-memory computing platform comes into play.

Using a cloud platform not only enables innovation at scale, but also the connection of disparate data sources and technologies together. Just look at companies in industries such as airlines, delivery, and healthcare (to name just a few). Disruptors abound that are building “cloud first” and are not just using analytics internally but are providing it to customers as a competitive advantage.

You, like many of the leaders we speak with, might be wondering, What are some examples of digital transformation where analytics and data have played a critical role? What are some of the biggest challenges or roadblocks to more pervasive and effective analytics, and what are some of the best practices you have seen in the use of analytics?

SAP recently completed a new paper with IDC on the Value of Analytics in Digital Transformation. In it, we answer all of these common questions and more with the help of industry experts.  Read the whitepaper today to learn how you and your organization can leverage the disruptive powers of analytics in your next digital transformation. And see this infographic for a bird’s-eye view of the key findings.

Comments

Eamon Ida

About Eamon Ida

Eamon is a passionate, creative, and data-driven analytics product marketer at SAP. He has a special interest in technology and its role at the forefront of how we live and work together.

Running Future Cities on Blockchain

Dan Wellers , Raimund Gross and Ulrich Scholl

Building on the Blockchain Framework

Some experts say these seemingly far-future speculations about the possibilities of combining technologies using blockchain are actually both inevitable and imminent:


Democratizing design and manufacturing by enabling individuals and small businesses to buy, sell, share, and digitally remix products affordably while protecting intellectual property rights.
Decentralizing warehousing and logistics by combining autonomous vehicles, 3D printers, and smart contracts to optimize delivery of products and materials, and even to create them on site as needed.
Distributing commerce by mixing virtual reality, 3D scanning and printing, self-driving vehicles, and artificial intelligence into immersive, personalized, on-demand shopping experiences that still protect buyers’ personal and proprietary data.

The City of the Future

Imagine that every agency, building, office, residence, and piece of infrastructure has an entry on a blockchain used as a city’s digital ledger. This “digital twin” could transform the delivery of city services.

For example:

  • Property owners could easily monetize assets by renting rooms, selling solar power back to the grid, and more.
  • Utilities could use customer data and AIs to make energy-saving recommendations, and smart contracts to automatically adjust power usage for greater efficiency.
  • Embedded sensors could sense problems (like a water main break) and alert an AI to send a technician with the right parts, tools, and training.
  • Autonomous vehicles could route themselves to open parking spaces or charging stations, and pay for services safely and automatically.
  • Cities could improve traffic monitoring and routing, saving commuters’ time and fuel while increasing productivity.

Every interaction would be transparent and verifiable, providing more data to analyze for future improvements.


Welcome to the Next Industrial Revolution

When exponential technologies intersect and combine, transformation happens on a massive scale. It’s time to start thinking through outcomes in a disciplined, proactive way to prepare for a future we’re only just beginning to imagine.

Download the executive brief Running Future Cities on Blockchain.


Read the full article Pulling Cities Into The Future With Blockchain

Comments

About Dan Wellers

Dan Wellers is founder and leader of Digital Futures at SAP, a strategic insights and thought leadership discipline that explores how digital technologies drive exponential change in business and society.

Raimund Gross

About Raimund Gross

Raimund Gross is a solution architect and futurist at SAP Innovation Center Network, where he evaluates emerging technologies and trends to address the challenges of businesses arising from digitization. He is currently evaluating the impact of blockchain for SAP and our enterprise customers.

Ulrich Scholl

About Ulrich Scholl

Ulrich Scholl is Vice President of Industry Cloud and Custom Development at SAP. In this role, Ulrich discovers and implements best practices to help further the understanding and adoption of the SAP portfolio of industry cloud innovations.

Tags:

4 Traits Set Digital Leaders Apart From 97% Of The Competition

Vivek Bapat

Like the classic parable of the blind man and the elephant, it seems everyone has a unique take on digital transformation. Some equate digital transformation with emerging technologies, placing their bets on as the Internet of Things, machine learning, and artificial intelligence. Others see it as a way to increase efficiencies and change business processes to accelerate product to market. Some others think of it is a means of strategic differentiation, innovating new business models for serving and engaging their customers. Despite the range of viewpoints, many businesses are still challenged with pragmatically evolving digital in ways that are meaningful, industry-disruptive, and market-leading.

According to a recent study of more than 3,000 senior executives across 17 countries and regions, only a paltry three percent of businesses worldwide have successfully completed enterprise-wide digital transformation initiatives, even though 84% of C-level executives ranks such efforts as “critically important” to the fundamental sustenance of their business.

The most comprehensive global study of its kind, the SAP Center for Business Insight report “SAP Digital Transformation Executive Study: 4 Ways Leaders Set Themselves Apart,” in collaboration with Oxford Economics, identified the challenges, opportunities, value, and key technologies driving digital transformation. The findings specifically analyzed the performance of “digital leaders” – those who are connecting people, things, and businesses more intelligently, more effectively, and creating punctuated change faster than their less advanced rivals.

After analyzing the data, it was eye-opening to see that only three percent of companies (top 100) are successfully realizing their full potential through digital transformation. However, even more remarkable was that these leaders have four fundamental traits in common, regardless of their region of operation, their size, their organizational structure, or their industry.

We distilled these traits in the hope that others in the early stages of transformation or that are still struggling to find their bearings can embrace these principles in order to succeed. Ultimately I see these leaders as true ambidextrous organizations, managing evolutionary and revolutionary change simultaneously, willing to embrace innovation – not just on the edges of their business, but firmly into their core.

Here are the four traits that set these leaders apart from the rest:

Trait #1: They see digital transformation as truly transformational

An overwhelming majority (96%) of digital leaders view digital transformation as a core business goal that requires a unified digital mindset across the entire enterprise. But instead of allowing individual functions to change at their own pace, digital leaders prefer to evolve the organization to help ensure the success of their digital strategies.

The study found that 56% of these businesses regularly shift their organizational structure, which includes processes, partners, suppliers, and customers, compared to 10% of remaining companies. Plus, 70% actively bring lines of business together through cross-functional processes and technologies.

By creating a firm foundation for transformation, digital leaders are further widening the gap between themselves and their less advanced competitors as they innovate business models that can mitigate emerging risks and seize new opportunities quickly.

Trait #2: They focus on transforming customer-facing functions first

Although most companies believe technology, the pace of change, and growing global competition are the key global trends that will affect everything for years to come, digital leaders are expanding their frame of mind to consider the influence of customer empowerment. Executives who build a momentum of breakthrough innovation and industry transformation are the ones that are moving beyond the high stakes of the market to the activation of complete, end-to-end customer experiences.

In fact, 92% of digital leaders have established sophisticated digital transformation strategies and processes to drive transformational change in customer satisfaction and engagement, compared to 22% of their less mature counterparts. As a result, 70% have realized significant or transformational value from these efforts.

Trait #3: They create a virtuous cycle of digital talent

There’s little doubt that the competition for qualified talent is fierce. But for nearly three-quarters of companies that demonstrate digital-transformation leadership, it is easier to attract and retain talent because they are five times more likely to leverage digitization to change their talent management efforts.

The impact of their efforts goes beyond empowering recruiters to identify best-fit candidates, highlight risk factors and hiring errors, and predict long-term talent needs. Nearly half (48%) of digital leaders understand that they must invest heavily in the development of digital skills and technology to drive revenue, retain productive employees, and create new roles to keep up with their digital maturity over the next two years, compared to 30% of all surveyed executives.

Trait #4: They invest in next-generation technology using a bimodal architecture

A couple years ago, Peter Sondergaard, senior vice president at Gartner and global head of research, observed that “CIOs can’t transform their old IT organization into a digital startup, but they can turn it into a bi-modal IT organization. Forty-five percent of CIOs state they currently have a fast mode of operation, and we predict that 75% of IT organizations will be bimodal in some way by 2017.”

Based on the results of the SAP Center for Business Insight study, Sondergaard’s prediction was spot on. As digital leaders dive into advanced technologies, 72% are using a digital twin of the conventional IT organization to operate efficiently without disruption while refining innovative scenarios to resolve business challenges and integrate them to stay ahead of the competition. Unfortunately, only 30% of less advanced businesses embrace this view.

Working within this bimodal architecture is emboldening digital leaders to take on incredibly progressive technology. For example, the study found that 50% of these firms are using artificial intelligence and machine learning, compared to seven percent of all respondents. They are also leading the adoption curve of Big Data solutions and analytics (94% vs. 60%) and the Internet of Things (76% vs. 52%).

Digital leadership is a practice of balance, not pure digitization

Most executives understand that digital transformation is a critical driver of revenue growth, profitability, and business expansion. However, as digital leaders are proving, digital strategies must deliver a balance of organizational flexibility, forward-looking technology adoption, and bold change. And clearly, this approach is paying dividends for them. They are growing market share, increasing customer satisfaction, improving employee engagement, and, perhaps more important, achieving more profitability than ever before.

For any company looking to catch up to digital leaders, the conversation around digital transformation needs to change immediately to combat three deadly sins: Stop investing in one-off, isolated projects hidden in a single organization. Stop viewing IT as an enabler instead of a strategic partner. Stop walling off the rest of the business from siloed digital successes.

As our study shows, companies that treat their digital transformation as an all-encompassing, all-sharing, and all-knowing business imperative will be the ones that disrupt the competitive landscape and stay ahead of a constantly evolving economy.

Follow me on twitter @vivek_bapat 

For more insight on digital leaders, check out the SAP Center for Business Insight report, conducted in collaboration with Oxford Economics,SAP Digital Transformation Executive Study: 4 Ways Leaders Set Themselves Apart.”

Comments

About Vivek Bapat

Vivek Bapat is the Senior Vice President, Global Head of Marketing Strategy and Thought Leadership, at SAP. He leads SAP's Global Marketing Strategy, Messaging, Positioning and related Thought Leadership initiatives.