Data Wrangling And Democratization: CIOs Should Facilitate The Discovery Of Data

Gerry Brown

According to the New York Times in August 2014, data scientists spend 50% to 80% of their days “data wrangling” for the business, or in other words, preparing digital data before it can be leveraged for useful insights. A data wrangler would gather and organize data sets collected from various sources. In the meantime, line managers within many organizations are demanding and procuring tools to gain insights to understand and anticipate business performance. This so-called “data democratization” is allowing business managers to make quicker decisions in the short term, but in the long term it can have serious consequences for the information strategy. As data is scattered in silos across the organization, it becomes inconsistent, generates skepticism, and ends up being inadequate for analysis. It will ultimately be the responsibility of the CIO to ensure this does not happen.

When thinking about the use and discovery of data in this context, it is often immediately associated with Big Data analytics. However, data discovery is allowing organizations to identify new business models and establish innovation labs that can redefine how industries work. Integrating and extracting data, with robust processes for capture, curation, validation, retention, and disposal, and making it available to business users are critical steps in the move toward digital transformation. This can also be seen as the “datafication” of the business, quantifying (or visualizing) latent processes and activities – primarily driven by the urge to optimize – and developing business value to generate new income and higher returns.

Ericsson looks at datafication as it relates to the use of digital technologies to surface the knowledge associated with physical objects by extracting the data associated with them. For instance, real-estate organizations have the opportunity to improve how they classify districts by extracting details from open and business sources, allowing representatives to better locate their operations. In manufacturing, by using IoT, datafication can lead to improved product development by collecting feedback from products “in use,” and to new revenue streams by selling the data anonymously to other manufacturers or developers that are searching for ways to increase efficiencies across supply chains.

Most CIOs with Big Data practices have already moved beyond data warehouses and operational data stores to include some level of analysis and data discovery on unstructured data. However, to augment the business and enable digital transformation, CIOs should be looking to develop a data discovery environment that provides datasets from multiple sources and powerful toolsets to uncover value both for data scientists and business analysts, and beyond this at enterprise-wide utility data services covering all facets of the business ecosystem.

However, to move across these maturity stages, data governance often becomes an important obstacle because it is a broader organizational responsibility. It requires CIOs to work with business leaders to build a top-down approach to capture, protect, and share data on different levels across the organization. CIOs play an important role in establishing or supporting a data transformation and governance framework. To do this, a CIO may want to consider creating a chief data officer role. This role is still emerging and can be shaped in different ways depending on specific circumstances. While fewer than 30% of organizations have a chief data officer in place, a majority of organizations are assessing the need to create or expand this role. Some companies are even looking to create several chief data officer positions for different business groups.

Brown fig 1

The ability to leverage data will become a critical differentiator between organizations. IDC predicts that over the next few years global CIOs, especially those at highly regulated businesses like financial institutions and healthcare providers, will realize the importance of initiating a data transformation and governance framework that enables them to take maximum advantage of information while minimizing associated risks and costs. Organizations using a siloed approach that is based on individual departmental needs will not be able to manage the high demands of data management and usage. Organizations that evolve from a traditional data management approach to mastering a differentiated information value chain will gain business leadership and competitive advantage – and time is of the essence.

To meet the requirements of both the legacy environment and the digital business under construction, CIOs should start immediately by examining opportunities to leverage existing data and information initiatives. Focusing on meaningful but limited initiatives before investing in larger ventures is often a good way to start small. If it is not in place already, it will be critical to build a solid business case for information governance, and with this an information governance framework that makes business executives accountable for their data. Over time, it will be important for CIOs to evolve the information architecture, as well as the organizational culture, to adapt to new business requirements. Investments in Big Data, analytics, and data scientists, along with other talent needed to distill insight into monetary value, are important steps going forward; appointing a chief data officer will help accelerate this process even further.

IDC believes that the information strategy, and with that its data discovery initiatives, is a critical area of investment as organizations evolve their digital transformation road map. In order to understand the maturity of the organization as it relates to digital transformation, we recommend that CIOs leverage IDC’s MaturityScape Snapshot to identify the key areas to focus on.


Gerry Brown

About Gerry Brown

Gerry Brown has 10 years’ experience as a senior analyst and consultant in customer-oriented enterprise applications and services. His research coverage for IDC includes enterprise collaboration, digital transformation, customer experience, customer communications, and cross-channel engagement. Previously Gerry was a senior analyst for Ovum, where he specialized in digital marketing technologies, and at Bloor Research where he focused on analytics, performance management, and CRM. For seven years Gerry was a lecturer for the UK’s Chartered Institute of Marketing (CIM) on its Masters-level course in strategic marketing management.