Most organisations (if not all) would like to create a data-driven culture where individuals and teams make decisions based on data. We now live in the Digital Age where rapid advancements in technology allow for a greater range of devices to collect data (e.g. The Internet-of-Things devices such as mobile phones, smartwatches and TVs etc.).

A result of this is that we are becoming more data-aware and the terms business intelligence, data science and data analytics are becoming more commonly used throughout organisations.

However, saying you would like to be more data-driven and being more data-driven are two very different things!

What is being Data-Driven?

A simplistic explanation is that data-driven is using data to make informed decisions.

Why should you be Data-Driven?

There are many reasons to be data-driven, but primarily it provides the opportunity to make decisions based on data, as opposed to assumption, hunches or gut instinct.

Data can provide amazing insights, and when coupled with experience and knowledge it can add value, whether that is assisting a clinician with personalised treatment for a patient, or, the creation of a marketing campaign geared to target particular demographics.

7 Steps to Becoming Data-Driven

I've been working with data for 15 years doing a variety of activities from collecting, validating, transforming and analysing. I've been doing this for personal gain, but also for multiple organisations. Based on my experience, these are some of the steps which I feel will help to build a data-driven organisation.

1. Top Management Buy-In

The creation of a data-driven organisation requires a change in how people interact with data. Change initiatives are often difficult because people are taken out of their comfort zone and placed into an environment or scenario with which they are unfamiliar. The same challenge applies when building a data-driven organisation.

The image above is taken from 'Making Change Work...While the Work Keeps Changing' by the IBM Institute for Business Value (2014). Top Management Sponsorship is regarded as the most important aspect of successful change.

To create a data-driven culture throughout an organisation, it must start at top-level management. This means staff at this level using data to make decisions, and their data-driven activities being seen and communicated throughout the organisation. For example, commence daily or weekly performance meetings with a 5 or 10-minute data-driven update for all stakeholders. Show key metrics, performance since a change initiative was launched, comparisons etc., and then use this data to drive the rest of the meeting and create actions based on data.

Behaviour breeds behaviour. If it can regularly be communicated through the organisation that top management is using data to make informed decisions, then this behaviour can be filtered down through other areas of the organisation with the help of data awareness and data literacy sessions.

It's important to note that pockets of an organisation can still become data-driven without guidance from top management. I've seen this happen and kudos to those individuals and teams, however, to scale this behaviour across an organisation it needs to start at the top.

2. Data Strategy

An evolving data strategy is critical for any organisation, regardless of size, that is attempting to create a data-driven culture. The strategy should include, but not limited to:

  • How the organisation can use data to achieve its objectives and outcomes.
  • List the benefits of using data, but also the challenges.
  • Assess the current working environment (people, skillsets, culture, data literacy levels, access to data, data quality and technology) and evaluate if it can deliver the organisation's data related outcomes.
  • Identify what is needed to address the shortcomings of the current working environment and then create a phased outcome-driven plan, broken down into small milestones, that will give the organisation the best opportunity to create a data-driven culture.

Without a data strategy, it will be difficult for an organisation to fully realise its data-driven potential. Instead, they will likely revert to an ad-hoc approach where silos of teams and individuals within the organisation will attempt to become data-driven. This often leads to problems, including resources not being used efficiently, lack of knowledge sharing, increasing costs and data quality issues.

3. Data Access and Availability

If an organisation wants to create a data-driven culture, then staff must have access to data.

A single data platform, with user authentication and authorisation applied, should be created to consolidate data from the various information systems within an organisation. This will help to create a Single Source of the Truth, reduce rogue datasets and also be the location that staff use to access data. This centralised approach to storing data should be encouraged throughout the organisation, however, due to the ease of access to desktop software such as Microsoft Excel, Microsoft Access, Libre Office etc. and the relatively shallow learning curve to use these products, it's difficult to stop employees creating their own datasets. This can become a problem as often the data is unregulated and shared.

Data availability is a complex area, particularly in organisations that have multiple information systems. Although a data-driven culture is the desired outcome, staff should never have access to data which they are not permitted to view.

Data governance plays a key role in the creation of a data-driven organisation and an area that I believe needs to be on board from the outset. This can help ease the concerns of data controllers as well as educating others in the organisation on the processing of data, privacy and security.

4. Data Analytic Technology

We are now living in the digital age, surrounded by data that can offer rich insights and intelligence. Nevertheless, for all of the technology advancements, many organisations still have valuable data stored within spreadsheets (e.g. Microsoft Excel) or systems that offer poor data interoperability. Even for organisations with minimal sources of information, the repetitive hours associated with manually cleaning and integrating data from these types of sources can be time-consuming and expensive. Data analytic and business intelligence software can help to overcome these problems with the ability to connect data from virtually anywhere, transform and integrate it, and then provide rapid user-driven intelligence.

There are a number of these software platforms on the market (e.g. Qlik Sense, Microsoft Power BI, Tableau etc.) and the image below, taken from the Gartner Magic Quadrant 2021 for Analytics and Business Intelligence Platforms, helps to identify the most commonly used.

The majority of the analytic vendors in the image above offer a variety of pricing options depending on the number of end-users. However, these platforms certainly aren't the only way to develop analytics and remove the data processing issues mentioned above. Previously I have developed custom-built data analytic software, using SQL, C#, ASP.Net MVC, JavaScript and CanvasJS, with successful outcomes. Other programming languages such as R and Python can be used to achieve a similar level of data processing and analytics without user licensing costs that are associated with many off the shelf analytic products. However, the learning curve is much steeper (in my opinion) than using software like Qlik Sense or Power BI etc.

5. Automation

It is often estimated that between 60% and 80% of the time spent on data analytic work is used cleaning and transforming the data ready for analysis. Often referred to as Extraction, Transformation and Loading (ETL), these tasks can be very time consuming if carried out manually, and can act as a barrier to becoming data-driven if they have to be executed regularly.

The majority of business intelligence and data analytic platforms (e.g. Qlik Sense, Microsoft Power BI etc.) offer automated ETL. The caveat with this approach is that any data transformation work carried out using these types of software is then often reliant and tied to the analytic platform of choice. Cleaning and transforming data can be a lengthy process, and if using a platform such as Qlik Sense etc., then it is likely that data transformation work can not be reused (without other proprietary software) in other analytic platforms.

There are alternative approaches to automated data ETL which are not tightly coupled to a particular type of software, however, regardless of the approach, I am a strong advocate of data automation where possible. It can hugely improve productivity, data quality and data security. Also, data automation offers the ability to integrate vast arrays of complex data from different datasets which can then provide innovative data discovery.

6. Data Quality

The importance of data quality can not be overstated. If the consumers of data do not have confidence in the quality of the data, then it can act as a barrier to creating a data-driven organisation.

The continued advancements in technology mean that greater amounts of data are being collected and communicated, and the appetite for using data to make improved decisions is growing. Whilst this behaviour should be encouraged, all data should be quality assured before it is used to aid decision making.

Every organisation should have a data quality operation, with increased resources for organisations that use distributed information systems. Technology, such as the data analytic and business intelligence platforms mentioned previously in this text, can be used to identify bad data and provide the opportunity to make it correct. However, greater awareness and collaboration regarding data quality is needed across an entire organisation. Staff should be educated on the importance of data quality, how it can benefit them and the consequences of bad data entered at the source.

7. Change Management

Creating a data-driven organisation requires behavioural and cultural change with how people interact and use data. Change can be complex, therefore, change management is required to prepare and support individuals and teams when making this organisational change.

Support can come in many forms, e.g. data awareness and data literacy sessions, change agents and a culture that motivates and promotes the use of data. This support should be constant and easily accessible, especially during the early stages of a venture to become data-driven, as it is usually during this period that an organisation can be faced with the most challenges.

A common challenge experienced by organisations during their early steps to become data-driven is the issue of data quality. If the data quality is poor, data consumers can quickly lose faith and revert to previous ways of working and decision making. However, this is why support and especially top management buy-in is so important, to keep staff motivated and continually promote a data-driven culture.

Summary

To become a data-driven organisation, collective dedication and commitment are required from staff, particularly top management. Before an organisation attempts to become data-driven, they need to understand and be clear on how they can use data to improve. The current landscape (staff, structures, skillsets, culture etc.) needs to be evaluated and then address the shortcomings. Technology also plays an important role, especially when vast amounts of distributed data is involved, however, people are what make changes work.

I hope this text has provided an interesting read. It is based on my experience of working with data. If you would like to discuss, please use the comments form below, or reach out to me on Twitter.