Afterthoughts on Data Governance for BI

Why Business Intelligence needs a specific approach to data governance

During my talk at the Data Governance Conference, at least one of my audience was paying attention and asked me a pertinent question. “Why should you need a separate approach for data governance in Business Intelligence?”

My first reaction was “’Oops, I’ve skipped a few stadia in my introduction…” So here’s an opportunity to set things right.

Some theory, from the presentation

At  the conference, I took some time to explain the matrix below.

Data portfolio management as presented at the 2016 data governance conference in London

If you analyse the nature of the data present in any organisation, you can discern four major types.

Let’s take a walk through the matrix in the form of an ice cream producer.

Strategic Data: this is critical to future strategy development; both forming and executing strategy are supported by the data. By definition almost, strategic data are not in your process data or at best are integrated data objects from process data and/or external data. A simple example: (internal) ice cream consumption per vending machine matched with (external) weather data and an (external) count of competing vending machines and other competing outlets create a market penetration index which in its turn has a predictive value for future trends.

Turnaround Data: critical to future business success as today’s operations are not supported, new operations will be needed to execute. E.g.: new isolation methods and materials make ice cream fit for e-commerce. The company needs to assess the potential of this new channel as well as the potential cannibalizing effect of the substitute product. In case the company decides not to compete in this segment, what are the countermeasures to ward off the competition? Market research will produce the qualitative and quantitative data that need to be mapped on the existing customer base and the present outlets.

Factory Data: this is critical to existing business operations. Think of the classical reports, dashboards and scorecards. For example: sales per outlet type in value and volume, inventory turnover… all sorts of KPIs marketing, operations and finance want every week on their desk.

Support Data: these data are valuable but not critical to success. For instance reference data for vending locations, ice cream types and packaging types for logistics and any other attribute that may cause a nuisance if it’s not well managed.

If you look at the process data as the object of study in data governance, they fall entirely in the last two quadrants.

They contribute to decision making in operational, tactical and strategic areas but they do not deliver the complete picture as the examples clearly illustrate. There are a few other reasons why data governance in BI needs special attention, If you need to discuss this further, drop me a line via the Lingua Franca contact form.

Data Governance in Business Intelligence, a Sense of Urgency is Needed

The Wikipedia article on data governance gives a good definition and an overview of the related topics. But although you may find a few hints on how data governance impacts the business intelligence and analytics practice, the article is living proof that the link data governance with BI and Analytics is not really on the agenda of many organisations.

Sure, DAMA and the likes are reserving space in their Body of Knowledge for governance but it remains on the operational level and data  governance for analytics is considered a derived result from data governance for on line transaction processing (OLTP). I submit to you that it should be the other way around. Data governance should start from a clear vision on what data with which degree of consistency, accuracy and general quality measures to support the quality of the decision making process is needed. In a second iteration this vision should be translated into a governance process on the source data in the OLTP systems. Once this vision is in place, the lineage from source to target becomes transparent, trustworthy and managed for changes. Now the derived result is compliance with data protection, data security and auditability to comply with legislation like Sarbanes Oxley or the imminent EU directives on data privacy.

Two observations to make my point

Depending on the source, between 30 and 80 percent of all Business Intelligence projects fail. The reasons for this failure are manifold: setting expectations too high may be a cause but the root cause that emerges after thorough research is a distrust in the data itself or in the way data are presented in context and defined in their usability for the decision maker. Take the simple example of the object “Customer”. If marketing and finance do not use the same perspective on this object, conflicts are not far away. If finance considers anyone who has received an invoice in the past ten years as a customer, marketing may have an issue with that if 90 % of all customers renew their subscription or reorder books within 18 months.  Only clear data governance rules supported by a data architecture that facilitates both views on the object “Customer” will avoid conflicts.

Another approach: only 15 – 25 % of decision making is based on BI deliverables. On the plus side it may mean that 75 % of decision making is focused on managing uncertainty or nonsystematic risk which can be fine. But often it is rather the opposite: the organisation lacks scenario based decision making to deal with uncertainty and uses “gut feeling” and “experience” to take decisions that could have been fact based, if the facts were made available in a trusted setting.

Let’s spread the awareness for data governance in BI

More on this in my speech at the Data Governance Conference in London, the 17thMay

In the meantime,I’d like your opinion on this matter. It takes just three minutes to answer four questions. 

Many thanks in advance!

A Little Side Step into Norway: BI 2016 Forecast

The Norwegian leading blog on Business Intelligence published my view on BI in 2016.

Check the prognosis here and get back to me at the end of the year to see if my crystal ball was clear on at least a few issues.

A Small Presentation on Big Data

In eight minutes I make the connection between marketing, information management and Big Data to position the real value of it and separate it from the hype.
Click here for the presentation.

Wishing you a great 2014, where you will make decisions based on facts and data be successful in all your endeavours.

Kind regards,

bert

Interview with a Business Intelligence User

Let’s call him Eric. Because after the interview Eric decided he’d better remain anonymous. Some of his answers could cause too much controversy in the organization, a major European logistics company.

Eric is BI manager in this company and when listening to his vision, his worries and his concerns, it is like taking stock of the most common disconnects between IT and the business.

Question: What struck you the most when reading the book “Business Analysis for Business Intelligence”?

Eric: I think you have documented your book well and chose a useful starting point. Most literature in Business Intelligence (BI) is divided in two categories.  On one side you have a myriad of theoretical works on strategy and management,  performance management  and the inevitable scorecards and dashboards. On the other side are plenty technical publications available discussing IT performance and optimum data structures. What many of these books lack is a vision of how business and IT should join hands to produce optimum BI results. From my 20 years’ experience with BI, this is a serious problem.

Question: What are the major impediments for your performance as a BI manager?

Eric: I see three roadblocks: IT is either unaware or unwilling to admit that BI cannot be standardized. But the business itself is not always capable of producing crisp and consistent definitions to produce a coherent analytical frameworks changes its mind”. And last but not least: the complexity of some analytics also causes a lot of problems and is –of course- compounded by the two previous roadblocks.

Question: Why would IT not be aware of the need for flexibility? Some IT guys we know say stuff like “The business guys always change their mind”

Eric:  No, it’s not about business changing its mind because that can be prevented through thorough analysis as described in your book. It is more about the prejudice that BI solutions are templates you can use anywhere. IT people underestimate the uniqueness of each business process and its context, culture and informal issues that make every business unique. Management can shift its attention and rearrange its priority list in days and weeks. If IT can’t follow, the users look for ad hoc (and often badly architected) solutions.

Thank you for sharing this with  us, Eric. 

To our readers: don’t hesitate to share your experience with the gap between business and IT in BI. We can all learn from this!