Real-time BI, who would have thought that twenty years ago?

(Abstract)

Business Intelligence has come a long way. From the static, rigorous decision support systems to the agile BI architectures and the volatile Big Data stores... It has been a journey filled with success and failure, with never ending discussions about which architecture would provide a robust future proof delivery platform.

With social analytics as a new addition to real time BI, another discussion is opened: 

where and how is (near-) real time BI worth the investment?

And what aspects need to be considered to produce a balanced investment appraisal?

These are the topics I will address in this article which is published here.

A Small Presentation on Big Data

In eight minutes I make the connection between marketing, information management and Big Data to position the real value of it and separate it from the hype.
Click here for the presentation.

Wishing you a great 2014, where you will make decisions based on facts and data be successful in all your endeavours.

Kind regards,

bert

Download our Webinar slides from the ITMPI

We have posted the slides of two webinars on our Lingua Franca site for you to download:

How Business Analysis for Business Intelligence Creates Strategic Value

How to Keep Business Intelligence in Sync with Your Strategic Priorities

Send us your feedback, we look forward to reading your comments.

Business Intelligence has become too big to allow failure.

Four speakers between Ralph Kimball’s sessions, four topics and one unifying thought: BI is getting to big to allow failure. The first Business Analytics for All Insight session which took place in Brussels the 12th November gathered over 250 attendees to hear Ralph Kimball’s insights on the data warehouse design principles and how the Big Data phenomenon fits in this architecture. I gladly refer to the Kimball Group’s website with articles like these for his vision on Big Data.  

But between Ralph’s talks in the morning and in the afternoon, four other topics were discussed which all lead to the same conclusion: BI has become too big, too much of a strategic commitment to allow for sloppy business analysis and project management.

Annelies Aquarius, European BI Project manager from the Coca-Cola Company illustrated the anytime- anything-anywhere aspects of mobile BI. Jelle Defraye from Laco made a case for self service BI.  Jos Van Dongen from SAS taught us the basics from data visualisation and Guy Van der Zande from USG ICT Professionals explained why a well organised BI Competence Center (BICC) is essential to manage technology trends and changing business requirements.

For a full description of their talks we refer to the website.

It is time for proper BI business analysis and project management

Let me explain my point. With the growth of users, user types, data a lot of side effects have come into play since the early days of DSS where you offloaded a few tables to make reports for the CFO.

Exabytes of data flowing in at incredibly high speeds from a myriad of data sources in structured, semi-structured and structured formats need to be exploited by more people in a faster decision making cycle which is not limited to the strategic apex anymore. Thus the feedback loops become more complicated as the one-to-many relationship of top management and the workforce now becomes a many-to-many relationship between more and more decision making actors in the organisation. Self service BI, mobile BI and visualisation are all part of the solution and the problem if your organisation has no duopolistic governance from IT and the business. because both business processes and data management processes need to be mutually adjusted to allow for maximum return on investment . The alternative is chaos. So there you have the true value of  a well working BICC.

But to get there and to stay on that level, only a thorough business analysis process and the proper BI project management method will increase the success rate of business analytics. This success rate worries me. Because after twenty years in this business I am still seeing failure rates of 80% in BI. If we’d had the same rate of improvement in medicine as in BI we would still be using leeches and bleeding our patients regardless of the disease.

Interview with a Business Intelligence User

Let’s call him Eric. Because after the interview Eric decided he’d better remain anonymous. Some of his answers could cause too much controversy in the organization, a major European logistics company.

Eric is BI manager in this company and when listening to his vision, his worries and his concerns, it is like taking stock of the most common disconnects between IT and the business.

Question: What struck you the most when reading the book “Business Analysis for Business Intelligence”?

Eric: I think you have documented your book well and chose a useful starting point. Most literature in Business Intelligence (BI) is divided in two categories.  On one side you have a myriad of theoretical works on strategy and management,  performance management  and the inevitable scorecards and dashboards. On the other side are plenty technical publications available discussing IT performance and optimum data structures. What many of these books lack is a vision of how business and IT should join hands to produce optimum BI results. From my 20 years’ experience with BI, this is a serious problem.

Question: What are the major impediments for your performance as a BI manager?

Eric: I see three roadblocks: IT is either unaware or unwilling to admit that BI cannot be standardized. But the business itself is not always capable of producing crisp and consistent definitions to produce a coherent analytical frameworks changes its mind”. And last but not least: the complexity of some analytics also causes a lot of problems and is –of course- compounded by the two previous roadblocks.

Question: Why would IT not be aware of the need for flexibility? Some IT guys we know say stuff like “The business guys always change their mind”

Eric:  No, it’s not about business changing its mind because that can be prevented through thorough analysis as described in your book. It is more about the prejudice that BI solutions are templates you can use anywhere. IT people underestimate the uniqueness of each business process and its context, culture and informal issues that make every business unique. Management can shift its attention and rearrange its priority list in days and weeks. If IT can’t follow, the users look for ad hoc (and often badly architected) solutions.

Thank you for sharing this with  us, Eric. 

To our readers: don’t hesitate to share your experience with the gap between business and IT in BI. We can all learn from this!

Book Review Retail Analytics, The Secret Weapon by Emmett Cox

by Emmett Cox Wiley and SAS Business Series 2012 

The book cover opens with promising references and from Tom Davenport and all this on just 142 pages of 10 point Times new Roman with double spaced interlines. Wow! This needs further reading I guess. Chapter One gives a very high level introduction to the main process: sales and replenishment and chapter two introduces the ins and outs of retail data management. A diagram of the transaction log files could improve the didactic impact of this chapter and the comparison between denormalised and normalised data is at best superficial. But I guess the readers that, according to the editor are helped with their critical management decisions don’t bother much about technical mumbo jumbo, so let’s move on. Although this inspires me to publish an in depth blog on the pros and cons of these datamodels that might interest senior-level management. Because it is about money, time and quality and there is no one size fits all choice to be made.

Inspiring introduction

Cox sums up a few interesting case studies ranging from trade area modelling, site selection modelling, competitive threat analysis, merchandise mix modelling, marketing effectiveness tracking, brand analysis, clicks and mortar analysis, cross sell analysis to market basket analysis. And he does so in plain English which at least will trigger the interest of senior management. The schema on page 25 gives a quick and clear view on how the analytical domains are part of a chain driving important retail and merchandising decisions. While reading these case studies you are confronted with a very mild and unobtrusive form of product placement with sentences like “The statistical software company SAS has a strong set of utilities within its SAS Enterprise Miner that we used heavily” (p. 33) Yet, the case studies are very credible and well presented, even if they don’t go in depth into the statistical details. I guess the target group of this book will tell guys like me: “Find out how we can apply this method in our stores”. Chapter three pays attention to the specifics of the apparel industry and does so in a didactic manner, illustrating clearly the constraints apparel buyers have to work with: long order lead times, optimum price elasticity management, optimum sales promotion management,… Having worked in sales promotion for a mail order company doing 80% of its sales in fashion I recognised the stressful situations even the best retail analytics won’t attenuate. The fourth chapter spends quite a few pages on explaining what a GIS tool can do and how it works which is a bit redundant in the days of Google and Wikipedia, leaving only a few pages for the application of GIS in retail. For example: the combination of category management and GIS analytics can produce powerful results I am sure the author could illustrate to his readers. It is as if Mr. Cox apologises for this in the fifth chapter opening with “Chapter 4 was an extremely technical illustration of the GIS tool.” Er.. well,… I wasn’t bothered with points, vectors and surfaces of the GIS world, I just wanted to know more about the applications…. 

 A 101 on in store promotion 

 In-store marketing and presentation get a lot more attention with 31 of the 142 pages. Pricing strategies are explained very well as they are the meat of retail analytics. But Cox’ emphasis on convenience shopping hides its counterpart: fun in shopping which also needs tailor made analytical insights to enhance the profitability potential. His reference to DB2’s datablades brought me back to the heydays of Informix, once the leading RDBMS before Oracle displayed a superior sales and marketing strategy. All in all this chapter is a little too narrowly focused on the US situation. Phrases like “Most retail chains are now open 24 hours a day…” (p. 98) will raise eyebrows in Europe and sometimes the author gets lost in details like the description on how floor graphics can be removed “without leaving any residue behind”. I wonder who in the analytical community will bother about this. And when I read stuff like “another form of subliminal messaging” (p. 101) related to clearly detectable things like baking smells I get confused. “Subliminal” means a stimulus the audience is not aware of. And literature and research are not in unison about the effects of subliminal messages in stores. Chapter 6 handles the store operations, an underestimated source of information opportunities to be used; Think of workforce management where you optimise the match between labour cost and service level based on POS data and longitudinal time series. Think of differentiated messages to consumers, HVAC control, intrastore communication, replenishment: Cox explains these operations and the link with retail analytics in a clear and concise way. 

Introduction to loyalty marketing

Last but not at all least, the chapter on loyalty marketing. Since Reichheld and Sasser’s publications on customer loyalty we are aware that the link between excellence, customer satisfaction and loyalty is not always obvious and in many cases it is not even existent! Measuring and managing customer loyalty is a real pain in retail. To me, attracting customers to loyalty programs is not equal to creating customer loyalty. It is just a form of prolonged sales promotion that –if stopped- would cause an immediate drop in sales. True loyalty is quite different. It took me one and a half page to read about it: “(…) as long as the consumers do not just cherry-pick the discounts and rewards offers.” Spot on Emmett! Weeding out the cherry pickers is the true challenge for retail analytics. The author offers a checklist before you begin a loyalty program.

To conclude

This book will certainly deliver value for novices in retail marketing and analytics. If you're a CEO or CMO of a retail chain and wonder whether you should invest heavily in an analytical environment, you are probably too late as the competition will have obliterated your obsolete business.That doesn't mean other C-level executives couldn't benefit from this book.  But to be really relevant for other profiles like a CIO or COO, it should have related data and process management to analytics with diagrams and models. 

A Short Memo for Big Data Sceptics

In an article in the NY Times from 17th August by James Glanz, a few Big Data sceptics are quoted. Here is a literal quote: Robert J. Gordon, a professor of economics at Northwestern University, said comparing Big Data to oil was promotional nonsense. “Gasoline made from oil made possible a transportation revolution as cars replaced horses and as commercial air transportation replaced railroads,” he said. “If anybody thinks that personal data are comparable to real oil and real vehicles, they don’t appreciate the realities of the last century.” I respectfully disagree with the learned scholar: the new oil is a metaphor for how our lives have changed through the use of oil in transportation. Cars and planes have influenced our social lives immensely but why shouldn't Big Data do so in equal or even superior order? Let me name just a few: 

• Big Data reducing traffic jams (to stick close to the real oil world) 
• Big Data improving the product-market match to the level of one to one, tailoring product specifications and promotions to individual preferences, 
• Big Data improving diagnostics and treatments in health care combining the wisdom of millions of health care workers and logged events in diagnostics epidemiologic data, death certificates etc... 
• Big Data and reduction of energy consumption via the smart grid and Internet of things to automate the match between production and consumption, 
• Big Data in text mining to catch qualitative information on a quantitative scale improving the positioning of qualitative discriminants in fashion, music, interior decorating etc... and of course... politics Ask the campaign team from 44th President of the United States and they will tell you how Big Data oiled their campaign.

As soon as better tools for structuring and analysing Big Data become available and as soon as visionary analysts are capable of integrating Big Data in regular BI architectures the revolution will grow in breadth and depth. Some authors state that entirely new skills will be needed for this emerging market. If I were to promote training and education I'd say the same. But from where I stand today I think the existing technological skills in database and file management may need a little tweaking say a three or five day course but no way is there a need for an MBD (Master in Big Data) education. On the business side of things there may some need for explaining the works of semi structured and unstructured data and their V's which already add up to seven. I believe it is going in the same direction as the marketing P's where Kottler's initial four P's were upgraded to over thirty as one professor of marketing churned out this intellectual athletic performance. Let's sum them up and see if someone can top them: Volume: a relative notion as processing and storage capabilities increase over time Velocity: ibid. Variety: also a relative notion as EBCDIC, ASCII, UTF-8 etc... are now in the company of video and speech thanks to companies like Lernhout and Hauspie whatever the courts may have decided on their Language Development Companies, Volatility: I have added this one in an article you can find on the booksite on "Business Analysis for Business Intelligence" because what is true today may not be true tomorrow, so it is not about the time horizon you need to store the data as some authors claim because that will be defined by the seasonality. The problem with these data is there might not be any seasonality in them! Veracity: how meaningful are the data for the problem or opportunity at hand? Validity: meaning Big Data can only be useful if validated by a domain expert who can identify its usefulness. Value: what can we invest in recording, storing and analysing Big Data in return for what business value? This is one of the toughest questions today as many innovative organisations follow the Nike principle: "Just do it". And that, professor Gordon is just what all the pioneers did when they introduced the car and the aeroplane to their society, ignoring the anxious remarks from horse breeders and railroad companies. Remember how the first cars where slower than trains and horses? I rest my case.