Elections’ Epilogue: What Have We Learned?

First the good news: a MAD of 1.41 Gets the Bronze Medal of All Polls!

The results from the Flemish Parliament elections with all votes counted are:

Party	Results (source: Het Nieuwsblad)	SAM’s forecast
Christian democrats (CD&V)	20,48 %	18,70 %
Green (Groen)	8,7 %	8,75 %
Flemish nationalists (N-VA)	31,88 %	30,32 %
Liberal democrats (open VLD)	14,15 %	13,70 %
Social democrats (SP-A)	13,99 %	13,27 %
Nationalist & Anti-Islam party (VB)	5,92%	9,80%

Table1. Results Flemish Parliament compared to our forecast

And below is the comparative table of all polls compared to this result and the Mean Absolute Deviation (MAD) which expresses the level of variability in the forecasts. A MAD of zero value means you did a perfect prediction. In this case,with the highest score of almost 32 % and the lowest of almost six % in only six observations  anything under 1.5 is quite alright.

Table 2. Comparison of all opinion polls for the Flemish Parliament and our prediction based on Twitter analytics by SAM.

Compared to 16 other opinion polls, published by various national media our little SAM (Social Analytics and Monitoring) did quite alright on the budget of a shoestring: in only 5.7 man-days we came up with a result, competing with mega concerns in market research.

The Mean Absolute Deviation covers up one serious flaw in our forecast: the giant shift from voters from VB (The nationalist Anti Islam party) to N-VA (the Flemish nationalist party). This led to an underestimation of the N-VA result and an overestimation  of the VB result. Although the model estimated the correct direction of the shift, it underestimated the proportion of it.

If we would have used more data, we might have caught that shift and ended even higher!

Conclusion

Social Media Analytics is a step further than social media reporting as most tools nowadays do. With our little SAM, built on the Data2Action platform, we have sufficiently proven that forecasting on the basis of correct judgment of sentiment on even only one source like Twitter can produce relevant results in marketing, sales, operations and finance. Because, compared to politics, these disciplines deliver far more predictable data as they can combine external sources like social media with customer, production, logistics and financial data. And the social media actors and opinion leaders certainly produce less bias in these areas than is the -case in political statements. All this can be done on a continuous basis supporting day-to-day management in communication, supply chain, sales, etc...

If you want to know more about Data2Action, the platform that made this possible, drop me a line: contact@linguafrancaconsulting.eu 

Get ready for fact based decision making 

on all levels of your organisation

Business Intelligence has become too big to allow failure.

Four speakers between Ralph Kimball’s sessions, four topics and one unifying thought: BI is getting to big to allow failure. The first Business Analytics for All Insight session which took place in Brussels the 12th November gathered over 250 attendees to hear Ralph Kimball’s insights on the data warehouse design principles and how the Big Data phenomenon fits in this architecture. I gladly refer to the Kimball Group’s website with articles like these for his vision on Big Data.  

But between Ralph’s talks in the morning and in the afternoon, four other topics were discussed which all lead to the same conclusion: BI has become too big, too much of a strategic commitment to allow for sloppy business analysis and project management.

Annelies Aquarius, European BI Project manager from the Coca-Cola Company illustrated the anytime- anything-anywhere aspects of mobile BI. Jelle Defraye from Laco made a case for self service BI.  Jos Van Dongen from SAS taught us the basics from data visualisation and Guy Van der Zande from USG ICT Professionals explained why a well organised BI Competence Center (BICC) is essential to manage technology trends and changing business requirements.

For a full description of their talks we refer to the website.

It is time for proper BI business analysis and project management

Let me explain my point. With the growth of users, user types, data a lot of side effects have come into play since the early days of DSS where you offloaded a few tables to make reports for the CFO.

Exabytes of data flowing in at incredibly high speeds from a myriad of data sources in structured, semi-structured and structured formats need to be exploited by more people in a faster decision making cycle which is not limited to the strategic apex anymore. Thus the feedback loops become more complicated as the one-to-many relationship of top management and the workforce now becomes a many-to-many relationship between more and more decision making actors in the organisation. Self service BI, mobile BI and visualisation are all part of the solution and the problem if your organisation has no duopolistic governance from IT and the business. because both business processes and data management processes need to be mutually adjusted to allow for maximum return on investment . The alternative is chaos. So there you have the true value of  a well working BICC.

But to get there and to stay on that level, only a thorough business analysis process and the proper BI project management method will increase the success rate of business analytics. This success rate worries me. Because after twenty years in this business I am still seeing failure rates of 80% in BI. If we’d had the same rate of improvement in medicine as in BI we would still be using leeches and bleeding our patients regardless of the disease.