Posts tonen met het label Agile BI. Alle posts tonen
Posts tonen met het label Agile BI. Alle posts tonen

maandag 26 mei 2014

Elections’ Epilogue: What Have We Learned?

First the good news: a MAD of 1.41 Gets the Bronze Medal of All Polls!

The results from the Flemish Parliament elections with all votes counted are:

Party
 Results (source: Het Nieuwsblad)
SAM’s forecast
20,48 %
18,70 %
Green (Groen)
8,7 %
8,75 %
31,88 %
30,32 %
Liberal democrats (open VLD)
14,15 %
13,70 %
13,99 %
13,27 %
5,92%
9,80%

Table1. Results Flemish Parliament compared to our forecast

And below is the comparative table of all polls compared to this result and the Mean Absolute Deviation (MAD) which expresses the level of variability in the forecasts. A MAD of zero value means you did a perfect prediction. In this case,with the highest score of almost 32 % and the lowest of almost six % in only six observations  anything under 1.5 is quite alright.

Table 2. Comparison of all opinion polls for the Flemish Parliament and our prediction based on Twitter analytics by SAM.

Compared to 16 other opinion polls, published by various national media our little SAM (Social Analytics and Monitoring) did quite alright on the budget of a shoestring: in only 5.7 man-days we came up with a result, competing with mega concerns in market research.
The Mean Absolute Deviation covers up one serious flaw in our forecast: the giant shift from voters from VB (The nationalist Anti Islam party) to N-VA (the Flemish nationalist party). This led to an underestimation of the N-VA result and an overestimation  of the VB result. Although the model estimated the correct direction of the shift, it underestimated the proportion of it.
If we would have used more data, we might have caught that shift and ended even higher!

Conclusion

Social Media Analytics is a step further than social media reporting as most tools nowadays do. With our little SAM, built on the Data2Action platform, we have sufficiently proven that forecasting on the basis of correct judgment of sentiment on even only one source like Twitter can produce relevant results in marketing, sales, operations and finance. Because, compared to politics, these disciplines deliver far more predictable data as they can combine external sources like social media with customer, production, logistics and financial data. And the social media actors and opinion leaders certainly produce less bias in these areas than is the -case in political statements. All this can be done on a continuous basis supporting day-to-day management in communication, supply chain, sales, etc...
If you want to know more about Data2Action, the platform that made this possible, drop me a line: contact@linguafrancaconsulting.eu 

Get ready for fact based decision making 
on all levels of your organisation





dinsdag 21 mei 2013

Statement of BI EA principles


In my book site in support of Business Analysis for Business Intelligence  I have summed up seven principles guiding Business Intelligence Enterprise Architecture. These are the contents of the article:

1. Focus on people, not technology:
2. Perfection is the enemy of the good:
3. Iterative and incremental:
4. Build it first, then talk about it:
5. See the entire picture but use small increments:
6. Make it attractive: user centric and practical, no academic geek[1] gymnastics:
7. These are the deliverables:
·         Customer support for the enterprise architecture
·         A vision and plan to achieve that vision
·         A collection of models and documentation describing the architecture

Whether you agree or disagree, I look forward to reading your comments

My Agile BI Enterprise Architecture Manifesto


I have waited almost a quarter of a century to state these principles, because I kept looking for falsification of the agile paradigm in Business Intelligence architecture. The early gurus in data warehousing all talked about scoping and building the entire data warehouse before delivering any report or analysis to the customer. But  I soon found out that this wasn’t the way to successful BI projects. “Successful” meaning accepted and used by the customers even if the deliverables weren’t complete or 100 %  accurate: as soon as the users experienced significant improvement in the speed and accuracy of the analytics and the reports, they were willing to go along with the BI solutions and applications. So there I learned my first lesson: perfection is the enemy of the good. Later, when I absorbed the theories and practice of Ralph Kimball and his approach with conformed dimensions I learned a second lesson: if you can see the entire picture, you know where to start the first iteration. And you don’t have to worry about costly rework while delivering rapid answers to your customers’ questions. That was one of the reasons why I wrote the book Business Analysis for Business Intelligence: I was sure that BI analysts were not capable of seeing the entire picture: from strategy formation and formulation to a BI system that delivered value in all layers and corners of the organisation.
Later in my career, I worked together with agile software developers and it all became clear: I had grown up into an agile believer without knowing it. Agile Enterprise Architecture is about individuals and interactions, not tools and processes, not 3NF and multidimensional models but ball point sketches and pencil drawings that communicate, convince, align and present a vision for all parties concerned.
So without further ado, here is my agile BI Enterprise Architecture (BI EA) manifesto.

1. Focus on people, not on technology: 

Quality and speed of decision making is the BI EA driver, together with the strategic directions from the Board of Directors. The first argument is a persistent intrinsic motivator, the latter an exogenous hygienic factor, urging management and associates to comply with the strategic vision. Whenever you can use the mission and vision statement to promote BI EA you increase the success rate of your BI EA initiative. Why? Because Business Intelligence Enterprise Architecture is a social construct that can only survive if 80 % of the organisation adopts the construct.


2. Perfection is the enemy of the good: 

Do you throw away the menu if a starter or a main course is no longer available? Of course not. He who wants to wait for a complete picture will have to wait until the cows come home. An enterprise is a living body, adapting and responding to internal and external impulses. No doubt enterprise architecture will follow these movements in a structured and manageable way. It is better to adapt fast to changing technologies via high level design principles and structures to realign with reality  than elaborate the finest details of the architecture. That would be as absurd as an Enterprise Architect writing code. Trust your designers and builders to fill in the details where and when necessary.

3. Iterative and incremental:

The BI EA grows at the rhythm of acceptance and implementation possibilities. If BI EA remains PowerPoint ware, disconnected from everyday practice, its impact will dwindle before the last PPT is projected on the screen. This is also where I apply the 3x over 1/3x rule: for every piece of information you ask somebody, return him three times the value of his input. Demonstrating increased reliability and accuracy of reports and analytics will convince even the staunchest adversary to collaborate with the BI EA initiative. It is all about motivation and acceptance.



4. Build it first, then talk about it:

“I can only respond when I see the picture” is a common remark in BI EA exercises. This is my story of the Inuit and the palm tree. You can describe how a palm tree looks to an Inuit but you will never create a correct image of a palm tree in his mind until you have shown him a few photos of palm trees. If possible, bring along a small palm tree and he will be convinced there are such funny things as palm trees. That’s  why we need the dialectic approach, in small increments as the next principle states.
The clearer you can convey the concept, the more chance of initiating a meaningful dialogue about enterprise architecture.


5. See the entire picture but use small increments:

There is no excuse for creating chaos through wrongly interpreted Agile principles. You have to overview the entire canvass before picking the right increment and working it out in detail. Coherence/Consistency and Agile are two sides of the same coin. It is no coincidence that the Kimball method of conformed dimensions works perfect in all situations. Whether you need to deliver a point solution or an enterprise data warehouse, if you can do it right the first time because you know what is ahead, beyond the point solution, the BI initiative is in good hands.
Margy Ross from the Kimball Group makes a strong plea for the bus matrix as support for agile data warehouse and BI development which you can read here.


Project management for Business Intelligence: the Matrix knows
Example of a Bus Matrix, an architectural view on all dimensions and facts needed for analysis and reporting (Reprint from Business Analysis for Business Intelligence, CRC Press 2013)


6. Make it attractive: user centric and practical, no academic geek gymnastics:

This is the elevator pitch: better, faster and cheaper decision making because of:
Consistent data definitions that enable enterprise wide communication
Manageable architecture of process and related data, process and related decision making and timeliness, accuracy and relevance of the available information
So the first use interview will start with “in an ideal world, what would be the perfect information available for your decision making tasks/processes/bottlenecks?”
Deconstruction of the answers will lead to a user centric approach.


7.Agile BI Enterprise Architecture has three goals:


1. Customer support for the enterprise architecture:
Some professionals ague it can take time before you have convinced the key stakeholders but the alternative can be a lot worse. Think of the energy, time and cost of errors to reduce skunk works, outright dissidence and chaos leading to underperforming  and unmanageable BI infrastructure.
2. A vision and plan to achieve that vision:
Management is about offering perspective and goals that lead people in their everyday actions. But it is also about embedding an ideology, a corporate culture that provides guidance when unforeseen situations occur. That is the value of a BI EA vision, supported by the CEO and his strategic vision as expressed in meetings, documents and publications.
3. A collection of models and documentation describing the architecture:
This is the physical deliverable and its value is determined by the previous two goals and their level of user acceptance.
Better a simple sketch that is understood and accepted by the BI stakeholders than a fancy and comprehensive model and documentation with a sophisticated tool



Tools like ARIS, Enterprise Architect or MEGA will provide maximum added value when the BI stakeholders have contributed to the pencil drawings and accepted the consequences of the drawing.

Be advised that 90% of the work is people management, not just modelling and documentation. Only in extreme cases of dissidence may enforcement work, but “a man convinced against his will is of the same opinion still”.

The ultimate goal is to improve the quality and speed of the decision making process in the organisation allowing for optimum return on investment because alignment between business and information management will tear down the BU walls and departmental silos where local optimization may result in suboptimal group results.