tag:blogger.com,1999:blog-45204385346406990012024-03-23T11:02:55.740+01:00Business Analysis for Business Intelligence Blog: BA4BIBlogThoughts on business intelligence and customer relationship management as customer analytics need process based support for meaningful analysis.Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.comBlogger74125tag:blogger.com,1999:blog-4520438534640699001.post-20217042559078272982024-02-02T19:00:00.001+01:002024-02-02T20:38:08.090+01:00Geospatial Data Warehouse and Spatially Enabled Data Warehouse: Turf Wars or Symbiosis?<p><i>Over the course of more than 25 years, I have been involved in numerous discussions between the GIS buffs and my tribe, the data warehouse team over where geospatial information should reside. </i></p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0Z2t9g327FDl0OviCNhyphenhyphen7sgKxj4j6clFPN3XcpHW8QCNy6AuVYojfov1F1qrb1Fk72mpDL5IOyqVCU6EuSKhqvQrjWn3rXuuJqdTYXEUkihk6_wdshsAk-iJY5Cjm7QfxBmI013kN-ysv58GS3oifoBtbrSZ2Uz-gdtGEYgk83uNOOoWcjXPRNcqlmIY/s700/Narnia.JPG" style="margin-left: auto; margin-right: auto;"><img alt="Narnia map" border="0" data-original-height="626" data-original-width="700" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0Z2t9g327FDl0OviCNhyphenhyphen7sgKxj4j6clFPN3XcpHW8QCNy6AuVYojfov1F1qrb1Fk72mpDL5IOyqVCU6EuSKhqvQrjWn3rXuuJqdTYXEUkihk6_wdshsAk-iJY5Cjm7QfxBmI013kN-ysv58GS3oifoBtbrSZ2Uz-gdtGEYgk83uNOOoWcjXPRNcqlmIY/s16000/Narnia.JPG" title="A map of Narnia, what geospatial information should we link to that map?" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">A geospatial map of Narnia. What attributes should reside in the geospatial system?</td></tr></tbody></table><i><br /></i><p></p><p>Since almost all measures have a location aspect, the spatial data warehouse was promoted as the single source of truth, able to visualize data in an unparalleled way whereas the opponents stated that all you need is to define a good location dimension and the data warehouse could do without the expensive software and the scarce resources in the geospatial domain. </p><p>I will spare you the avalanche of technical arguments back and forth between the two, leading to tugs of war between the teams and I propose an approach from the business user’s point of view.</p><p>The essential question is: “What information am I looking for?” Is it about one or more measures that need to be put in context using a majority of dimensions outside the geospatial domain, even if it includes DimLocation or is it exclusively related to questions “What happened or happens in this particular location, i.e. at this point, line or polygon?” , “What are the measures within a radius of point (x,y) on the map?” or “What is the intersect between location A and location B as far as measure Z is concerned?”.</p><p>It is clear that in the first case, the performance and cost of a classical data warehouse with a location dimension will prove to be the better choice. But if location is the point of entry to a query, then the spatial data warehouse is the smartest tool in the shed. </p><h4 style="text-align: left;">Symbiosis is the way forward</h4><p>There are many reasons why the two environments make sense. For executive and managerial information based on structured data, the data warehouse has proven to be the platform of choice and will continue to do so. For location based analysis, the geospatial data warehouse outperforms the latter. At the same time it is much closer to operational analytics and it can even be a part of operational applications like CRM, SCM or any other OLTP system.</p><p></p><p>To enable symbiosis, the location dimension needs some connection to the geospatial system. Some plead for a simple snapshot of a shapefile, some want a full duplication of all geospatial data and their timestamp. The latter may lead to an avalanche of data as any little correction of the shape on the GIS system will send new time stamped data. This can’t be a workable situation. Either the snapshot ignores updates but takes in the original GIS object ID to secure a trace or it overwrites any location data and keeps the last version as an active one. Because the only objective here is to provide a path to analysts who need a deeper geospatial analysis of one or more measurements registered in the data warehouse. </p><h4 style="text-align: left;">Let's open the debate</h4><div>I am sure I have missed a few points here and there. Let me know your position on this issue via the comments or a personal message via the contact form on our website: https://www.linguafrancaconsulting.eu/ </div><p style="text-align: left;"><a href="https://www.linguafrancaconsulting.eu/ict-focus-areas-for-board-and-managers" target="_blank">This is one of the topics of our course "ICT Focus Areas for Board Members & Management"</a></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgo3T-Rb_an2aDKAhRd2txzR_jRHVIqro6mpnd8E89LAKXgjXwY6Ym6HWWjDf9ojUmwQ2VbSvUSapV9qGkku8VaFcvuckJB4um6w834Jl2rhnZvAzGtObKwk_ZoZhyphenhyphenAcl4Y9u_3qFcKecLieCImRv_cL6XWFk36RCckScJjPGzmbD9Vlbl0DocXFkSut7c/s911/BABlogAntwerp.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="450" data-original-width="911" height="317" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgo3T-Rb_an2aDKAhRd2txzR_jRHVIqro6mpnd8E89LAKXgjXwY6Ym6HWWjDf9ojUmwQ2VbSvUSapV9qGkku8VaFcvuckJB4um6w834Jl2rhnZvAzGtObKwk_ZoZhyphenhyphenAcl4Y9u_3qFcKecLieCImRv_cL6XWFk36RCckScJjPGzmbD9Vlbl0DocXFkSut7c/w640-h317/BABlogAntwerp.JPG" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><div><br /></div><div><br /></div><div><br /></div>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-82352855053156050222023-11-27T06:30:00.003+01:002023-11-27T09:55:06.275+01:00Governing the Data Ingestion Process<p>“Data lakehousing” is all about good housekeeping
your data. There is, of course, room for ungoverned data which are in a
quarantine area but if you want to make use of the structured and especially
the semi structured and unstructured data you’d better govern the influx of
data before your data lake becomes a swamp producing no value whatsoever.</p>
<h4 style="text-align: left;"><span lang="EN-GB">Three data flavours need three different
treatments</span></h4>
<p class="MsoNormal"><span lang="EN-GB"><b>Structured data</b> are relatively easy to
manage: profile the data, look for referential integrity failures, outliers,
free text that may need categorising etc… In short: harmonise the data with the
target model which can be one or more unrelated tables or a set of data marts
to produce meaningful analytical data.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><b>Semi structured data</b> demand a data pipeline
that can combine the structured aspects of clickstream or log files analysis
with the less structured parts like search terms. It also takes care of
matching IP addresses with geolocation data since ISPs sometimes sell blocks of
IP ranges to colleagues abroad.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><b>Unstructured data</b> like text files from
social media, e-mails, blogposts, document and the likes need more complex
treatment. It’s all about finding structure in these data. Preparing these data
for text mining means a lot of disambiguation process steps to get from text
input to meaning output:<o:p></o:p></span></p>
<p class="MsoNormal"></p><ul style="text-align: left;"><li><span lang="EN-GB">Tokenization of the input is the process of
splitting a text object into smaller chunks known as tokens. These tokens can
be single words or word combinations, characters, numbers, symbols, or n-grams.</span></li><li><span lang="EN-GB">Normalisation of the input: separating prefixes
and/or suffixes from the morpheme to become the base form, e.g. unnatural ->
nature</span></li><li><span lang="EN-GB">Reduce certain word forms to their lemma,
e.g. the infinitive of a conjugated verb</span></li><li><span lang="EN-GB">Tag parts of speech with their grammatical
function: verb, adjective,..</span></li><li><span lang="EN-GB">Parse words as a function of their position
and type</span></li><li><span lang="EN-GB">Check for modality and negations: “could”, “should”,
“must”, “maybe”, etc… express modality</span></li><li><span lang="EN-GB">Disambiguate the sense of words: “very” can
be both a positive and a negative term in combination with whatever follows</span></li><li><span lang="EN-GB">Semantic role labelling: determine the
function of the words in a sentence: is the subject an agent or the subject of
an action in “I have been treated for hepatitis B”? What is the goal or the
result of the action in “I sold the house to a real estate company”?</span></li><li><span lang="EN-GB">Named entity recognition: categorising text
into pre-defined categories like person names, organisation names, location
names, time denominations, quantities, monetary values, titles, percentages,…</span></li><li><span lang="EN-GB">Co-reference resolution: when two or more expressions
in a sentence refer to the same object: “Bert bought the book from Alice but
she warned him, he would soon get bored of the author’s style as it was a
tedious way of writing.” In this sentence, “him” and “he” refer to “Bert”, “she”
refers to “Alice” while “it” refers to “the author’s style”.</span></li></ul><p></p>
<h4 style="text-align: left;"><span lang="EN-GB">What architectural components support these
treatments?</span></h4>
<p class="MsoNormal"><span lang="EN-GB">The first two data types can be handled
with the classical Extract, Transform and Load or Extract, Load and Transform
pipelines, in short: ETL or ELT. We refer to ample documentation about these
processes in the footnote below[1].<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">But for processing unstructured data, you
need to develop classifiers, thesauri and ontologies to represent your “knowledge
inventory” as reference model for the text analytics. This takes up a lot of
resources and careful analysis to make sure you come up with a complete, yet
practical set of tools to support named entity recognition. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">The conclusion is straightforward: the less
structure predefined in your data, the more efforts in data governance are
needed.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ4NeOZRx7DWN6CD8aCzRlc5QEXLk0XBPMJCC31C7l6cuEmOzbpXtE1HBwnuDBNHiwsc9y9QM1EBulxUHP4UlctXGXwNKkr-sdovcL2IuvPW_J9XzVUvF7FTBziz-wn_ih7IuzUC8DXpU8MfsgwSb26tgUIjiuvTZgjxcoJI-t-_lnnhD-vlZDJ18qUkQ/s1567/Thesaurus.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="874" data-original-width="1567" height="357" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ4NeOZRx7DWN6CD8aCzRlc5QEXLk0XBPMJCC31C7l6cuEmOzbpXtE1HBwnuDBNHiwsc9y9QM1EBulxUHP4UlctXGXwNKkr-sdovcL2IuvPW_J9XzVUvF7FTBziz-wn_ih7IuzUC8DXpU8MfsgwSb26tgUIjiuvTZgjxcoJI-t-_lnnhD-vlZDJ18qUkQ/w640-h357/Thesaurus.jpg" width="640" /></a></div><div style="text-align: center;">An example of a thesaurus metamodel</div><p></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span> </p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">[1] Three reliable sources, each with their nuances and perspectives on ETL/ELT:</span></p><p class="MsoNormal"><span lang="EN-GB">https://aws.amazon.com/what-is/etl/<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">https://www.ibm.com/topics/etl</span></p><p class="MsoNormal"><span lang="EN-GB">https://www.snowflake.com/guides/what-etl</span></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-9927976028865012642023-11-18T17:20:00.001+01:002023-11-28T09:42:09.868+01:00Best Practices in Defining a Data Warehouse Architecture<p class="MsoNormal"><span lang="EN-GB"><i>This blogpost is part of a series of which
the following posts have been published:<o:p></o:p></i></span></p><p class="MsoNormal"><span lang="EN-GB"><i><a href="https://ba4bi.blogspot.com/2023/10/why-data-governance-is-here-to-stay.html" target="_blank">The opening statement</a><o:p></o:p></i></span></p><p class="MsoNormal"><span lang="EN-GB"><i><a href="https://ba4bi.blogspot.com/2023/10/a-data-mesh-z-hamak-dehgani-cornered.html" target="_blank">What is a data mesh?</a><o:p></o:p></i></span></p><p><span face="Calibri, sans-serif" style="font-size: 11pt;"><i>
</i></span></p><p class="MsoNormal"><span lang="EN-GB"><i><a href="https://ba4bi.blogspot.com/2023/11/start-with-defining-coherent-business.html" target="_blank">Coherent business concepts keep the data relevant</a></i><o:p></o:p></span></p><p><span face="Calibri, sans-serif" style="font-size: 11pt;">In
any data mesh architecture, the data warehouse is and will be a critical
component for many reasons. First and foremost: some analytics need
industrialised solutions, automating the entire flow from raw data tot finished
reports. Structured data will always contribute to the analytical environment
and will need a relational model to provide the foundation for analyses. In my
experience, the most flexible and sustainable model is the process based star
schema architecture from Ralph Kimball. <a href="https://ba4bi.blogspot.com/2022/12/Data-structure-follows-strategy.html " target="_blank">In</a></span><a href="https://ba4bi.blogspot.com/2022/12/Data-structure-follows-strategy.html " target="_blank"><span face="Calibri, sans-serif" style="font-size: 11pt;">
</span><span face="Calibri, sans-serif" style="font-size: 11pt;">one of my previous posts I have made the case for this approach.</span></a></p><p><span face=""Calibri",sans-serif" style="font-size: 11pt; line-height: 107%; mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> <span lang="EN-GB">And in the context of a data
lake project<a href="https://ba4bi.blogspot.com/2021/06/managing-data-lake-project-part-iii.html" target="_blank"> I positioned the Kimball approach as the best in class</a></span></span></p><p><span face=""Calibri",sans-serif" style="font-size: 11pt; line-height: 107%; mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"></span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="font-size: 11pt; line-height: 107%; mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><span lang="EN-GB">The process diagram below tells the story
of requirements gathering, ingesting all sorts of data in the lake and making
the distinction between structured and unstructured data. Identifying the
common dimensions and facts is crucial to make the concept work. Either you
provide an increment to an existing data mart bus or you introduce a new
process metrics fact table with foreign keys from existing and new dimensions. <o:p></o:p></span></span></p><span face=""Calibri",sans-serif" style="font-size: 11pt; line-height: 107%; mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><br /></span><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPNdkIywVk4gXXrso_swDj2yI9yvd75ykFljUUJi1XvfWBRgd9UUVYPokkDYlnZww4BKeAGeIN_BQk-NtMkCDH7XXrxd7pix7R5ZJr8XBietS_WxdKRlW0xtlDYoRSz-oN1cMeOj9-aqFF7zHttbMV4xjKfbrc4aHC-bWPiT0uhIQQwUplCAVoUKYM1Vk/s605/BestPracticesDWH.jpg" style="margin-left: auto; margin-right: auto;"><img alt="Best practices in DWH" border="0" data-original-height="261" data-original-width="605" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPNdkIywVk4gXXrso_swDj2yI9yvd75ykFljUUJi1XvfWBRgd9UUVYPokkDYlnZww4BKeAGeIN_BQk-NtMkCDH7XXrxd7pix7R5ZJr8XBietS_WxdKRlW0xtlDYoRSz-oN1cMeOj9-aqFF7zHttbMV4xjKfbrc4aHC-bWPiT0uhIQQwUplCAVoUKYM1Vk/s16000/BestPracticesDWH.jpg" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Managing structured and unstructured data in a data mesh environment</td></tr></tbody></table><br /><p></p><p></p><h4 style="text-align: left;"><span lang="EN-GB">Making the case for the data warehouse as
an endpoint of unstructured analysis</span></h4><p class="MsoNormal"><span lang="EN-GB">A lot of advanced analytics can be
facilitated by the data lake. Think of text analytics, social media analytics
and image processing. The outcomes of these analyses may find their way to the
data warehouse. For example: polarity analysis in social media. Imagine a bank
or a telecom provider capturing the social media comments on its performance.
As we all know from customer feedback analysis, only the emotions two or three
sigma away from the mean make it to social media. The client is either very
satisfied or very dissatisfied and wants the world to know. Taking snapshots of
the client’s mood and relating it to his financial or communication behaviour may
yield interesting information. Already today, some banks are capturing their
client’s mood to determine the optimum conditions to present their services.
Aggregating these data may even provide macro-economic data correlating with
the business cycle. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Have a look at the diagram below and
imagine the business questions it can answer for you.<o:p></o:p></span></p><p class="MsoNormal"></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUpOWgFQ17PvahLLHO0krrbDpP3AFLK7mamxvnxi3ogjsExMpjm6-ZX9rpT7u7sU8fF_VOv8Vc_7bsJyMYr10MFXzn9xoKxp6BPgXWTAfTsDlYw4zEaY5ZesJiHU3kgQId5n6DE4wcCvYYZ_yrmBnvYpoaV7umUedk2k6Vxfpd9uXrHaXhlsEUNJSOvms/s708/DWH%20example.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="625" data-original-width="708" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUpOWgFQ17PvahLLHO0krrbDpP3AFLK7mamxvnxi3ogjsExMpjm6-ZX9rpT7u7sU8fF_VOv8Vc_7bsJyMYr10MFXzn9xoKxp6BPgXWTAfTsDlYw4zEaY5ZesJiHU3kgQId5n6DE4wcCvYYZ_yrmBnvYpoaV7umUedk2k6Vxfpd9uXrHaXhlsEUNJSOvms/s16000/DWH%20example.jpg" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">A high level star schema integrating social messages and their polarity with sales metrics</td></tr></tbody></table><span lang="EN-GB"><br /></span><p></p><p class="MsoNormal"><span lang="EN-GB">Think of time series: is there a some form
of a leading indicator of sales in the polarity of this customer’s social
messages?<o:p></o:p></span></p><p></p><p class="MsoNormal"><span lang="EN-GB">If one of our products is the subject of a
social media post, has this any (positive or negative) effect on sales of that
particular product?<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">What social media sources have the greatest
impact on our brand equity?<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">I am sure you will add your dimensions and
business questions to the model. And by doing so you are realising one of the
main traits of a data mesh: delivering data as a product.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">
</span></p><p class="MsoNormal"><span lang="EN-GB">I hope I have made my point clear: even in
the most sophisticated data lakehouse supporting a data mesh architecture, the
data warehouse is not going away.<o:p></o:p></span></p><p></p><p class="MsoNormal"><span lang="EN-GB">In the next blog article we will focus on
<a href="https://ba4bi.blogspot.com/2023/11/governing-data-ingestion-process.html" target="_blank">governing the data ingestion process</a>. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Stay tuned!<o:p></o:p></span></p><br /><p></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-34947223504121004062023-11-11T13:25:00.007+01:002023-11-26T11:20:12.516+01:00 Start with Defining Coherent Business Concepts<p style="text-align: left;"><span style="font-size: large; font-weight: normal;">Below is a diagram describing the governance process of defining and implementing business concepts in a data mesh environment. The business glossary domain is the user facing side of a data catalogue whereas the data management domain is the backend topology of the data catalogue. It describes how business concepts are implemented in databases, whether in virtual or persistent storage.</span></p><p><span style="font-size: medium;">But first and foremost: it is the glue that holds any dispersed data landscape together. If you can govern the meaning of any data model, any implementation of concepts like PARTY, PARTY ROLE, PROJECT, ASSET and PRODUCT to name a few, the data can be anywhere, in any form but the usability will be guaranteed. Of course, data quality will be a local responsibility in case global concepts need specialisation to cater for local information needs.</span></p><div><br /></div><div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjEEm2LSer87ve96zw4_MPeqAJGmAZoqHfZ5dZqOqN40ek9FsOq8AsJxIp1UhnzcElO1tRiHE4CXSIcNfnCvDGdHUyoHTEY3I1GyV9v_qBstVYHEZ7gjcqrKFgXGXC2XOW09od6p8dJ_lUU2s1csyEhnJvuwR-oq7g2AoUsKB-uYgYCG3e8KxZadc74hE/s605/DefineBizConcept.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="370" data-original-width="605" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjEEm2LSer87ve96zw4_MPeqAJGmAZoqHfZ5dZqOqN40ek9FsOq8AsJxIp1UhnzcElO1tRiHE4CXSIcNfnCvDGdHUyoHTEY3I1GyV9v_qBstVYHEZ7gjcqrKFgXGXC2XOW09od6p8dJ_lUU2s1csyEhnJvuwR-oq7g2AoUsKB-uYgYCG3e8KxZadc74hE/w697-h426/DefineBizConcept.jpg" width="697" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span face="Arial, Tahoma, Helvetica, FreeSans, sans-serif" style="background-color: white; font-size: 10.4px;">Business perspective on defining and implementing a business concept for a data mesh</span></td></tr></tbody></table><h3 style="text-align: left;"><span lang="EN-GB"><span style="font-size: large;">FAQ on this process model</span></span></h3><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>Why does the process owner initiate the
process?</b><o:p></o:p></span></span></p>
<p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">The reason is simple: process owners have a
transversal view on the enterprise and are aware they organisation needs
shareable concepts.<o:p></o:p></span></span></p>
<p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>Do we still need class definitions and
class diagrams in data lakehouses?</b><o:p></o:p></span></span></p>
<p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">Yes, since a great deal of data is still in
a structured ”schema on write” form and even unstructured or “schema on read”
data may benefit from a class diagram creating order in and comprehension from the underlying data. Even
streaming analytics use some tabular form to make the data exploitable.<o:p></o:p></span></span></p>
<p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>What is the role of the taxonomy editor?</b><o:p></o:p></span></span></p>
<p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">He or she will make sure the published
concept is in synch with the overall knowledge categorisation, providing “the
right path” to the concept.<br /></span><o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>Is there always need for a physical data
model?</b></span></span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">
</span></span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">Sure, any conceptual data model can be
physically implemented via a relational model, a NoSQL model in any of the
flavours or a native graph database. So yes, if you want complete governance
from business concept to implementation, the physical model is also in scope. </span><o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>Any questions you might have?</b></span></span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">Drop me line or reply in the comments.</span></span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">The next blog article <i><a href="https://ba4bi.blogspot.com/2023/11/best-practices-in-defining-data.html" target="_blank">Best Practices in Defining a Data Warehouse Architecture</a> </i>will focus on the place of a data warehouse in a data mesh.</span></span></p></div><p style="text-align: left;"><br /></p><br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-14722517628100404652023-10-31T14:24:00.005+01:002023-11-11T13:26:22.596+01:00 Defining a Data Mesh<h1 style="text-align: left;"><span style="font-weight: normal;"><i><span style="font-size: large;">Z</span><span style="font-size: large;">hamak Dehgani cornered the concept of a
data mesh in 2019. The data mesh is characterised by four important aspects:</span></i></span></h1><p class="MsoNormal"></p><ul style="text-align: left;"><li><span lang="EN-GB"><span style="font-size: medium;"><i>Data is organised by business domain;</i></span></span></li><li><span lang="EN-GB"><span style="font-size: medium;"><i>Data is packaged as a product, ready for
consumption;</i></span></span></li><li><span lang="EN-GB"><span style="font-size: medium;"><i>Governance is federated</i></span></span></li><li><span lang="EN-GB"><span style="font-size: medium;"><i>A data mesh enables self-service data
platforms.</i></span></span></li></ul><p></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;">
</span></span></p><!--[if !supportLists]--><p></p>
<p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>B</b>elow is an example of a data mesh architecture.
The HQ of a multinational food marketer is responsible for the global
governance of customers (i.e. retailers and buying organisations), assets (but
limited to the global manufacturing sites), products (i.e. the composition of
global brands) and competences that are supposed to be present in all
subsidiaries. </span></span></p><p class="MsoNormal"><span style="font-size: medium;"><span lang="EN-GB"><o:p></o:p></span><b>T</b>he metamodels are governed at the HQ and
data for the EMEA Branch are packaged with all the necessary metadata needed
for EMEA Branch consumption. These data products are imported in the EMEA Data Mesh
where they will be merged with EMEA level data on products (i.e. localised and
local brands), local competences, local customer knowledge and local assets
like vehicles, offices…</span></p><p class="MsoNormal"><span lang="EN-GB"><o:p></o:p></span></p>
<br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDtlNXTKW99JuiY8Sjsh1ridLqQRCscA12gKSGlwUgW4hg0jakYauNDaqt5akCgTmM2PD6VQYvC6DlnKPhLk4CFsXXGGPl6YD-iLQdMneJslKid1bJYgjAqAYn2jcP98eYXWRNnqq897x48LPcUcftJjWwufGu6I3LEKiI0d2fDHc8_KQPGDJQ5IyRurY/s605/Mesh%20Voorbeeld.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="341" data-original-width="605" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDtlNXTKW99JuiY8Sjsh1ridLqQRCscA12gKSGlwUgW4hg0jakYauNDaqt5akCgTmM2PD6VQYvC6DlnKPhLk4CFsXXGGPl6YD-iLQdMneJslKid1bJYgjAqAYn2jcP98eYXWRNnqq897x48LPcUcftJjWwufGu6I3LEKiI0d2fDHc8_KQPGDJQ5IyRurY/w640-h360/Mesh%20Voorbeeld.jpg" width="640" /></a></p></td></tr><tr><td class="tr-caption" style="text-align: center;"><p>Example of a data mesh architecture, repackaging data from the HQ Domains into an EMEA branch package</p><div style="text-align: left;"><p class="MsoNormal"><span style="font-size: medium;"><b>T</b>he data producer’s domain knowledge and place in the organisation enables the domain experts to set data governance policies focused on business definitions, documentation, data quality, and data access, i.e. information security and privacy. This “data packaging” enables self-service use across an organisation.</span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>T</b>his federated approach allows for more flexibility compared to central, monolithic systems. But this does not mean traditional storage systems, like data lakes or data warehouses cannot be used in a mesh. It just means that their use has shifted from a single, centralized data platform to multiple decentralized data repositories, connected via a conceptual layer and preferably governed via a powerful data catalogue.</span></span></p><p class="MsoNormal"><span lang="EN-GB"><span style="font-size: medium;"><b>T</b>he data mesh concept is easy to compare to microservices helping business audiences understand its use. As this distributed architecture is particularly helpful in scaling data needs across complex organizations like multinationals, government agencies and conglomerates, it is by no means a useful solution for SME or even larger companies that sell a limited range of products to a limited type of customers.<o:p></o:p></span></span></p><p class="MsoNormal"><span style="font-size: medium;"><span lang="EN-GB"><b>I</b>n the next blog </span><i><a href="https://ba4bi.blogspot.com/2023/11/start-with-defining-coherent-business.html" target="_blank">Start with defining coherent business concepts</a> </i>we will illustrate a data governance process, typical for a data mesh architecture. </span></p><p class="MsoNormal"><span lang="EN-GB">…</span></p></div></td></tr></tbody></table>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-27544621870752939552023-10-24T13:49:00.006+02:002023-11-08T10:36:39.297+01:00Why Data Governance is here to stay<p><i style="font-size: large;">More than a fairly stable Google Trend
Index, proving that Data Governance issues won’t go away is the fact that “Johnny-come-lately-but-always-catches-up-in-the-end”
Microsoft is seriously investing in its data governance software. After letting
the playing field for innovators like <a href="https://www.ataccama.com/" target="_blank">Ataccama</a>, <a href="https://www.alation.com/" target="_blank">Alation</a>,<span style="mso-spacerun: yes;"> </span><a href="https://alexsolutions.com/" target="_blank">Alex Solutions</a> and <a href="https://www.collibra.com/us/en" target="_blank">Collibra</a>, Microsoft is
ramping the functionality of its data catalogue product, <a href="https://azure.microsoft.com/en-gb/products/purview" target="_blank">Purview</a>.</i></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size: medium;"><o:p> </o:p></span></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2OCzQz0j5pSkqXZlocRppZuyLeh3uQXVqSLOUQY3Ag9WyrRpwal3GDcyidcWE3yfgQeifRnM9-soP6dHqaHPZSTYCaHjLRHmiDsezoR53klYlnIE1VxIuzTYLt4wPUDc9Va4UgjqI_J2cKRP4XZzVf4yyO4uUIaUi5ZMvybm4cdSmxP2yz2rw9xtN1sY/s1412/Data%20Governance.JPG" style="margin-left: auto; margin-right: auto;"><img alt="Google Trend Index "Data Governance"" border="0" data-original-height="400" data-original-width="1412" height="147" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2OCzQz0j5pSkqXZlocRppZuyLeh3uQXVqSLOUQY3Ag9WyrRpwal3GDcyidcWE3yfgQeifRnM9-soP6dHqaHPZSTYCaHjLRHmiDsezoR53klYlnIE1VxIuzTYLt4wPUDc9Va4UgjqI_J2cKRP4XZzVf4yyO4uUIaUi5ZMvybm4cdSmxP2yz2rw9xtN1sY/w513-h147/Data%20Governance.JPG" title="A fairly stable trend on Google: Data Governance is here to stay..." width="513" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Google Trend Index on "Data Governance"</td></tr></tbody></table><br /><p></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size: medium;">The reason for this is twofold: the
emerging multicloud architectures as well as the advent of the data mesh architecture
driving new data ecosystems for complex data landscapes.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size: medium;">Without firm data governance processes and
software supporting these processes, the return on information would produce
negative figures.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: medium;"><span lang="EN-GB">In the next blog <i><a href="https://ba4bi.blogspot.com/2023/10/a-data-mesh-z-hamak-dehgani-cornered.html" target="_blank">Defining a Data Mesh</a></i> I will define what a data mesh is about and in the following blog articles </span>I will suggest a few
measures needed to avoid data swamps. Stay tuned!</span></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-22870043240863080182023-02-21T09:30:00.003+01:002023-11-15T14:46:41.878+01:00How will ChatGPT affect the practice of business analysis and enterprise architecture?<p> </p><p class="MsoNormal"><span lang="EN-GB"><i>ChatGPT (Chat Generative Pre-trained
Transformer), is a language model-based chat bot developed by OpenAI enabling
users to refine and steer a conversation towards a desired length, format,
style, level of detail, and language. <span style="mso-spacerun: yes;"> </span>Many of my colleagues are assessing the impact
of Artificial Intelligence products on their practice and the jury is still out
there: some of them consider it a threat that will wipe out their business
model and others see it as an opportunity to improve productivity and
effectiveness of their practice</i>. <o:p></o:p></span></p><p class="MsoNormal"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifR5109atbFb-E9WjdeuRyAJOLR0xRVqnzND4hp-ebuL-dMedJiM9glyngNs9u2t3lUhw5Xn86XRfmBmAKFmGKD_0kheZeYDs9MM2EcPcbTGM8WkwCziY0TgapZsVxUrUtM1sz3K8-8lnOK_YlGFEUKLJf5BvTW-fQYfitXYGHrad490LvMxO-eCk93lY/s800/ChatGPT_logo.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="800" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifR5109atbFb-E9WjdeuRyAJOLR0xRVqnzND4hp-ebuL-dMedJiM9glyngNs9u2t3lUhw5Xn86XRfmBmAKFmGKD_0kheZeYDs9MM2EcPcbTGM8WkwCziY0TgapZsVxUrUtM1sz3K8-8lnOK_YlGFEUKLJf5BvTW-fQYfitXYGHrad490LvMxO-eCk93lY/s320/ChatGPT_logo.jpg" width="320" /></a></div><br /><span lang="EN-GB"><br /></span><p></p><p class="MsoNormal"></p><p class="MsoNormal"><span lang="EN-GB">I have a somewhat different opinion. Language
training models use gigantic amounts of data to train the models but I am
afraid if you want to use the Internet data you certainly have a massive amount
of data but of dubious and not always verifiable quality.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">General Internet data is polluted with
commercial content, hoaxes and ambiguous statements that need strong cultural
background analysis to make sense of it.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">The data that has better quality than
general Internet data is almost always protected by a copyright; Therefore use
without permission is not always gentlemanlike to say the least.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Another source of training data are the
whitepapers and other information packages you get in exchange for your data:
e-mail, function, company,… These documents often start with stating a problem
in a correct and useful way but then direct you to the solution delivered by
their product.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">The best practices in business analysis and
enterprise architecture are -I am afraid- not on the Internet. They’re like
news articles behind a paywall. So if you ask CHAT GPT a question like “Where
can I find information to do business analysis for analytics and business
intelligence?” You get superficial answers that –at best- provide a starting
point to study the topic.<o:p></o:p></span></p><p class="MsoNormal"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhURb51aig_evBK9SUEHz3cWR64uBAKaVfp-hJ2YjUFwOeeNunojuzgAgL9jiUVj2SkvfmNz6HEJ3TQW_jJMhZWzcHdfMgj8gE-sQJbZba4Z1HkKtA8NQQSf0UGXmhnDu0g-e96CklVWa6pkiRQz9rQ3I46USJuzTbxX4Ps1D_gw6d7zCHudvAKM1H2Ejc/s783/CHATGPT%20gedoe.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="658" data-original-width="783" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhURb51aig_evBK9SUEHz3cWR64uBAKaVfp-hJ2YjUFwOeeNunojuzgAgL9jiUVj2SkvfmNz6HEJ3TQW_jJMhZWzcHdfMgj8gE-sQJbZba4Z1HkKtA8NQQSf0UGXmhnDu0g-e96CklVWa6pkiRQz9rQ3I46USJuzTbxX4Ps1D_gw6d7zCHudvAKM1H2Ejc/s16000/CHATGPT%20gedoe.JPG" /></a></div><span lang="EN-GB"><br /></span><p></p>
<p class="MsoNormal"><span lang="EN-GB">A screenshot of the shallow and casual
reply. It goes on with riveting advice like “Stay Informed”, “Training and Certification”,
“Networking”, “Documentation”,…<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">And the question “What are Best Practices
in business intelligence” leads to the same level of platitudes and triteness: <o:p></o:p></span></p>
<p class="MsoNormal"></p><ul style="text-align: left;"><li><span lang="EN-GB">“Align with business goals” who would have
thought that? </span></li><li><span lang="EN-GB">“User involvement and collaboration”
Really?</span></li><li><span lang="EN-GB">“Data Quality and Governance” Sure, but
how? And when and where?</span></li></ul><p></p>
<p class="MsoNormal"><span lang="EN-GB">In conclusion: a professional analyst or
enterprise architect has nothing to fear from ChatGPT. </span></p><p class="MsoNormal"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEic1cyqBZFBGIHyvdoaafLxIiP7ptkna5pwWWGuEBZcXdrbxZ53gxZ9Z8g9UFF-OokTt6ZZLgcKPLzb8tUeHNPRjW-cdqZcoZEVnh6BEvoY_pR_3dSor0tdToxWVG-IrkdTn-57pDC1g3xkUqZIvwm8BLGO-WnMsFYGQSxGvCy9pEtSsK_D2WRI0Cm6ZAg/s950/GoogleChat.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="466" data-original-width="950" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEic1cyqBZFBGIHyvdoaafLxIiP7ptkna5pwWWGuEBZcXdrbxZ53gxZ9Z8g9UFF-OokTt6ZZLgcKPLzb8tUeHNPRjW-cdqZcoZEVnh6BEvoY_pR_3dSor0tdToxWVG-IrkdTn-57pDC1g3xkUqZIvwm8BLGO-WnMsFYGQSxGvCy9pEtSsK_D2WRI0Cm6ZAg/s16000/GoogleChat.JPG" /></a></div><span lang="EN-GB"><br /></span><p></p><p class="MsoNormal"><span lang="EN-GB">At best, it provides a
somewhat more verbose and redacted answer to a question saving you the time to
plough through over a billion answers from Google.<o:p></o:p></span></p><br /><p></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0Terneuzen, Nederland51.3322854 3.832426523.022051563821158 -31.3238235 79.642519236178856 38.9886765tag:blogger.com,1999:blog-4520438534640699001.post-11502000159730349642022-12-05T12:09:00.065+01:002023-11-09T11:53:02.810+01:00Data Architecture as a Consequence of Organisation Design<p> </p><p style="text-align: left;"><i>Lingua Franca was involved in the data
architecture of an organisation which name and type is of no interest for the
case I am making. Namely, <span style="mso-spacerun: yes;"> </span>the way an
organisation functions and is structured determines the data architecture. It
is a text book example of many organisations today.</i></p><div><span lang="EN-GB"><p class="MsoNormal"><span lang="EN-GB">The organisation was a merger of various business units which all used their own proprietary business processes, data standards and data definitions.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">The CIO had a vision of well governed, standardised processes that would create a unified organisation that operated in a predictable and transparent manner.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB"></span></p><h4 style="text-align: left;"><span lang="EN-GB">Harmonised End to End Processes Are the Basis of Transparent Decision Making</span></h4><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjMejzQZLQCUwyPvSXNCr7ZtJujtTk7aqaNNJnELhZzwHV1HgSUDRsj1yIsJrWvYtwN_gP9sTVNaV5x67aiaUGO6W2KXW5tPOByRFxJLhz24lo58f67eUGXZb3CWiyCNOCYNE3TNFTLoMZgLNkLonR8-d9TLEY2cP0ZYUnEm2HT2wy8FYXsXI_2y8nD" style="margin-left: 1em; margin-right: 1em;"><img alt="Common dimensions and common facts" data-original-height="557" data-original-width="945" height="378" src="https://blogger.googleusercontent.com/img/a/AVvXsEjMejzQZLQCUwyPvSXNCr7ZtJujtTk7aqaNNJnELhZzwHV1HgSUDRsj1yIsJrWvYtwN_gP9sTVNaV5x67aiaUGO6W2KXW5tPOByRFxJLhz24lo58f67eUGXZb3CWiyCNOCYNE3TNFTLoMZgLNkLonR8-d9TLEY2cP0ZYUnEm2HT2wy8FYXsXI_2y8nD=w640-h378" title="High level dimensional model based on Kimball's data mart architecture" width="640" /></a></div>
<p class="MsoNormal" style="text-align: center;"><span lang="EN-GB">Shared facts and dimensions assure a scalable and manageable analytics architecture</span></p><p class="MsoNormal"><span lang="EN-GB">The case for a Kimball approach in data
warehousing was clear: if every department, every knowledge unit would use the
same processes, the shared facts and common dimension architecture was a no
brainer.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">As the diagram suggests: it takes effort to
make sure everybody is on the same page about the metrics and the dimensions
but once this is established, new iterations will go smoothly and build trust
in the data.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">
</span></p><p class="MsoNormal"><span lang="EN-GB">For more than 4 years, the resistance to
change wore the CIO, data warehouse team and finally the data architect out
when the CIO left the organisation. The new CIO decided to not continue the
fight for harmonised processes and saw this as a reduced need for a data
warehouse. If every business unit would use its own operational reporting, it
would produce rapid results at a far lower cost than a data warehouse
foundation delivering the reports. A new crew was on boarded: two ETL
developers, two front end developers and a data architect.<o:p></o:p></span></p><h4><span lang="EN-GB">Satisfying Clients in Their Operational Silos Creates Technical Debt</span></h4><div><span lang="EN-GB"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg7R1skSmh3SGNHbYl8BpQwDgzWJU_VaXtpyEjecNKrp5XV7epYsUVIxSWFxDXXBsGisb7qSj5tPHFZ81S4B6ymMdyEaJBG2gkPKFh4cx04zTWe_kFFE1ZkP8CPD99Kt0IYMrFn2dvIf-79LYymiYEupbzrOsuJFluaCrRqT4EyfuIX9ZC0Lvu_cdQW" style="margin-left: 1em; margin-right: 1em;"><img alt="A third normal form data model for operational reporting" data-original-height="365" data-original-width="945" height="248" src="https://blogger.googleusercontent.com/img/a/AVvXsEg7R1skSmh3SGNHbYl8BpQwDgzWJU_VaXtpyEjecNKrp5XV7epYsUVIxSWFxDXXBsGisb7qSj5tPHFZ81S4B6ymMdyEaJBG2gkPKFh4cx04zTWe_kFFE1ZkP8CPD99Kt0IYMrFn2dvIf-79LYymiYEupbzrOsuJFluaCrRqT4EyfuIX9ZC0Lvu_cdQW=w640-h248" title="Quick and dirty data model for operational reporting" width="640" /></a></div><br /><div style="text-align: center;">Cutting corners for fast delivery creates technical debt, that needs to be repaid</div></span></div><div><span lang="EN-GB"><br /></span></div><div><span lang="EN-GB"><p class="MsoNormal"><span lang="EN-GB">As this diagram suggests, the client
defines his particular needs, asking for a report not on SKU level because he’s
only interested in product sets. The sets require special handling so they are
linked to specific shippers who have their delivery areas.<span style="mso-spacerun: yes;"> </span>Although this schema may cause no problems
for the frontend developer to produce a nice looking report, consolidating the
information on corporate level will take time and effort. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">The reality will prove differently, of
course. If every business unit uses its own definitions, metrics and dimensions
there is no chance of having correct, aggregated information for strategic decision
making. To remedy this shortcoming, the new data architect will have to go back
to 2008, publish date of Bill Inmon’s<span style="mso-spacerun: yes;"> </span>DW
2.0. The idea is to create the operational report as fast as possible and after
delivering the product refactor the underlying data to make them compatible
with other data used in previous reports.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">The result is a serious governance effort,
lots of rework and an ever growing DW 2.0 in the third normal form that one day
may contain sufficient enterprise wide data to produce meaningful aggregates
for strategic direction. The Corporate Information factory (CIF) revisited so
to speak.<o:p></o:p></span></p><h4 style="text-align: left;"><span lang="EN-GB">Why the CIF Never Realised Any Value<o:p></o:p></span></h4><div><span lang="EN-GB"><p class="MsoNormal"><span lang="EN-GB">In Inmon’s world, it was recommended to
build the entire data warehouse before extracting any data marts. These data
marts are aggregates, based on user profiles or functions in the organisation
and are groupings of detailed data that may change over time.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">This led to many problems on the sites I
have visited during my career as a business analyst and data architect. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">First and foremost: by the time you have
covered the entire scope of the CIF, the world has changed and you can refactor
entire parts of the data model and
reload quite a few data to be in synch with new realities. Doing this on a 3NF
data schema can be quite complex and time and resource consuming. And then
there is the data mart management problem: if requirements for aggregations
change over time, keeping track of historical changes in aggregations and
trends is a real pain.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB"><br /></span></p><h4 style="text-align: left;"><span lang="EN-GB">About DW 2.0: the Data Quagmire<o:p></o:p></span></h4><div><span lang="EN-GB"><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh4QQD8p0DP9p5usnRc_yeVv4AhJBSUN6hbu63EJke3rAFScAidColsPraYKXLk9qDej5_RlzUHtWYnq95aLHjQeMV-o8IEw7sOvJFmVLJ2c5eeR2BJsDPRSjAbPXcGsSLyLXyZFfqkmXYXr8t93CjTRp1m2uipSX7e3Hg93a486RIQmnrnDfYgOD_k" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="503" data-original-width="408" height="400" src="https://blogger.googleusercontent.com/img/a/AVvXsEh4QQD8p0DP9p5usnRc_yeVv4AhJBSUN6hbu63EJke3rAFScAidColsPraYKXLk9qDej5_RlzUHtWYnq95aLHjQeMV-o8IEw7sOvJFmVLJ2c5eeR2BJsDPRSjAbPXcGsSLyLXyZFfqkmXYXr8t93CjTRp1m2uipSX7e3Hg93a486RIQmnrnDfYgOD_k=w325-h400" width="325" /></a></div><br /><br /></div><p class="MsoNormal"><span lang="EN-GB">To anyone who hasn’t read this book: it’s
the last attempt of the “father of data warehousing” to defend his erroneous
Corporate Information Factory (CIF), adding some text data to a structured data
warehouse in the third normal form. The book is full of conceptual drawings but
that is all they are; not one implementation direction follows up on the
drawings. Compare this to the Kimball books where every architectural concept
is translated into SQL scripts and clear instructions and you know where the real value
is.</span></p>
<p class="MsoNormal"><span lang="EN-GB">With DW 2.0 the organisation is trying to
salvage some of the operational reports’ value but at a cost, significantly
higher than respecting the principle “Do IT right the first time”. The only
good thing about this new approach is that nobody will notice the cost overrun
because it is spread over numerous operational reports over time. Only when the
functional data marts need rebuilding, may some people notice the data quagmire
the organisation has stepped into.<o:p></o:p></span></p><span style="font-family: verdana; font-style: italic;">Conclusion, to paraphrase A.D. Chandler: Data structure follows strategy</span></span></div><div><br /></div></span></div><p class="MsoNormal"><span lang="EN-GB"><br /></span></p><br /></span></div></span></div>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-12095686646856421612021-06-28T17:27:00.007+02:002021-07-12T07:55:25.382+02:00Managing a Data Lake Project Part III: Architecture Drives the Project Method<p><i>Remember
the old days when the data warehouse was the only source of the facts and
answered almost any business question, provided the data were available in the
source systems? Today, more and more data is beyond our control. “Control” in
the sense of precooked structures, well documented and well governed data
objects. More and more data is generated from sources beyond our control. And
only the data lake can facilitate comprehensive analytics.</i></p>
<p class="MsoNormal"><span lang="EN-GB"><i>To make
clear how the architecture of a data lake drives the project approach, it is
necessary we review the three major data warehouse architectures and their
project approach before we present the new methods needed in a data lake
environment.</i><o:p></o:p></span></p>
<h4 style="text-align: left;"><span lang="EN-GB"> The Kimball architecture and its project approach</span></h4><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-F26DyAw3uOM/YMxWdikqssI/AAAAAAAAd5s/TYIUHLwFNCIn34uRBbPZtd9I5iOeCr4bQCLcBGAsYHQ/s696/Kimball.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="561" data-original-width="696" src="https://1.bp.blogspot.com/-F26DyAw3uOM/YMxWdikqssI/AAAAAAAAd5s/TYIUHLwFNCIn34uRBbPZtd9I5iOeCr4bQCLcBGAsYHQ/s320/Kimball.jpg" width="320" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;"><p class="MsoNormal"><span lang="EN-GB">Ralph
Kimball’s star schema approach is the most used -and as far as I am concerned-
the most pragmatic low-threshold approach to data warehousing. Each dimension is
constructed with an enterprise view and shared in the appropriate data marts.
And each data mart represents a business process. For project managers, this
means that an enterprise scan is needed to define the dimensions, followed by a
study on the combination “information value times feasibility” to pick the
order of execution. </span></p></div><h4 style="text-align: left;">The Lindstedt architecture and its project approach</h4><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-mOHTLu0kK8c/YMxWeSVDLVI/AAAAAAAAd54/rvi1RSPv-yoH7BFIFdKGuioglcWdLU9pQCLcBGAsYHQ/s630/Linstedt.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="578" data-original-width="630" src="https://1.bp.blogspot.com/-mOHTLu0kK8c/YMxWeSVDLVI/AAAAAAAAd54/rvi1RSPv-yoH7BFIFdKGuioglcWdLU9pQCLcBGAsYHQ/s320/Linstedt.jpg" width="320" /></a></div><div class="separator" style="clear: both; text-align: left;"><p class="MsoNormal"><span lang="EN-GB">The great
advantage of a data vault is its flexibility to adapt to new situations, new
data sources and other changes in the data landscape. Like the Kimball method,
it focuses on business processes and models these in a highly normalised way
using hashes to “freeze” temporal links between objects and their
attributes. What this means to the project
approach is obvious: we postpone the materialisation of a queryable schema
until we are sure about the data persistence. In many of the projects we
managed, a seamless transition from a data vault to star schema was made. For
project managers, this means a heavy focus on the business process and a
flexible way of representing all the processes and delivering queryable data
whenever the need for it was expressed by the business.<o:p></o:p></span></p></div><h4 style="clear: both;"><br /></h4><h4 style="clear: both;">The Corporate Information Factory architecture from Inmon and its project approach</h4><br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-FTnjkF8NoZE/YMxWdulO2jI/AAAAAAAAd50/JsbfKWyuNzkJfBp2IWTrPLEyqSKPYlEqgCLcBGAsYHQ/s711/Inmon.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="535" data-original-width="711" src="https://1.bp.blogspot.com/-FTnjkF8NoZE/YMxWdulO2jI/AAAAAAAAd50/JsbfKWyuNzkJfBp2IWTrPLEyqSKPYlEqgCLcBGAsYHQ/s320/Inmon.jpg" width="320" /></a></div><div class="separator" style="clear: both; text-align: left;"><p class="MsoNormal"><span lang="EN-GB">The Inmon
approach is something completely different from the previous methods. As of the
early 1990s Inmon has made his case for a corporate information factory (CIF)
that would take every data source in scope, build a target model in the third Normal
Form (3NF) and once this Herculean task was competed it was finally time to
deliver. In his method functional data marts would provide extracts from the
CIF. Think of an HR data mart, a marketing data mart, a finance data mart, etc…
No need to say this can only work in very stable environments where the
external factors don’t influence too much the approach to analytics. In all the
projects me and my colleagues have been involved this was the Never-ending
Project. Please don’t go there. And if, by any chance there is a business case
for this approach, allow for sufficient time and resources. You will need it. <o:p></o:p></span></p></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><h4 style="text-align: left;">The data lake architecture and its project approach</h4><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-MtzGCghg8ZI/YMxWdl6-DNI/AAAAAAAAd5w/EHTw2CpqCeAGai6sqtWtcXeboGcoLE-vACLcBGAsYHQ/s1280/DataLakeArch.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="598" data-original-width="1280" height="299" src="https://1.bp.blogspot.com/-MtzGCghg8ZI/YMxWdl6-DNI/AAAAAAAAd5w/EHTw2CpqCeAGai6sqtWtcXeboGcoLE-vACLcBGAsYHQ/w640-h299/DataLakeArch.jpg" width="640" /></a></div></div><p class="MsoNormal"><span lang="EN-GB">A data lake
project is a completely different story form the previous three: no more up
front analysis of concepts, objects, entities and attributes that contribute to
these concepts before building the data stores. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">In a
nutshell, a data lake project is about looking for cheap and simple storage
like S3 on Amazon Web Services or ALS on Azure, making sure the ingestion data
pipelines are in place to receive all sorts of data and once these data are in
place, making sure they are ready for exploitation. For project managers, this means a totally different
project management flow. Contrary to the three previous architectures, there is
no synching between business and tech: after a high level business analysis,
the technical track will provide for data storage, data access and data
cataloguing to make it exploitable for the business. <o:p></o:p></span></p><p><br /></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com1tag:blogger.com,1999:blog-4520438534640699001.post-84611442464263719062021-06-16T20:00:00.004+02:002021-07-03T10:31:21.143+02:00Managing a Data Lake Project Part II: A Compelling Business Case for a Governed Lake<p> </p><p class="MsoNormal"><span lang="EN-GB"><i>In Part I A Data Lake and its Capabilities
we already hinted towards a business case but in this blog we make it a little
bit more explicit.</i></span></p><p class="MsoNormal"><span lang="EN-GB"></span></p><div class="separator" style="clear: both; text-align: center;"><span lang="EN-GB"><a href="https://1.bp.blogspot.com/-bC_ZjRqd_WY/YMsC6Dnjy8I/AAAAAAAAd4g/shJJMTRg7z01TWfl6Cu3lScXKGG2BU2swCLcBGAsYHQ/s684/recap%2Bcapabilities.JPG" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="510" data-original-width="684" height="478" src="https://1.bp.blogspot.com/-bC_ZjRqd_WY/YMsC6Dnjy8I/AAAAAAAAd4g/shJJMTRg7z01TWfl6Cu3lScXKGG2BU2swCLcBGAsYHQ/w640-h478/recap%2Bcapabilities.JPG" width="640" /></a></span></div><span lang="EN-GB"><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><div style="text-align: center;"><span style="font-style: italic;">A recap from Part I: the data lake capabilities</span></div></span><p></p>
<p class="MsoNormal"><span lang="EN-GB"><i>The business case for a data lake has many
aspects, some of them present sufficient rationale on their own but that
depends of course on the actual situation and context of your organisation.
Therefore, I mention about eleven rationales, but feel free to add yours in the
comments.</i></span></p><p class="MsoNormal"><b> We are mixing on-premise data with Cloud
based systems which causes new silos</b></p><p class="MsoNormal"><span lang="EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">The Cloud providers deliver software for
easy switching your on-premise applications and databases to Cloud versions. But
there are cases where this isn’t possible to do this in one fell swoop:<o:p></o:p></span></p>
<p class="MsoNormal"></p><ul style="text-align: left;"><li><span lang="EN-GB">Some applications require refactoring
before moving them to the Cloud;</span></li><li><span lang="EN-GB">Some are under such strict info security
constraints that even the best Cloud security they can’t be relied on. I know
of<span style="mso-spacerun: yes;"> </span>retailer who keeps his excellent
logistic system in something close to a bunker!</span></li><li><span lang="EN-GB">Sometimes the budget or the available
skills are insufficient to support a 100 % Cloud environment, etc…</span></li></ul><p></p>
<p class="MsoNormal"><span lang="EN-GB">This provides already a very compelling business case for a governed data lake: a catalog that manages lineage and meaning will make the transition smoother and safer.</span></p><p class="MsoNormal"><span lang="EN-GB"><b>Master data is a real pain in siloed data storage, as is governance...</b></span></p><p class="MsoNormal"><span lang="EN-GB"></span></p><p class="MsoNormal"><span lang="EN-GB">A governed data lake can improve master
data processes by involving the end users in evaluating intuitively what’s in the
data store. Using both predefined data quality rules and machine learning to
detect anomalies and implicit relationships in the data as well as defining the
golden record for objects like CUSTOMER, PRODUCT, REGION,… the data lake can
unlock data even in technical and physical silos. <o:p></o:p></span></p><p class="MsoNormal"><b>We now deal with new data processing and storage technologies other than ETL and relational databases: NoSQL, Hadoop, Spark or Kafka to name a few</b></p><p class="MsoNormal"></p><p class="MsoNormal"><span lang="EN-GB">NoSQL has many advantages for certain
purposes but from a governance point of view it is a nightmare: any data
format, any level of nesting and any undocumented business process can be
captured by a NoSQL database.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Streaming (unstructured) data is unfit for
a classical ETL process which supports structured data analysis so we need to
combine the flexibility of a data lake ingestion process with the governance
capabilities of a data catalogue or else we will end up with a data swamp.<o:p></o:p></span></p><b>We don't have the time, nor the resources to analyse up front what data are useful for analysis and what data are not</b><p></p><p class="MsoNormal"><span lang="EN-GB">There is a shortage of experienced data
scientists. Initiatives like applications to support data citizens may soften
the pain here and there but let’s face it, most organisations lack the
capabilities for continuous sandboxing to discover what data in what form can
be made meaningful. It’s easier to accept indiscriminately all data to move
into the data lake and let the catalogue do some of the heavy lifting. <o:p></o:p></span></p><p class="MsoNormal">
</p><p class="MsoNormal"><b>We need to scale horizontally to cope with massive unpredictable bursts of of data</b></p><p class="MsoNormal"><span lang="EN-GB">Larger online retailers, event
organisations, government e-services and other public facing organisations can
use the data lake as a buffer for ingesting massive amounts of data and sort
out its value in a later stage. <o:p></o:p></span></p><p class="MsoNormal"><b>We need to make a rapid and intuitive connection between business concepts and data that contribute, alter, define or challenge these concepts</b></p><p class="MsoNormal"><span lang="EN-GB">This has been my mission for about three
decades: to bridge the gap between business and IT and as far as “classical”
architectures go, this craft was humanly possible. But in the world of NoSQL,
Hadoop and Graph databases this would be an immense task if not supported by a
data catalogue. <o:p></o:p></span></p><p class="MsoNormal"><b>Consequently, we need to facilitate self-service data wrangling, data integration and data analysis for the business users</b></p><p class="MsoNormal">A governed data lake ensures trust in the data, trust in what business can and can't do. This can speed up data literacy in the organisation by an order of magnitude.</p><p class="MsoNormal"><b>We need to get better insight in the value and the impact of data we create, collect and store.</b></p><p class="MsoNormal"><span lang="EN-GB">Reuse of well-catalogued data will enable
this: end users will contribute to the evaluation of data and automated meta-analysis
of data in analytics will reinforce the use of the best data available in the
lake. Data lifecycle management becomes possible in a diverse data environment.</span></p><p class="MsoNormal"><span lang="EN-GB"><b>We need to avoid fines like those stipulated in the GDPR from the EU which can amount up to 4% of annual turnover!</b></span></p><p class="MsoNormal"><span lang="EN-GB">Data privacy regulations need functionality
to support “security by design” which is delivered in a governed data lake.
Data pseudonimisation, data obfuscation or anonimisation come in handy when these
functions are linked to security roles and user rights. <o:p></o:p></span></p><p class="MsoNormal"><b>We need a clear lineage of the crucial data to comply with stringent laws for publicly listed companies</b></p><p class="MsoNormal"><span lang="EN-GB">Sarbanes Oxley and Basel III are examples
of legislation that require accountability at all levels and in all business
processes. Data lineage is compulsory in these legal contexts. <o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">But more
than all of the above IT based arguments, there is one compelling business case
for C-level management: speeding up the decision cycle time and the
organisation’s agility in the market. <o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">Whether
this market is a profit generating market or a non-profit market where the
outcomes are beneficial to society, speeding up decisions by tightening the
integration between concepts and data is the main benefit of a governed data
lake. <o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">Anyone who
has followed the many COVID-19 kerfuffles, the poor response times and the
quality of the responses to the pandemic sees the compelling business case:<o:p></o:p></span></p><p class="MsoNormal"></p><ul style="text-align: left;"><li><span lang="EN-GB">Rapid meta-analysis
of peer reviewed research papers;</span></li><li><span lang="EN-GB">Social
media reporting on local outbreaks and incidents;</span></li><li><span lang="EN-GB">Second use
opportunities from drug repurposing studies;</span></li><li><span lang="EN-GB">Screening
and analysing data from testing, vaccinations, diagnoses, death reports,…</span></li></ul><p></p><p class="MsoNormal"><span lang="EN-GB">I am sure
medical professionals can come up with more rationales for a data lake, but you
get the gist of it.</span></p><p class="MsoNormal">So, why is
there a need for a special project management approach to a data lake introduction?
That is the theme of Part III. But first,
let me have your comments on this blogpost.</p><p class="MsoNormal"><span lang="EN-GB"><br /></span></p><p class="MsoNormal"><span lang="EN-GB"><br /></span></p><p class="MsoNormal"><br /></p><p class="MsoNormal"><br /></p><p class="MsoNormal"><br /></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-66528328392879278282021-06-09T19:19:00.000+02:002021-06-09T19:19:17.502+02:00Managing a Data Lake Project Part I: A Data Lake and its Capabilities<p><i> A data lake can provide us with the
technology to cope with the challenges of various data formats arriving in
massive amounts, too fast and diverse for a classic data pipeline resulting in
a data warehouse. As a the data warehouse is optimised for analysis of structured
data, the inflow of unstructured data strings, entire documents, JSONs with n
levels of nesting, binaries, etc… is simply too much for a data warehouse.</i></p>
<p class="MsoNormal"><span lang="EN-GB">A data lake is an environment that manages
any type of data from any type of source or process in a transparent way for
the business. In tandem with a data catalogue, a lake provides data governance
and facilitates data wrangling, <span style="mso-spacerun: yes;"> </span>trusted
analytical capabilities as well as self-service analytics to name a few.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">If we zoom in on these capabilities, we can
list these as the basic requirements for a minimum viable product:<o:p></o:p></span></p>
<p class="MsoNormal"></p><ul style="text-align: left;"><li><span lang="EN-GB">Automated discovery, cataloguing and
classification of ingested data;</span></li><li><span lang="EN-GB">Collaborative options for evaluating the
ingested data;</span></li><li><span lang="EN-GB">Governance of quality, reliability,
security and privacy aspects as well as lifecycle management; </span></li><li><span lang="EN-GB">Facilitates data preparation for analytical
purposes in projects as well as for unsupervised and spontaneous self-service
analytics;</span></li><li><span lang="EN-GB">Provides the business end users with an
intuitive search and discovery platform;</span></li><li><span lang="EN-GB">Archives data where and when necessary. </span></li></ul><p></p>
<p class="MsoNormal"><span lang="EN-GB"><span style="mso-spacerun: yes;"> </span></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-H9CSOgYEcmE/YLur5kM7-bI/AAAAAAAAdxA/Ja27XvkgE1AtFoXw4rQ-KtTlcTfPqlArQCLcBGAsYHQ/s1160/Data%2Bgalore.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Generic data processing map" border="0" data-original-height="584" data-original-width="1160" height="322" src="https://1.bp.blogspot.com/-H9CSOgYEcmE/YLur5kM7-bI/AAAAAAAAdxA/Ja27XvkgE1AtFoXw4rQ-KtTlcTfPqlArQCLcBGAsYHQ/w640-h322/Data%2Bgalore.png" title="A schematic representation of data variety from business processes and outside events" width="640" /></a></div><div style="text-align: center;"><i>Data comes from events that lead to business processes as well as from outside events that may become part of the business processes</i></div><o:p></o:p><p></p><p class="MsoNormal"><span lang="EN-GB">Some vendors launch the term “data
marketplace” to stress the self-service aspects of a data lake. But this
position depends on the analytical maturity of the organisation. If introduced
too early it may provide further substantiation for the claim that:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">“Analytics is a process of ingesting,
transforming and preparing data for publication and analysis to end up in Excel
sheets, used a “proof” for a management hypothesis”.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB"></span></p><h4 style="text-align: left;"><span lang="EN-GB">What makes a data lake ready for use?</span></h4>
<p class="MsoNormal"><span lang="EN-GB">Meta data: data describing the data in the
lake: its providence, the data format(s), the business and technical
definitions,…;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Governance: business and IT control over
meaning, application and quality of data as well as information security and
data privacy regulation;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Cataloguing: either by machine learning or
precooked categories and rule engines, data is sorted and ordered according to
meaningful categories for the business. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Structuring: data increases in meaning if
relationships with other concepts are modelled in hierarchies, taxonomies and
ontologies;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Tagging: both governed and ungoverned tags
(i.e. user tags) dramatically improve the usability of the ingested data. If
these tags are evaluated on practical use by the user community they become
part of a continuous quality improvement process; <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Hierarchies: identical to tagging, there
may be governed and personal hierarchies in use; <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Taxonomies: systematic hierarchies, based
on scientific methods; <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Ontologies: a set of concepts and
categories in a subject area or data domain that shows their properties and the
relations between them to model the way the organisation sees the world.<o:p></o:p></span></p><br /><p></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-2787262779752317302021-05-29T19:20:00.001+02:002021-06-02T15:12:13.415+02:00Managing a Data Lake Project<i>With the massive growth of online generated data and IoT data, the proportion of unstructured and semi-structured data constitutes the bulk of the data that needs to be analysed. Whereas a 50 Gigabyte data warehouse to facilitate analysis of structured data was quite an achievement up to now, this number dwindles compared to the unstructured and semi-structured data avalanche.
<i></i></i>
<div><i><br /></i></div><h4 style="text-align: left;"><span style="font-size: medium;">Data Avalanche?</span></h4><div><span style="font-size: medium;"><br /></span></div><div><span style="font-size: medium;">Yes, because compared to the steady stream of data from transaction processing systems, we now have to deal with irregular flows and massive bursts of incoming data that needs to be adequately processed to provide meaning to the data.</span></div><div><span style="font-size: medium;">New data sources emerge, other than social media and IoT data, like smart machines and machine learning systems generating new data, based on existing sources. Managing various data types and metadata in impressive volumes are just a few technical aspects which can be solved by technology. The HR- , legal- and organisational aspects are level more complex, but aspects these are not in scope of this series of blog posts. </span></div><div><span style="font-size: medium;">We are adding extra process and event based decision support to our management capabilities and that alone is worth the cost, the trouble and the change management efforts to introduce a data lake.<br /></span><h4 style="text-align: left;"><span style="font-size: medium;">See you at the Webinar!</span></h4></div><div><span style="font-size: medium;">Wednesday 9th June you can tune in on a short webinar hosted by the Great IT Professional. <a href="https://www.greatpro.org/webinar-live-register/?id=1852" target="_blank">You can still register via this link.</a> The webinar will be followed by a series of articles on how to manage the Data Lake project. Stay tuned!</span></div><div><span style="font-size: medium;"><br /></span></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://www.greatpro.org/webinar-live-register/?id=1852" imageanchor="1" style="margin-left: 1em; margin-right: 1em;" target="_blank"><img alt="Bert Brijs Webinar on Managing a Data Lake Project" border="0" data-original-height="651" data-original-width="860" height="303" src="https://1.bp.blogspot.com/-C174xZNII9Q/YLeCWym9r-I/AAAAAAAAdw0/n288p8IvPkIt_-DX_dusfew5g2VJSJn9QCLcBGAsYHQ/w400-h303/Webinar.JPG" title="Register for the Webinar "Managing a Data Lake Project"" width="400" /></a></div><br /><span style="font-size: medium;"><br /></span></div>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-18054669582879673722020-12-30T17:59:00.007+01:002021-01-04T18:23:04.839+01:00New Inroads for Analytics in the Post Corona Era<p> </p><p class="MsoNormal"><span lang="EN-GB" style="mso-ansi-language: EN-GB;"><i>OK, 2021
will not get rid of the virus immediately but the new consumer behaviour,
induced by the pandemic will have lasting effects that need to be taken care of
by brand owners, distribution channels and -consequently- by the analytics approach
and infrastructure.</i><o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB" style="mso-ansi-language: EN-GB;"></span></p><h3 style="text-align: left;"><span lang="EN-GB">So, what is
exactly this new consumer behaviour?</span></h3><p class="MsoNormal"><span lang="EN-GB">You already
guessed: more online shopping and more pervasive switching to web shops from
the local shops to compete with the incumbents. The local shop owners finally
have understood the value of proximity combined with the convenience of online
browsing and online ordering or preordering and collecting the order at the
local shop.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">But there’s
more. Not only have the predominant shopping logistics changed; the product
range has also undergone the influence of the various lockdown periods.
Consumers have a tendency to shop for more luxury products in the food section
as a means of self-indulgence and the dichotomy between convenience and fun
shopping is getting clearer and larger. Some retail chains are already
experimenting with automatic replenishment of convenience products using
automated algorithms. But some supermarkets in the Benelux are combining
convenience, fun and self-indulgence offering prepared meals that can be
consumed in the shop. Plus, Albert Heyn and Jumbo are experimenting with the
concept. This can have an impact on local restaurants who have survived on
their take away service during the pandemic.<o:p></o:p></span></p><p class="MsoNormal"><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-31ucptiZ5ys/X_NLXdT_MiI/AAAAAAAAdU8/6AaNmniUdSMrv2DXp0G8EQdGfmwNYOorgCPcBGAsYHg/s4624/20210104_134819.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="4624" data-original-width="3468" height="400" src="https://1.bp.blogspot.com/-31ucptiZ5ys/X_NLXdT_MiI/AAAAAAAAdU8/6AaNmniUdSMrv2DXp0G8EQdGfmwNYOorgCPcBGAsYHg/w300-h400/20210104_134819.jpg" width="300" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Due to Covid-19 this section where you can have a meal at a Plus supermarket is closed...</td></tr></tbody></table></p><h3 style="text-align: left;"><span lang="EN-GB">And how
does this emerging consumer behaviour affect the analytics profession?</span></h3><p class="MsoNormal"><span lang="EN-GB">The larger
distribution chains will continue to develop their centralised analytical
systems. The data flows from the outlets’ cash registers to the central data
warehouse and delivers customer and product insights as this has been the case
since AC Nielsen built the first embryo of a retail data warehouse somewhere in
the seventies. <o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">New
opportunities for innovation in analytics for large retailers lie in edge
computing. Think of directed dialogues with the customer, analysing conversion
rates from looking at products, holding them, inspecting them and finally
putting the product in the shopping trolley and feeding it back to the pricing
and communication in the isles.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">Now, as
local shops discover the value of customer data, syndicators will emerge to
provide economies of scale and of scope to aggregate data of the local shops
and provide benchmarks and high level customer insights as a first deliverable.
It will take some serious investments in persuading the local shops to share
their data but it will happen in the next three to five years. My experience
with a data warehouse project for an association of independent retailers tells
me it’s doable if you mimic the architecture of epidemiological analytics. These
systems have the highest levels of information security combined with state of
the art analytical capabilities. And so another product of this pandemic may
contribute to new analytical solutions. <o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">But the
major shift in the analytics landscape is happening with the brand owners. Up
to now, most brand owners were OK with the idea that customer behaviour data
resided in the systems of large retailers. Some of the clever ones developed a
data sharing approach with the retailers accepting the possibility of a biased
view on their final customers. <o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB">Now the
need for massive customer data for brand owners is unavoidable. New ways of
collecting unfiltered customer data will emerge. Smartphones, fit bits and
other devices will have new roles to play in this strategic movement. <o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-GB"> </span></p><p class="MsoNormal"><span lang="EN-GB"> </span></p><p class="MsoNormal"><span lang="EN-GB"> </span></p><p class="MsoNormal"><span lang="EN-GB">
</span></p><p class="MsoNormal"><span lang="EN-GB"> </span></p><p class="MsoNormal"><span lang="EN-GB" style="mso-ansi-language: EN-GB;"><br /></span></p>Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-49266595177071413752019-12-30T12:10:00.003+01:002019-12-30T14:20:46.509+01:00See you Next Year!<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-Ldda-HdCxSI/XgnaZ0PnnbI/AAAAAAAATDU/Ht6nM6KuvJoV41XlwQ7F4hyO3BjUr0ApwCLcBGAsYHQ/s1600/New%2BYear%2B2020.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="New Year's resolutions for 2020" border="0" data-original-height="1600" data-original-width="709" src="https://1.bp.blogspot.com/-Ldda-HdCxSI/XgnaZ0PnnbI/AAAAAAAATDU/Ht6nM6KuvJoV41XlwQ7F4hyO3BjUr0ApwCLcBGAsYHQ/s1600/New%2BYear%2B2020.jpg" title="Seasons Greetings and a Happy New Year" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-49721852994529010752019-10-30T06:00:00.000+01:002019-10-31T09:27:34.488+01:00Enterprise Architectures for Artificial Intelligence (III)<br />
<h2>
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Taxonomies of Artificial Intelligence</span></h2>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">There are at least five ways to position AI in
the enterprise landscape:<o:p></o:p></span></div>
<div class="MsoNormal">
</div>
<ol>
<li><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">By processing method: batch, micro batch and
real time</span></li>
<li><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">By algorithm type: pattern recognition,
clustering, associations, scoring, predictive, classification, text, speech and
image mining, …</span></li>
<li><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">By data type: high vs low dimensionality,
graph data, self-describing data vs structured schema data, machine vs human
sourced data, mediated data registration vs direct data registration,…</span></li>
<li><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">By data behaviour: volatile vs stable data
values, long vs data persistency, </span></li>
<li><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">By analytics goals and or business process:
churn prevention, prospect qualification, complex evaluations of loan
applications, CVs, customer feedback, basket analysis, next best action
proposals, fraud detection …</span></li>
</ol>
<br />
<div class="MsoNormal">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">The enterprise architect will choose the relevant
combinations between these taxonomies to produce a coherent end-to-end vision
on the architecture. A possible selection criterion is the governance model
used in the organisation. In a business monopoly analytics goals will be
leading and combined with algorithm type. In an<span style="mso-spacerun: yes;">
</span>IT monopoly processing methods combined with data behaviour is the most
probable direction and in a duopoly, well,… that depends.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Let’s do an exercise and suppose this is the
outcome of a duopoly governance model: combining the processing method with the
algorithm type to indicate which processing method is most suited for the
chosen algorithm type. Using this schema may help to manage expectations
between the business and the IT people better.<o:p></o:p></span></div>
<br />
<br />
<table border="1" cellpadding="0" cellspacing="0" class="MsoTable15Grid4Accent1" style="border-collapse: collapse; border: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-padding-alt: 0cm 5.4pt 0cm 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: -1; mso-yfti-lastfirstrow: yes;">
<td style="background: #5B9BD5; border-right: none; border: solid #5B9BD5 1.0pt; mso-background-themecolor: accent1; mso-border-bottom-alt: solid #5B9BD5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-left-alt: solid #5B9BD5 .5pt; mso-border-left-themecolor: accent1; mso-border-themecolor: accent1; mso-border-top-alt: solid #5B9BD5 .5pt; mso-border-top-themecolor: accent1; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 5;">
<br /></div>
</td>
<td style="background: #5B9BD5; border-bottom: solid #5B9BD5 1.0pt; border-left: none; border-right: none; border-top: solid #5B9BD5 1.0pt; mso-background-themecolor: accent1; mso-border-bottom-alt: solid #5B9BD5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themecolor: accent1; mso-border-top-alt: solid #5B9BD5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themecolor: accent1; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 1;">
<b><span lang="EN-GB" style="color: white; mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin; mso-themecolor: background1;">Batch<o:p></o:p></span></b></div>
</td>
<td style="background: #5B9BD5; border-bottom: solid #5B9BD5 1.0pt; border-left: none; border-right: none; border-top: solid #5B9BD5 1.0pt; mso-background-themecolor: accent1; mso-border-bottom-alt: solid #5B9BD5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themecolor: accent1; mso-border-top-alt: solid #5B9BD5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themecolor: accent1; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 1;">
<b><span lang="EN-GB" style="color: white; mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin; mso-themecolor: background1;">Micro
batch<o:p></o:p></span></b></div>
</td>
<td style="background: #5B9BD5; border-left: none; border: solid #5B9BD5 1.0pt; mso-background-themecolor: accent1; mso-border-bottom-alt: solid #5B9BD5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-right-alt: solid #5B9BD5 .5pt; mso-border-right-themecolor: accent1; mso-border-themecolor: accent1; mso-border-top-alt: solid #5B9BD5 .5pt; mso-border-top-themecolor: accent1; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 1;">
<b><span lang="EN-GB" style="color: white; mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin; mso-themecolor: background1;">Real
time<o:p></o:p></span></b></div>
</td>
</tr>
<tr style="mso-yfti-irow: 0;">
<td style="background: #DEEAF6; border-top: none; border: solid #9CC2E5 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 68;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">pattern recognition<o:p></o:p></span></b></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Ideal method for large data sets<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Suited for simple patterns<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Only as a binary in/out of pattern decision
which implies a large (batch) training set <o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 1;">
<td style="border-top: none; border: solid #9CC2E5 1.0pt; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 4;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">clustering<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Ideal method for large data sets<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Suited for simple clustering criteria<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Only as Y or N adherence to an existing cluster which implies a
large (batch) training set<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 2;">
<td style="background: #DEEAF6; border-top: none; border: solid #9CC2E5 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 68;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">associations<o:p></o:p></span></b></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Ideal method<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<br /></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Hardly possible<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">impossible<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 3;">
<td style="border-top: none; border: solid #9CC2E5 1.0pt; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 4;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">scoring<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Develop a base line <o:p></o:p></span></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Adjust the base line<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Score against the base line<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 4;">
<td style="background: #DEEAF6; border-top: none; border: solid #9CC2E5 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 68;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">predictive<o:p></o:p></span></b></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Develop a base line <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<br /></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Adjust the base line<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Match with the trend<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 5;">
<td style="border-top: none; border: solid #9CC2E5 1.0pt; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 4;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">classification<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Train the dataset<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Classify new data<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Simple classification<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 6;">
<td style="background: #DEEAF6; border-top: none; border: solid #9CC2E5 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 68;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">text mining<o:p></o:p></span></b></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Train the dataset<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Reveal polarity, topics, etc…<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Deliver alerts<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 7;">
<td style="border-top: none; border: solid #9CC2E5 1.0pt; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 4;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">speech mining<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Train the dataset<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Reveal polarity, topics, etc…<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Deliver alerts<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 8; mso-yfti-lastrow: yes;">
<td style="background: #DEEAF6; border-top: none; border: solid #9CC2E5 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-themecolor: accent1; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 68;">
<b><span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Image & video mining<o:p></o:p></span></b></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.25pt;" valign="top" width="139"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Train the dataset<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="140"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Classify images<o:p></o:p></span></div>
</td>
<td style="background: #DEEAF6; border-bottom: solid #9CC2E5 1.0pt; border-left: none; border-right: solid #9CC2E5 1.0pt; border-top: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid #9CC2E5 .5pt; mso-border-bottom-themecolor: accent1; mso-border-bottom-themetint: 153; mso-border-left-alt: solid #9CC2E5 .5pt; mso-border-left-themecolor: accent1; mso-border-left-themetint: 153; mso-border-right-themecolor: accent1; mso-border-right-themetint: 153; mso-border-themecolor: accent1; mso-border-themetint: 153; mso-border-top-alt: solid #9CC2E5 .5pt; mso-border-top-themecolor: accent1; mso-border-top-themetint: 153; padding: 0cm 5.4pt 0cm 5.4pt; width: 113.3pt;" valign="top" width="143"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Deliver alerts<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-yfti-cnfc: 64;">
<br /></div>
</td>
</tr>
</tbody></table>
<br />
<br />
<br />
<div class="MsoNormal">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">From this crosstab, it becomes possible to position
the concrete algorithms, the data sets and their life cycle management, the
ingestion volumes, timing and the technology to deliver on the various promises
made. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Other methods will give you paths to the same
end result: a coherent and methodical inventory of the landscape, linking
business processes to AI and data mining initiatives and routines as well as
the data and the applications to deliver the goods. Based on a gap analysis,
the enterprise architect can develop a roadmap that communicates with all
parties concerned.<o:p></o:p></span></div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-63690653396573611162019-09-30T20:35:00.000+02:002019-10-01T10:37:27.193+02:00Enterprise Architectures for Artificial Intelligence (II)<br />
<h3>
<span lang="EN-GB">A generic model for primary processes</span></h3>
<br />
<br />
<div class="MsoNormal">
<span lang="EN-GB">Every organisation is unique but most
organisations share some basic principles in the way they operate. Business
processes have some form (between 5 and 100%) of support by online transaction
systems (OLTP). Business drivers like consumer demand, government regulations,
special interest groups, technological evolutions, availability of raw
materials and labour and many others influence the business processes intended
to deliver a product or service that meets market demand within a set of
constraints. These constraints can range from enforcing regulatory bodies to
voluntary self-regulation and measures inspired by public relations objectives.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">This is a high level approach of how AI can
support business processes <o:p></o:p></span></div>
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-oX4CMeOfPhg/XZHOdtMxz9I/AAAAAAAARAc/4j0CfXmA4fIoAHMTWEfxuZcByprqXiwCQCLcBGAsYHQ/s1600/hi%2Blevel%2Bgeneric%2Barchitecture.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="AI and enterprise architecture" border="0" data-original-height="269" data-original-width="605" height="284" src="https://1.bp.blogspot.com/-oX4CMeOfPhg/XZHOdtMxz9I/AAAAAAAARAc/4j0CfXmA4fIoAHMTWEfxuZcByprqXiwCQCLcBGAsYHQ/s640/hi%2Blevel%2Bgeneric%2Barchitecture.jpg" title="Generic architecture for artificial intelligence" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">High level generic architecture</td></tr>
</tbody></table>
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-layout-grid-align: none; text-autospace: none;">
<span lang="EN-GB" style="color: black; mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-bidi-font-size: 12.0pt; mso-font-kerning: 12.0pt; mso-hansi-font-family: Calibri;">Business
drivers are at the basis of business processes to realise certain business
goals and delivering products for an internal or external customer.<span style="mso-spacerun: yes;"> </span>These processes are supported by applications,
the so-called online transaction processing (OLTP) systems.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-layout-grid-align: none; text-autospace: none;">
<span lang="EN-GB" style="color: black; mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-bidi-font-size: 12.0pt; mso-font-kerning: 12.0pt; mso-hansi-font-family: Calibri;">Business
process owners formulate an a priori scoring model that is constantly adapted
by both microscopic transaction data as well as historic trend data from the
data warehouse (DWH). Both data sources can blend into decision support data,
suited for sharply defined data requirements as well as vague assumptions about
their value for decision making.<span style="mso-spacerun: yes;"> </span>The
decisions at hand can be either microscopic or macroscopic.<span style="mso-spacerun: yes;"> </span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-layout-grid-align: none; text-autospace: none;">
<span lang="EN-GB" style="color: black; mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-bidi-font-size: 12.0pt; mso-font-kerning: 12.0pt; mso-hansi-font-family: Calibri;"><span style="mso-spacerun: yes;"><br /></span></span></div>
<h3>
<span lang="EN-GB">Introducing AI in the business
processes</span></h3>
<br />
<div class="MsoNormal">
<div class="MsoNormal">
<span lang="EN-GB">As an architect one of the first decisions to
make is whether and when AI becomes relevant enough to become part of routine
business processes. There are many AI initiatives in organisations but the
majority is still in R & D mode or –at best- in project mode. It takes special skills to determine when the
transition to routine process management can provide some form of sustainable
added value.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">I am not sure if these skills are all
determined and present in the body of knowledge of architects but here are
some proposals for the ideal set of competences. <o:p></o:p></span></div>
<div class="MsoNormal">
</div>
<ul>
<li><span lang="EN-GB">A special form of requirements management
which you can only master if the added value as well as the pitfalls of AI in
business processes are thoroughly understood,</span></li>
<li><span lang="EN-GB">As a consequence, the ability to produce use
cases for the technology,</span></li>
<li><span lang="EN-GB">Master the various taxonomies to position AI
in a correct way to make sure you obtain maximum value from the technology (more
on this in a next post),</span></li>
<li><span lang="EN-GB">Have clear insights in the lifecycle
management of the various analytical solutions in terms of data persistency,
tuning of the algorithm and translation into appropriate action(s).</span></li>
</ul>
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-layout-grid-align: none; text-autospace: none;">
</div>
</div>
<div class="MsoNormal">
<span lang="EN-GB"></span><br />
<div class="MsoNormal">
<span lang="EN-GB">In the next post, I will elaborate a bit more
on the various taxonomies to position AI in the organisation. <o:p></o:p></span></div>
</div>
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-layout-grid-align: none; text-autospace: none;">
<span lang="EN-GB" style="color: black; mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-bidi-font-size: 12.0pt; mso-font-kerning: 12.0pt; mso-hansi-font-family: Calibri;"><span style="mso-spacerun: yes;"><br /></span></span></div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com4tag:blogger.com,1999:blog-4520438534640699001.post-17091779704892760252019-09-19T11:27:00.001+02:002019-09-30T11:32:17.365+02:00Enterprise Architectures for Artificial Intelligence (I)<br />
<div class="MsoNormal">
<span lang="EN-GB">In the past three decades, I have seen artificial
intelligence (AI) coming and going a couple of times. From studying MYCIN via
speech technology in Flanders Language Valley to today’s machine learning and
heuristics as used by <a href="https://www.textgain.com/" target="_blank">Textgain from Antwerp University,</a> the technology is here
to stay this time.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Why? Because the cost of using AI has
fallen dramatically not just in terms of hard and software but also in terms of
acquiring the necessary knowledge to master the discipline.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Yet, most of the AI initiatives are still
very much in the R&D phase or are used in limited scope. But here and
there, e.g. in big (online) retail and telecommunications, AI is gaining
traction on enterprise level. <span style="mso-spacerun: yes;"> </span>And
through APIs, open data and other initiatives, AI will become available for
smaller organisations in the near future.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">To make sure this effort has a maximum
chance of success, CIOs need to embed this technology in an enterprise
architecture covering all aspects: motivations, objectives, requirements and
constraints, business processes, applications and data.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Being fully aware that I am trodding on
uncharted territory, this article is –for now- my state of the art.<o:p></o:p></span></div>
<h4>
<span lang="EN-GB">Introducing AI in the capability map</span></h4>
<div class="MsoNormal">
AI will enhance our capabilities in all
areas of Treacy & Wiersema’s model, probably in a certain order. First comes
operational excellence as processes and procedures are easier to describe,
measure and monitor. Customer intimacy is the next frontier as the existing
discipline of customer analytics lays the foundation for smarter interactions
with customers and prospects.</div>
<div class="MsoNormal">
<span lang="EN-GB">The toughest challenge is in the realm of
product leadership. This is an area where creativity is key to success. There
is an approximation of creativity using what I call “property exploration”
where a dimensional model of all possible properties of a product, a service, a
marketing or production plan are mapped and an automatic cartesian product of
all levels or degrees of each property with all the other properties is
evaluated for cost and effectiveness. Sales pitch: if you want more information
about this approach, <a href="mailto:contact@linguafrancaconsulting.eu" target="_blank">contact us</a>.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-jrPjPU2pdYk/XYNIML8GQaI/AAAAAAAAQxc/fLELWJI4jIw2sa73LsXjpZA0IFzSJEGbgCLcBGAsYHQ/s1600/AI%2BCapability%2Bmap.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="Capabilities and AI" border="0" data-original-height="289" data-original-width="605" height="304" src="https://1.bp.blogspot.com/-jrPjPU2pdYk/XYNIML8GQaI/AAAAAAAAQxc/fLELWJI4jIw2sa73LsXjpZA0IFzSJEGbgCLcBGAsYHQ/s640/AI%2BCapability%2Bmap.png" title="Artificial Intelligence and a capability map" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Capabilities where state of the art AI can play a significant role</td></tr>
</tbody></table>
<div class="MsoNormal">
<span lang="EN-GB">Examples of capabilities where AI can play
a defining role. Some of these capabilities are already well supported, to name
a few: inventory management (automatic replenishment and dynamic storage),
cycle time management (optimising man-machine interactions), quality management
(visual inspection systems), churn management (churn prediction and avoidance
in CRM systems), yield management (price, customer loyalty, revenue and
capacity optimisation) and talent management (mining competences from CVs).<o:p></o:p></span></div>
<br />
<div class="MsoNormal">
<span lang="EN-GB">Areas where AI is coming of age: loyalty
management and competitive intelligence, R & D management and product
development. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">
</span></div>
<div class="MsoNormal">
<span lang="EN-GB"><i>In the next post I will discuss a generic
architecture for AI in support of primary processes; Stay tuned and… share your
insights on this topic!</i><o:p></o:p></span></div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-10031572443119554142019-03-12T16:00:00.000+01:002019-12-04T16:04:25.444+01:00Relevance Lost 2.0<br />
<div class="MsoNormal">
<i style="mso-bidi-font-style: normal;"><span lang="EN-US" style="mso-ansi-language: EN-US;">When Robert Kaplan and Thomas Johnson wrote
about the fall of management accounting systems and pleaded for new
perspectives and approaches to strategic process control they wrote a most
relevant statement that still stands today. But as organisations are
implementing performance measurement systems as applications of their business
intelligence infrastructure, news risks of losing the most relevant part of strategic
management arise. There is a clear need for a new, extended approach to the
balanced scorecard (BSC). If you wonder what this may be, then read more.<o:p></o:p></span></i><br />
<i style="mso-bidi-font-style: normal;"><span lang="EN-US" style="mso-ansi-language: EN-US;"><br /></span></i></div>
<div class="MsoNormal">
<span lang="EN-US" style="mso-ansi-language: EN-US;">In this
article I describe the generic strategy process, compare this with the
performance management process, how it is implemented in some organisations and
analyze the aspects of this process related to the organization’s leadership
style and thus the consequences for the balanced scorecard’s effectiveness.<o:p></o:p></span><br />
<span lang="EN-US" style="mso-ansi-language: EN-US;"><br /></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="mso-bidi-font-style: normal;"><span lang="EN-US" style="mso-ansi-language: EN-US;">The Strategy Process:
Formation and Formulation<i><o:p></o:p></i></span></span></b></div>
<div class="MsoNormal">
<span lang="EN-US" style="mso-ansi-language: EN-US;">Every
organisation has its own way of forming and formulating a strategy. It is in
the organizational DNA: the way strategic objectives are grounded on a clear
problem statement, SWOT analysis, competitive analysis, etc… and evaluated,
proper to that same DNA.<o:p></o:p></span></div>
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-J7ldoq9dUgQ/XefBemaR6rI/AAAAAAAASb0/f3j-K13PBcYbQSGBeQGJMcaqvy53J0aNACLcBGAsYHQ/s1600/Strategy%2Bprocess.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="407" data-original-width="605" height="428" src="https://1.bp.blogspot.com/-J7ldoq9dUgQ/XefBemaR6rI/AAAAAAAASb0/f3j-K13PBcYbQSGBeQGJMcaqvy53J0aNACLcBGAsYHQ/s640/Strategy%2Bprocess.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><div style="text-align: left;">
<b><i><span lang="EN-US" style="font-family: "calibri" , sans-serif; font-size: 10.0pt;">Fig 1. The Strategy Process</span></i></b><i><span lang="EN-US" style="font-family: "calibri" , sans-serif; font-size: 10.0pt;"> The strategic objectives entail the position of
the company for mid- and short term, wanted behavior from clients, associates
and partners. These objectives are then dissected, evaluated, scored and
appreciated according to the organization’s values and beliefs, the
organizational culture and past experience. </span></i></div>
<div style="font-style: italic; text-align: left;">
<i><span lang="EN-US" style="font-family: "calibri" , sans-serif; font-size: 10.0pt;">The last phase is the decision
phase where the objectives are translated into concrete actions and measurable
products and outcomes of these actions.</span></i></div>
<span lang="EN-US"></span><br />
<div style="font-style: italic; text-align: left;">
<span lang="EN-US"><span style="font-family: "calibri" , sans-serif; font-size: 13.3333px;"><br /></span></span></div>
<span lang="EN-US">
<div style="text-align: left;">
<div style="font-style: italic;">
<span style="font-family: "calibri" , sans-serif; font-size: 13.3333px;"><br /></span></div>
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">As the illustration shows, the strategy formation process is usually done in four process steps:</span></span><br />
<br />
<ul>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">definition, </span></span></div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">dissection, </span></span></div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">evaluation and </span></span></div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">decision</span></span></div>
</span></li>
</ul>
<br />
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">In the definition process, the strategic problems and opportunities are described and expressed in statements like: “We are not well positioned in the high end market”. Input for this process step is SWOT analysis, as well as analysis of the competition and the customer base. </span></span><br />
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">In the dissection process, expressions like “We are not well positioned…” are set in context and translated into critical success factors, e.g. “In order to improve our position in the high end market we need to upgrade our image, educate the sales force, identify more high end customers.”</span></span><br />
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">The eva<span style="white-space: pre;"> </span>luation process is mainly concerned with prioritizing the dissected elements and looking for causal relationships between these elements to prepare a cohesive and consistent strategic plan which can be communicated to all parties concerned.</span></span><br />
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">Finally in the decision process, the prioritized and feasible targets and ways to achieve these targets with global action plans are chosen. Some targets are based on decisions with tangible and measurable results and others are based on long term decisions with immaterial and almost immeasurable outcomes. </span></span><br />
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">The following decision points are ranked from very concrete and “hard” targets to more “soft” and less quantifiable decisions:</span></span><br />
<br />
<ul>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">which critical performance indicators will receive priority in the action plans and budget allocation,</span></span></div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">which actions are needed in the market place,</span></span></div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">which organizational behavior needs adjusting</span></span></div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">what needs to be improved in the organization’s culture</span></span></div>
</span></li>
</ul>
<br />
<span style="font-size: 13.3333px;"><span style="font-family: calibri, sans-serif;"></span></span><br />
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;">In the strategy formulation process steps, communication exercises will adjust these elements to match them with the various target groups: associates, clients, suppliers, shareholders, government officials, press, etc…</span></span><br />
<div style="font-style: italic;">
<br /></div>
</div>
<div style="text-align: left;">
<div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">The Degrees of
Collaboration in the Strategy Process<o:p></o:p></span></b></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">If we
imagine a continuum in leadership styles, ranging from autocratic to democratic
leadership, the four strategy formation process steps will be subject to shared
inputs, transformation and outputs on a scale of zero to hundred percent.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">In an <b>autocratic leadership</b> all four process
steps are in the hands of the leader. Only the leader decides what is important
in the environmental scan, what the objectives are, how the actions are prioritized. Some autocratic leaders have
problems formulating the strategic plan as they still hold up the adage
“knowledge is power”. What they gain on control may get lost in the execution
phase when their subordinates try to interpret their ambiguous communication. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Long time
ago, I studied the biographies of dictators (which names I won’t mention
because they deserve to be forgotten) and it struck me that often they
communicated very vaguely about their strategic vision, priorities, objectives
and the way to accomplish these objectives. Zealous and ambitious subordinates
would then translate these cryptic messages into complete (and often horrific)
action plans which would then be meticulously and ruthlessly executed. After
which the dictator either rewarded the zealot or had him sent in exile because
he became too popular or was a liability for the regime to the outside world. Nevertheless, autocratic leaders can be highly
successful in process industries, retail, service organisations and this
leadership style emerges everywhere there is a crisis and fast response times
are more important than a well pondered decision making process, balancing all
market factors, interests and wishes in the organisation.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"> In the <b>democratic
leadership</b> all four process steps may be in the hands of the entire team.
Lengthy discussion and negotiation may require a lot of resources but the
upside is that everyone in the organisation is on the same page and the
execution phase has less need for control and clear instructions as the
organisation members act in a more autonomous way, responding faster to changes
on the terrain. Organisations of professionals prefer democratic leadership as
knowledge and competence are far more important than rank and power and the
status that go with it. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Finally in <b>mixed forms</b>, the definition process may
be initiated by the strategic apex, shared with the ranks and business analysts
may be called in to dissect the statements into
manageable chunks.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">The
evaluation and the decision process, depending on the level of democratic
leadership, may be done by the leader, a management team or a management team
extended with staff members and analysts. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Today, as
organisations become flatter and more democratic leadership styles are proper
to new industries, there is a need for faster feedback loops to combine the
advantages of autocratic leadership with the responsiveness of democratic
organisations.<o:p></o:p></span></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<br /></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">The Performance Management
Process<o:p></o:p></span></b></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">The
strategy process, seen from a performance management perspective, is a
machine-like approach to define, monitor and manage the actions as defined in
the strategic plan. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Let’s see
how the strategy process is broken down into the performance management
process. The performance management process breaks down the strategy formation process
into smaller chunks to decompose the formation (definition, dissection,
evaluation and decision) into nine steps. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">This is a
top down exercise: <o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">Analyze
the situation (SWOT, competition,…) (Definition Process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">Determine
the objectives after the analysis (Definition Process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">Define
the critical success factors (CSF) (Definition & Dissection Process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">Derive
the critical performance indicators (CPI) from these CSFs (Definition &
Dissection Process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">Mapping
the CPIs on the organizational units down to the individual associate
(Dissection & Evaluation Phase)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">Adapting
the HRM policies to these mappings (if trade unions allow, of course)
(Evaluation Process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">monitor,
manage and readjust the CPIs (Execution process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">monitor,
manage and readjust the CSFs (Execution process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">
</span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">Adapt the
objectives to the new SWOT results (Execution process)</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="font-family: calibri, sans-serif; font-size: 13.3333px; text-indent: -18pt;">
<br /></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Remark how
implicit the decision process is embedded in the dissection and evaluation
process steps of the performance management approach.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">Strategy automation?<o:p></o:p></span></b></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">In the past
ten years I have worked on IT-support for balanced scorecards (BSC) in a
university, a bank, an insurance company and a manufacturing company. In all of
these cases, a poignant conclusion was unavoidable: “If all you use is a
hammer, everything starts looking like a nail”. The ICT tool became a
substitute for the strategy process and forced a freeze on the organisation.
Let me explain this. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">The BSC was
used as an instrument to implement a top down strategic governance of the
organisation. The strategy decomposition as described above is then modeled in
the ICT tool creating links and correlations between the various CSFs and CPIs.
Identifying these cause and effect chains is not a trivial matter. If sales go
south, all sorts of explanations may present themselves to the organisation.
E.g.: Are lower sales due to:<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">… lower
consumer confidence?</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">… a
competitive move?</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">… a
government announcement?</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">… simple
seasonality or a gradual shift in seasonality?</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">… the
weather?</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">
</span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">… all of
the above?</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="font-family: calibri, sans-serif; font-size: 13.3333px; text-indent: -18pt;">
<br /></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Or can the
sales slump be explained by a factor we can influence like the number of sales
training hours received by the sales reps? But then the question arises if
there is a correlation between the amount and quality of the training received
and employee satisfaction? Or is it the other way around: because our employees are not very satisfied
with their job, they respond poorly to the training received<a href="file:///D:/01_Data/3_Marketing/BookLaunch/Relevance%20Lostversie20121205.docx#_edn1" name="_ednref1" title=""><span class="MsoEndnoteReference"><!--[if !supportFootnotes]--><span class="MsoEndnoteReference"><span lang="EN-US" style="font-family: "Calibri",sans-serif; font-size: 11.0pt; mso-ansi-language: EN-US; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">[i]</span></span><!--[endif]--></span></a>?<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">But that is
not all. As we all know, strategic management is about adapting to the
environment. If the ICT tool does not capture the environmental change either bottom up or
top down, then what? There is also another side of the coin: if the strategic
decomposition leads to individual targets, personal development plans and other
HRM tools how does this affect the flexibility of the organisation to adapt to
change?<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">People act
accordingly to the incentives from management: either they integrate the CPIs
in their work planning and their approach to the job or they look for ways to
beat the system. I remember sales people holding back order forms for a yearly
publication to “smoothen” the CPI measures of bookings per month since
management did not take seasonality into account when the performance indicator
was defined.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">Strategy Dialectics
Are the Way Forward</span></b><span lang="EN-US"><o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">It is clear
that the latter is unwanted behavior but those who conform with the system
should be rewarded, shouldn’t they?<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">The answer
is an ambiguous “Yes and No”. “Yes” if their response, steered and governed by
the performance indicator, is in sync with customer demand. And “No” if this is
not the case. Needless to add that any strategy which is not sanctioned by your
customers is not worth the paper it is written on. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">But how are
the designer, the monitor and the manager of the balanced scorecard to know
this? Henry Mintzberg (1994)<a href="file:///D:/01_Data/3_Marketing/BookLaunch/Relevance%20Lostversie20121205.docx#_edn2" name="_ednref2" title=""><span class="MsoEndnoteReference"><!--[if !supportFootnotes]--><span class="MsoEndnoteReference"><span lang="EN-US" style="font-family: "Calibri",sans-serif; font-size: 11.0pt; mso-ansi-language: EN-US; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">[ii]</span></span><!--[endif]--></span></a>
makes the distinction between intended and emergent strategies and the way I
see it and experienced it, the balanced scorecard is an almost perfect tool for
managing intended strategies. It uses a negative feedback loop, just like a
thermostat. And just like a thermostat it sometimes oversees the efforts needed
to keep everybody in line with the intended strategy. So people who don’t meet
their targets are stimulated to do so or they are made redundant if they are
not likely to comply with the desired behavior. But as John Lennon so rightly
said “life is what happens while you are making other plans”. Management may
have misinterpreted the signals from the environment or changes in the market may
be unnoticed by management. In that case,
emergent strategies may provide the answers to these situations as there
is some form of “wisdom of the crowds” in the collective response from front
office workers and anyone else who is in contact with customers, competitors,
prospects, suppliers, researchers and government officials, to name a few. To
capture these emergent strategies, the system needs to provide positive
feedback loops to reinforce unplanned but successful behavior, even when it is
non compliant with the intended strategy. In other words, if top management
makes a mistake, it will get noticed in three to five years but if the front
office worker makes a mistake, the organisation has an acute problem.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">This calls
for a special form of management, allowing dissidence in the ranks and
considering experimenters and contrarians as assets instead of a liability. “Is
this May 1968 all over again, when it was forbidden to forbid?”, I hear you
say. No, thanks. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">But imagine
an organisation form where the exchange between the hierarchy and the ranks is
formalized, open, unbiased and where everyone’s fact findings and opinions are
accessible to everyone for discussion, refining and leading to decisions and
actions.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Imagine a
special form of knowledge management which goes further than a glorified chat
room and text mining. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Imagine a
system supporting both bottom up and top down strategy processes, using the
collective wisdom of the entire organisation. Technology may be able to design
and build such a system but if management is not prepared to adapt its ways of
developing, forming and formulating strategies then the developers needn’t
bother.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<br /></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">Knowledge Management
and Performance Measurement Systems in Modern Organisations<o:p></o:p></span></b></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Remember
the initial point I was making: intended strategy is only a partial explanation
of the realized strategy because emergent or grassroots strategies contribute
to-or reduce- the results of the intended strategy. Since no entrepreneur or
manager likes to be only partially in control, we need a new approach to the
balanced scorecard implementation. Maybe that won’t be enough, maybe we need to
extend the scorecard’s toolset. <o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">What if the exchange process were more
important than the results of it?<o:p></o:p></span></b></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">What if the
true outcome of the dialectic strategy process were -other than a plan with
measurable results:<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">enhanced
motivation because people see the context, the bigger picture and have
contributed to it,</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">a shared
vision and sense of direction that enhances group cohesion,</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">a higher
level of entropy, turning each individual into autonomous decision making
entities without the usual chaotic side effects,</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">
</span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 18pt; text-indent: -18pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: Symbol; font-size: 10pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US" style="font-family: Calibri, sans-serif; font-size: 10pt;">increased
responsiveness to changing conditions or unexpected phenomena in the market?</span><span lang="EN-GB" style="font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="font-family: calibri, sans-serif; font-size: 13.3333px; text-indent: -18pt;">
<span lang="EN-US" style="font-family: Symbol; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">What if the strategy process became a strategy
dialogue?<o:p></o:p></span></b></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">What if the
system could capture the dialogues between the workers and:</span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->middle- and top management,<o:p></o:p></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->the customers,<o:p></o:p></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->the suppliers,<o:p></o:p></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->consultants,<o:p></o:p></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->academics,<o:p></o:p></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->opinion leaders,<o:p></o:p></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->government officials,<o:p></o:p></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->the data warehouse,<o:p></o:p></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">
</span></div>
<div class="MsoListParagraph" style="font-family: calibri, sans-serif; font-size: 13.3333px; margin-left: 53.4pt; text-indent: -35.4pt;">
<!--[if !supportLists]--><span style="font-family: "Calibri",sans-serif; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]-->the external information sources?<o:p></o:p></div>
<div class="MsoListParagraphCxSpFirst" style="font-family: calibri, sans-serif; font-size: 13.3333px; text-indent: -18pt;">
<br /></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">What if this dialogue were supported by a tool
requiring almost no extra effort from the organisation?<o:p></o:p></span></b></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Let’s
examine the actions people perform in an office which are –often without
knowing- valuable strategic information bits and are already captured partly or
wholly by the existing systems.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Searches on
the Intra- and Internet,<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Consulting
information providers ranging from Wikipedia to academic and government
sources,<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Sending and
receiving e-mails,<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Creating
and reading documents like meeting notes, documentation, process descriptions,
…<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Handling
customer complaints<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Dunning
customers,<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Online and
offline meetings, chats,<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">Analysis
and decision making,<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">etc…<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">All these
activities leave traces in some or another information system. What if you
could combine the most relevant words and constructs into input for your
strategic plan, supported by a balanced scorecard approach but avoiding a rigid
approach to the strategic process management? Let me have a go at specifying
such a system and check if the technology is already available.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><span lang="EN-US">Performance
Measurement Systems 2.0., a Functional Specification</span></b><span lang="EN-US" style="background: yellow; mso-ansi-language: EN-US; mso-highlight: yellow;"><o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">The short
description of this system is: ”A Collaborative Strategy Process Manager” The
architectural view is visualized in the schema below.<i><o:p></o:p></i></span></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
</div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<br /></div>
<div class="separator" style="clear: both; font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic; text-align: center;">
<a href="https://1.bp.blogspot.com/-n5FEFaqOeuE/XefDBSU1OmI/AAAAAAAAScA/CWlUcd1Oh-E3mvY4tRufchFygRAfB0oMgCLcBGAsYHQ/s1600/strategy%2Bmanagement.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="372" data-original-width="600" height="396" src="https://1.bp.blogspot.com/-n5FEFaqOeuE/XefDBSU1OmI/AAAAAAAAScA/CWlUcd1Oh-E3mvY4tRufchFygRAfB0oMgCLcBGAsYHQ/s640/strategy%2Bmanagement.png" width="640" /></a></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<br /></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<!--[if !supportEndnotes]--><b><i><span lang="EN-US" style="font-size: 10pt;">Fig. 2. The Core Architecture of a Collaborative Strategy Process Manager </span></i></b><i><span lang="EN-US" style="font-size: 10pt;">The architectural picture consists of three interconnected pillars: strategy, individual development and knowledge support. For simplicity reasons we leave aspects like servers, APIs and user interfaces out of the schema. Only the relevant functional blocks are listed. Remark that the planning and execution of the strategy is not in scope. Tools like ERP and CRM can provide the necessary support for that part.</span></i></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<br /></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<div class="MsoNormal">
<b><span lang="EN-US">The Organizational Strategy Management Process:</span></b><span lang="EN-US"> this is basically the support for
the balanced scorecard with a link to the personal development plans of the
people needed to execute the strategy. We refer to vendors like QPR, SAS, and
others to discover the features of a balanced scorecard software. But the link
to personal development plans (PDP) needs to be established. Imagine this PDP
as a database with relevant and “nice to have” competence development plans
which are maintained during appraisal and evaluation interviews. If the HRM application
can make the link with salary scales, a proper analysis can match the desired
competences with the future wage cost trend. <o:p></o:p></span></div>
<div class="MsoNormal">
<b><span lang="EN-US">The Individual Development Support: </span></b><span lang="EN-US">this is a personal balanced
scorecard where the interconnections between interests, competences, knowledge
and the track record inside and outside the organisation are managed. Imagine a
kind of personal LinkedIn with extra depth in the competence area. Instead of
generic labels like “marketing” one would find hierarchies like “Marketing >
Market Research>Qualitative Market Research>Focus Groups>Brand
Experience” <o:p></o:p></span></div>
<div class="MsoNormal">
<b><span lang="EN-US">The Knowledge Support System:</span></b><span lang="EN-US"> this pillar is not readily
available of the shelve, but some components are. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">The first
block <b>Search Engine Logs and Ratings </b>bundles the information search
behavior, the rating of the results but also the ratings of colleagues search
results, and all other communication and processed information. It also
includes ratings for decisions to be made by a group or an individual be it
multicriteria analysis, simple voting, rule based decision making or more
complex algorithms like ELECTRE. Bundling the information search and decision
behavior can yield interesting results for the knowledge management team
answering questions like “What information did he/she (or didn’t!) look for
before making a decision?” <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">The <b>Object
Database</b> is the engine behind
the object aware user interface, suggesting hyperlinks whenever users integrate
(potential) knowledge objects like information sources, notions, definitions,
persons, etc… in their communication. This forces users to be clear about what
they communicate and to what ontologies<a href="file:///D:/01_Data/3_Marketing/BookLaunch/Relevance%20Lostversie20121205.docx#_edn1" name="_ednref1" title=""><span class="MsoEndnoteReference"><!--[if !supportFootnotes]--><span class="MsoEndnoteReference"><span lang="EN-US" style="font-family: "Calibri",sans-serif; font-size: 11.0pt; mso-ansi-language: EN-US; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">[i]</span></span><!--[endif]--></span></a>
their concepts relate to. These objects, ready for the object aware user
interface are structured and edited by a knowledge manager or by the group,
depending on the configuration which supports both autocratic and democratic
environments. But the object database will also act as a repository for
unstructured data which can then be presented to the group or the knowledge manager
by “emerging publishers”. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">The Object
Database also disposes of easy configurable agents which respond to events or
trigger events themselves, e.g. “Present the weekly list of most used
ontologies”.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">The <b>Knowledge
Modeling System</b> is where ontologies, learning blocks and documentation
blocks are created and managed. These learning blocks and documentation blocks
are complete sentences or groups of sentences whether or not combined with
illustrations to create the basic material for documentation like ISO process
descriptions and procedures, help files and learning material for distance
learning or learning on the job.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Finally, the <b>E-Learning System</b> is where the
previous material is used in learning paths, documentation maps and presented
in a presentation layer which can be simple text and image, sound and/or video.
The material can be used on purpose or pop up spontaneously whenever users struggle
with knowledge gaps. <o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span lang="EN-US">The three connections:</span></b><span lang="EN-US"> the first connection is between the
organizational strategy management and individual development tools. With this
link, important questions can be answered and reality checks become
possible. A sample of these questions
and checks: <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="text-indent: -18pt;">
<span lang="EN-US" style="font-family: Symbol; text-indent: -18pt;"><span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span><span lang="EN-GB" style="font-family: Calibri, sans-serif; text-indent: -35.4pt;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><span lang="EN-GB" style="text-indent: -35.4pt;">“Do
we have the competences in house to deliver the desired actions?”</span><!--[if !supportLists]--></div>
<div class="MsoListParagraph" style="margin-left: 53.4pt; mso-list: l0 level1 lfo1; text-indent: -35.4pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: "Calibri",sans-serif; mso-ansi-language: EN-GB; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-GB">“How
big is the knowledge gap we need to close”<o:p></o:p></span></div>
<div class="MsoListParagraph" style="margin-left: 53.4pt; mso-list: l0 level1 lfo1; text-indent: -35.4pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: "Calibri",sans-serif; mso-ansi-language: EN-GB; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-GB">“Where
are biggest obstacles for change management?“<o:p></o:p></span></div>
<div class="MsoListParagraph" style="margin-left: 53.4pt; mso-list: l0 level1 lfo1; text-indent: -35.4pt;">
<!--[if !supportLists]--><span lang="EN-GB" style="font-family: "Calibri",sans-serif; mso-ansi-language: EN-GB; mso-fareast-font-family: Calibri;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-GB">“What
unused competences suggest opportunities for new strategic directions?”<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">The second
connection, between individual development and knowledge support tools makes
knowledge management more manageable by delivering information to questions
like:</span></div>
<div class="MsoNormal">
</div>
<ul>
<li><span lang="EN-US"><div style="text-align: left;">
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<div>
<div class="MsoNormal">
<span style="font-size: 10pt;">“What
knowledge objects are most used by individuals, groups, teams, divisions,… ?”</span></div>
</div>
</div>
</div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<div>
<div class="MsoNormal">
<span style="font-size: 10pt;">“When
and where does who contribute to the development or the extension of knowledge
objects?”</span></div>
</div>
</div>
</div>
</span></li>
<li><span lang="EN-US"><div style="text-align: left;">
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<div>
<div class="MsoNormal">
<span style="font-size: 10pt;">“What
e-courses have been successfully taken by whom?”</span></div>
</div>
</div>
</div>
</span></li>
</ul>
<div class="MsoNormal">
<span lang="EN-US">The third
connection, between the strategy management support and the knowledge support
challenges assumptions and probes for answers to questions like:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="text-indent: -18pt;">
</div>
<span lang="EN-US" style="font-family: Symbol; text-indent: -18pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span><span lang="EN-US" style="text-indent: -18pt;">“Do
we have the necessary capacity for the personal development plans?”</span><br /><span lang="EN-US" style="font-family: Symbol; text-indent: -18pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><span lang="EN-US" style="text-indent: -18pt;">“Have
we made the CSFs and KPIs sufficiently operational for the workforce to
understand, adopt and apply them?”</span><br /><span lang="EN-US" style="font-family: Symbol; text-indent: -18pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><span lang="EN-US" style="text-indent: -18pt;">“What
level of comprehension of the strategy matches with a certain level of
compliance to the proposed KPIs?”</span></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US" style="font-family: Symbol; text-indent: -18pt;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span><span lang="EN-US" style="text-indent: -18pt;">“What new information can influence adjustments to the initial strategy?</span><span style="font-size: 12.8px;">“What new information can influence
adjustments to the initial strategy?”</span></div>
<div class="MsoNormal">
<span lang="EN-GB"><o:p></o:p></span></div>
<div>
<div style="text-indent: -24px;">
<span style="font-family: calibri, sans-serif;"><span style="font-size: 13.3333px;"><br /></span></span></div>
<ul style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<li><span lang="EN-US"><div style="text-align: left;">
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<div>
<div class="MsoListParagraphCxSpLast" style="text-indent: -18pt;">
<span lang="EN-US">
<!--[endif]--><o:p></o:p></span></div>
</div>
</div>
</div>
</span></li>
</ul>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US">If the
“What” and the “Why” are implemented properly, the “Where” and the “How” will
become easier to manage in an adaptive way because management control 2.0. will
increase self control with individuals and groups but leave enough room f</span>or new
initiatives to respond better to consumer demand.</div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div class="separator" style="clear: both; font-family: calibri, sans-serif; font-size: 13.3333px; text-align: left;">
<a href="https://1.bp.blogspot.com/-nasCCDgajPg/XefFUPGHUoI/AAAAAAAAScM/BKNbbpq5Oh0jq_zijU8vDsutRfdnuzxRgCLcBGAsYHQ/s1600/strategic%2Bdirections.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="309" data-original-width="474" height="416" src="https://1.bp.blogspot.com/-nasCCDgajPg/XefFUPGHUoI/AAAAAAAAScM/BKNbbpq5Oh0jq_zijU8vDsutRfdnuzxRgCLcBGAsYHQ/s640/strategic%2Bdirections.png" width="640" /></a></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<b><i><span lang="EN-GB" style="font-size: 10.0pt; mso-bidi-font-size: 11.0pt;">Fig.
3 The Strategy Execution Process</span></i></b><i><span lang="EN-GB" style="font-size: 10.0pt; mso-bidi-font-size: 11.0pt;"> As
the illustration suggests, there are two major directions to take when
executing a strategic plan: changing the rules of the game or the
organisational behaviour in case the results are disappointing or optimize the
existing, successful strategy. The “how”
to this “what” can be both shock therapy or an incremental “frog in the pot of
tepid water” approach. ultimately it will lead to managing the organisational
competences, be they individual or group competences. The “where” is in
projects or in processes, i.e. in new ventures or in routine things people do
in the organisation. The “why” is not in this picture as it is supported by the
collaborative strategy process manager as described above.<o:p></o:p></span></i></div>
<div class="separator" style="clear: both; font-family: calibri, sans-serif; font-size: 13.3333px; text-align: center;">
<br /></div>
<div class="MsoNormal" style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<span lang="EN-US"><br /></span></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px;">
<div class="MsoNormal">
<b><span lang="EN-US">Conclusion<i><o:p></o:p></i></span></b></div>
<div class="MsoNormal">
<span lang="EN-US">Strategy
management is a slightly more complex phenomenon than the cybernetic view some
scholars and managers have. An organization is a living thing, the environment
is something even the biggest organisations can’t control (unless they are in a
socialist island republic). Therefore, adaptive strategy management is the way
forward.</span><span lang="EN-US"> </span><span lang="EN-GB"><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Strategy
management 2.0. will be adaptive or it will become obsolete in a flattened
society, where successful organisations in the new economy have exchanged the
hierarchical, top down, cybernetic management paradigm for a customer centric,
responsive and adaptive organisation where people are motivated, empowered and
share a clear vision, a sense of purpose and understand the general direction the organisation is heading. Only
that way, these organisations can face the challenges of a mobile, fragmented
and volatile generation Z and build a sustainable business.<i><o:p></o:p></i></span></div>
</div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<br /></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<br /></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<br /></div>
<div style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<!--[if !supportEndnotes]--><br clear="all" />
<hr align="left" size="1" width="33%" />
<!--[endif]-->
<div id="edn1">
<div class="MsoEndnoteText">
<a href="file:///D:/01_Data/3_Marketing/BookLaunch/Relevance%20Lostversie20121205.docx#_ednref1" name="_edn1" title=""><span class="MsoEndnoteReference"><span lang="EN-GB"><!--[if !supportFootnotes]--><span class="MsoEndnoteReference"><span lang="EN-GB" style="font-family: "Calibri",sans-serif; font-size: 10.0pt; mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">[i]</span></span><!--[endif]--></span></span></a><span lang="EN-GB"> </span><span lang="EN-US">I use the
definition from Tom Gruber from Stanford University: “</span><span lang="EN-GB">In
the context of knowledge sharing, I use the term ontology to mean a <i>specification
of a conceptualization</i>. That is, an ontology is a description (like a
formal specification of a program) of the concepts and relationships that can
exist for an agent or a community of agents. This definition is consistent with
the usage of ontology as set-of-concept-definitions, but more general. And it
is certainly a different sense of the word than its use in philosophy.” From:
T. R. Gruber. “A translation approach to portable ontologies.” <i>Knowledge
Acquisition</i>, 5(2):199-220, 1993<o:p></o:p></span></div>
<div class="MsoEndnoteText">
<br /></div>
</div>
</div>
<hr align="left" size="1" style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;" width="33%" />
<!--[endif]-->
<div id="edn1" style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<div class="MsoEndnoteText">
<a href="file:///D:/01_Data/3_Marketing/BookLaunch/Relevance%20Lostversie20121205.docx#_ednref1" name="_edn1" title=""><span class="MsoEndnoteReference"><span lang="EN-GB"><!--[if !supportFootnotes]--><span class="MsoEndnoteReference"><span lang="EN-GB" style="font-family: "Calibri",sans-serif; font-size: 10.0pt; mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">[i]</span></span><!--[endif]--></span></span></a><span lang="EN-GB"> </span><span lang="EN-US">In many
cases, this cause-effect identification process is a matter of preparing the
strategy formulation phase which has the implicit message: “This is how we see
things” or a misused word in the business jargon: the often overstated “paradigms”.<o:p></o:p></span></div>
<div class="MsoEndnoteText">
<br /></div>
</div>
<div id="edn2" style="font-family: calibri, sans-serif; font-size: 13.3333px; font-style: italic;">
<div class="MsoNormal">
<a href="file:///D:/01_Data/3_Marketing/BookLaunch/Relevance%20Lostversie20121205.docx#_ednref2" name="_edn2" title=""><span class="MsoEndnoteReference"><span lang="EN-GB"><!--[if !supportFootnotes]--><span class="MsoEndnoteReference"><span lang="EN-GB" style="font-family: "Calibri",sans-serif; font-size: 11.0pt; mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">[ii]</span></span><!--[endif]--></span></span></a><span lang="EN-GB"> From: Mintzberg, Henry: “The Rise and Fall of Strategic Planning”
pp. 24-27, The Free Press 1994.<o:p></o:p></span></div>
<div class="MsoEndnoteText">
<br /></div>
<div class="MsoEndnoteText">
<br /></div>
</div>
</div>
</div>
</div>
<!--[if !supportLineBreakNewLine]-->
<!--[endif]--></span></td></tr>
</tbody></table>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-60342874904518441022019-03-01T10:28:00.000+01:002019-08-22T15:38:08.602+02:00About Ends and Means, or Beginning and Ends...<br />
<div class="MsoNormal">
<span lang="EN-GB"><i>It has been a while since I published
anything on this blog. But after having been confronted with organisations that
–from an analytics point of view- live in the pre-industrial era, I need to get
a few things off my chest.<o:p></o:p></i></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><i>In these organisations (and they aren’t the
smallest ones)<span style="mso-spacerun: yes;"> </span>ends and means are mixed
up, and ends are positioned as the beginning of Business Intelligence. Let me
explain the situation.</i><o:p></o:p></span><br />
<span lang="EN-GB"></span><br />
<h4>
<span lang="EN-GB">Ends are the beginning</span></h4>
</div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-rPpf6nBU-Z4/XVJxv6gkq_I/AAAAAAAAQJo/BFPSHYTQiHcDqRkoTzSRcV_DizUlxb2QACLcBGAs/s1600/Drijfijs.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="sea ice" border="0" data-original-height="587" data-original-width="881" height="265" src="https://1.bp.blogspot.com/-rPpf6nBU-Z4/XVJxv6gkq_I/AAAAAAAAQJo/BFPSHYTQiHcDqRkoTzSRcV_DizUlxb2QACLcBGAs/s400/Drijfijs.JPG" title="Drift ice" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><div class="MsoNormal">
<span lang="EN-GB">A metaphor for a critical look at reporting
requirements is like watching heavy drift ice </span></div>
<div class="MsoNormal">
<span lang="EN-GB">and wondering whether it’s coming
from a land based glacier or from an iceberg...<o:p></o:p></span></div>
</td></tr>
</tbody></table>
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB">Business users formulate their requirements
in terms of reports. That’s OK, as long as someone, an analyst, an architect or
even a data modeller understands this is not the end of the matter, on the
contrary.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Yet too many information silos have been
created when this rule is ignored. If an organisation considers report
requirements as the start of a BI project they are skipping at least the
following questions and the steps needed to produce a meaningful analytics
landscape that can stand the test of time:<o:p></o:p></span></div>
<div class="MsoNormal">
<br />
<ul>
<li><span lang="EN-GB">New information silos emerge with an
end-to-end infrastructure to answer a few specific business questions leaving
opportunities for a richer information centre unexplored. </span></li>
<li><span lang="EN-GB">The cost per report becomes prohibitive.
Unless you think € 60.000 to create one (1) report is a cinch…</span></li>
<li>Since the same data elements run the risk
of being used in various data base schemas, the extract and load processes pay
a daily price in terms of performance and processing cost.</li>
</ul>
</div>
<div class="MsoNormal">
<br /></div>
<h4>
<span lang="EN-GB">Ends and means are mixed up</span></h4>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">A report is the result of an analytical
process, combining data for activities like variance analysis, trend analysis,
optimisation exercises, etc.. As such it is a means to support decision making;
so rather than accepting the report requirements as such, some reverse engineering is
advised:<o:p></o:p></span><br />
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<i style="mso-bidi-font-style: normal;"><span lang="EN-GB">What
are the decisions to be made for the various managerial levels, based on these
report requirements?<o:p></o:p></span></i><br />
<i style="mso-bidi-font-style: normal;"><span lang="EN-GB"><br /></span></i></div>
<div class="MsoNormal">
<span lang="EN-GB">You<span style="mso-spacerun: yes;">
</span>may wonder why this obvious question needs to be asked but be advised,
some reports are the equivalent of a news report. The requestor might just want
to know about what happens without ever drawing any conclusions let alone linking
any consequences to the data presented.<o:p></o:p></span><br />
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<i style="mso-bidi-font-style: normal;"><span lang="EN-GB">What
are the control points needed by the controller to verify aspects of the
operations and their link to financial results?<o:p></o:p></span></i><br />
<i style="mso-bidi-font-style: normal;"><span lang="EN-GB"><br /></span></i></div>
<div class="MsoNormal">
<span lang="EN-GB">Asking this question almost always leads to
extending the scope of the requirements. Controllers like to match data from
various sources to make sure the financial reports reflect the actual
situation. <o:p></o:p></span><br />
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<div class="MsoNormal">
<i><span lang="EN-GB">What
are the future options, potential requirements and / or possibilities of the
required enhanced with the available data in the sources?<o:p></o:p></span></i></div>
<div class="MsoNormal">
<i><span lang="EN-GB"><br /></span></i></div>
<div class="MsoNormal">
<span lang="EN-GB">This exercise is needed to discover
analytical opportunities which may not be taken at the moment for a number of
reasons like: insufficient historical data, lacking analytical skills to come
up with meaningful results… But that must not stop the design from taking the
data in scope from the start. Adding the data in a later stage will come at a
far greater cost than the cost of the scope extension.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<i><span lang="EN-GB">What
is the basic information infrastructure to facilitate the above? I.e. what is
the target model?<o:p></o:p></span></i></div>
<div class="MsoNormal">
<i><span lang="EN-GB"><br /></span></i></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-lw6qIjR2Lb8/XVJ0OONveOI/AAAAAAAAQJ8/PEM4m1sZNngmpa4lds47y_wViRPIfEOXgCLcBGAs/s1600/STAR%2BDataMart.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="626" data-original-width="666" height="600" src="https://1.bp.blogspot.com/-lw6qIjR2Lb8/XVJ0OONveOI/AAAAAAAAQJ8/PEM4m1sZNngmpa4lds47y_wViRPIfEOXgCLcBGAs/s640/STAR%2BDataMart.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A Star schema is the ideal communication platform between business and tech people.</td></tr>
</tbody></table>
<div class="MsoNormal">
Whatever modelling language you use, whatever
technology you use (virtualisation, in memory analytics, appliances, etc…) in
the end the front end tool will build a star schema. So take the time to build
a logical data star schema model that
can be understood by both technical people and business managers.</div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<i><span lang="EN-GB">What
is the latency and the history needed per decision making horizon?<o:p></o:p></span></i></div>
<div class="MsoNormal">
<i><span lang="EN-GB"><br /></span></i></div>
<div class="MsoNormal">
<span lang="EN-GB">The latency question deals with a multitude
of aspects and can take you to places you weren’t expecting when you were briefed
about report requirements. As a project manager I’d advise you to handle with
care as the scope may become unmanageable. Stuff like (near) real-time
analytics, in database analytics, triple store extensions to the data
warehouse, complex event processing mixing textual information with numerical measures…
But as an analyst I’d advise you to be aware of the potentially new horizons to
explore. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">The history question is more
straightforward and deals with the scope of the initial load. The slower the
business cycle, the more history you need to load to come up with useful data
sets for time series analysis.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<i><span lang="EN-GB">What
data do we present via which interface to support these various decision types?<o:p></o:p></span></i></div>
<div class="MsoNormal">
<i><span lang="EN-GB"><br /></span></i></div>
<div class="MsoNormal">
<span lang="EN-GB">This question begs a separate article but
for now, a few examples should make things clear.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Static reports for external stakeholders
who require information for legal purposes,<o:p></o:p></span></div>
<div class="MsoNormal">
</div>
<ul>
<li><span lang="EN-GB">Reports using prompts and filters for team
leaders who need to explore the data within predetermined boundaries,</span></li>
<li><span lang="EN-GB">OLAP cubes for managers who want to explore
the data in detail and get new insights,</span></li>
<li><span lang="EN-GB">A dashboard for C- level executives who
want the right cockpit information to run the business,</span></li>
<li><span lang="EN-GB">Data exploration results from data mining
efforts to produce valid, new and potentially useful insights in running the
business.</span></li>
</ul>
<br />
<div class="MsoNormal">
<span lang="EN-GB">If all these questions are answered
adequately, we can start the data requirements collection as well as the source
to target mappings. <o:p></o:p></span></div>
<br />
<div class="MsoNormal">
<br /></div>
</div>
<div class="MsoNormal">
<br /></div>
<h4>
<span lang="EN-GB">Three causes, hard to eradicate</span></h4>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">If your organisation shows one or more of
these three causes, you have a massive change management challenge ahead that
will take more than a few project initiation documents to remedy. If you don’t
get full support from top management, you’d better choose between accepting this
situation and become an Analytics Sisyphus or look for another job.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><b>Project based funding</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Government agencies may use the excuse that
there is no other way but moving from tender to tender, the French proverb “les
excuses sont faites pour s’en servir” [1] applies. A solid data and information
architecture, linked to the required capabilities and serving the strategic
objectives of a government agency can provide direction to these various
projects.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">A top performing European retailer had a
data warehouse with 1.500 tables, of which eight (8!) different time
dimensions. The reason? Simple: every BU manager had sovereign rule over his
information budget and “did it his way” to quote Frank Sinatra. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><b>Hierarchical organisations</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">I already mentioned the study of Prof. Karin
Moser introducing three preconditions for knowledge co-operation: reciprocity,
a long term perspective for the employees and the organisation and breaking the
hierarchical barriers. [2]<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">On the same pages I quote the authors Leliveld
& Vink and Davos & Newstrom who support the idea that knowledge
exchange based on reciprocity can only take place in organisational forms that
present the whole picture to their employees and that keep the distance between
co-workers and the company’s vision, objectives, customers etc. as small as
possible.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Hierarchical organisations are more about
power plays and job protection than knowledge sharing so the idea of having one
shared data platform for everyone in the organisation to extract his own analyses
and insights is an absolute horror scenario.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><b>Process based support</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Less visible but just as impactful, if IT
systems are designed primarily for process support instead of attending as well
to the other side of the coin, i.e. decision support, then you have a serious
structural problem. Unlocking value from the data may be a lengthy and costly
process. Maybe you will find some inspiration in a previous article on this
blog: Design from the Data. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><a href="https://ba4bi.blogspot.com/2015/07/the-future-of-information-systems.html">https://ba4bi.blogspot.com/2015/07/the-future-of-information-systems.html</a>
<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">In short: processes are variable and need
to be flexible, what lasts is the data. Information objects like a customer, an
invoice, an order, a shipment, a region etc… are far more persistent than the
processes that create or consume instances of these objects.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><o:p> [</o:p></span><span lang="EN-GB" style="text-indent: -18pt;">1]<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span><span lang="EN-GB" style="text-indent: -18pt;">Excuses are made to be used</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="text-indent: -18pt;"> [2]<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span><span lang="EN-GB" style="text-indent: -18pt;">Business Analysis for Business Intelligence pp. 35 -38 CRC Books, a Taylor & Francis Company October 2012</span></div>
<br />
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<br /></div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com4tag:blogger.com,1999:blog-4520438534640699001.post-4841893805808821872018-12-31T08:59:00.002+01:002018-12-31T08:59:30.656+01:00<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-uxgNo_Clzec/XCnMO3iwvhI/AAAAAAAALso/MAq670Qlo9EsTsrGVTHHo168UBtxOx0_ACLcBGAs/s1600/NewYear2019Personal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="755" data-original-width="1132" height="426" src="https://2.bp.blogspot.com/-uxgNo_Clzec/XCnMO3iwvhI/AAAAAAAALso/MAq670Qlo9EsTsrGVTHHo168UBtxOx0_ACLcBGAs/s640/NewYear2019Personal.png" width="640" /></a></div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-80926007045904446182018-12-29T18:12:00.001+01:002018-12-29T18:12:54.657+01:00Roadmap to a successful data lake<br />
<div class="MsoNormal">
<span lang="EN-GB"><i>A few years ago, a couple of eCommerce
organisations asked my opinion on the viability of a data lake in their
enterprise architecture for analytical purposes. After careful study the result
was 50 – 50: one organisation had no immediate advantage investing in a data
lake. It would become just another data silo or even a data junk yard with hard
to exploit data and no idea of the added value this would bring.<o:p></o:p></i></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><i>The other -€ 1 bn plus company- had all the
reasons in the world to start exploring the possibilities of a repository for
semi-structured and unstructured data. But it would take them at least two
years to set up a profitable infrastructure. Technology was not the problem:
low cost processing and storage as well as the software -mainly open source-
was no problem. They even had no problem attracting the right technical
profiles as their job offers topped everyone in the market. No, the real
problem was integrating and exploiting the new data streams in a sensible and
managed way. As I am about to embark on a new mission to rethink an analytical
infrastructure with the data lake in scope, I can share a few lessons from the
past and think ahead for what’s coming. </i><o:p></o:p></span></div>
<br /><br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-fTn4Oxm9wiw/XCeoQXwv9NI/AAAAAAAALok/k6CXzazbcVMvdCwicsXuypoNPtAmK9LgQCLcBGAs/s1600/DataLake.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1115" height="400" src="https://3.bp.blogspot.com/-fTn4Oxm9wiw/XCeoQXwv9NI/AAAAAAAALok/k6CXzazbcVMvdCwicsXuypoNPtAmK9LgQCLcBGAs/s400/DataLake.png" width="278" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Start from the data and work your way up to the business case<br /><br /></td></tr>
</tbody></table>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-size: large;">Analyse the Velocity, Variability and
Volume of the data to meet the analytical requirements</span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><b><span style="font-size: large;"><br /></span></b></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Is it stable and predictable? Then it’s
probably an indication that your organisation is not yet ready for this
investment. But if there is a rapid growth rate in at least one of these three
Vs, you better get planning and designing your data lake.<o:p></o:p></span></div>
<h4>
</h4>
<h4>
<span lang="EN-GB" style="font-weight: normal;">Planning: </span></h4>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
</div>
<ul>
<li><span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><span lang="EN-GB" style="font-weight: normal; text-indent: -18pt;">What time do we need to close
the skills gap and manage a Hadoop environment professionally?</span></li>
<li><span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><span lang="EN-GB" style="font-weight: normal; text-indent: -18pt;">What is a realistic timeframe
to connect, understand and manage the new semi-structured and unstructured data
sources?</span></li>
</ul>
<br />
<h4>
<span lang="EN-GB" style="font-weight: normal;">Designing: </span></h4>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
</div>
<ul>
<li><span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><span lang="EN-GB" style="font-weight: normal; text-indent: -18pt;">Do we put every piece of data
in the lake and write off our investments in the classical BI infrastructure or
do we choose a hybrid approach where only new data types will be filling the
lake?</span></li>
</ul>
<!--[if !supportLists]--><br />
<div class="MsoListParagraphCxSpMiddle" style="margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span lang="EN-GB" style="font-family: "Courier New"; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-GB">In case of a hybrid approach,
do we need to join between the two data sources?<o:p></o:p></span></span></div>
<div class="MsoListParagraphCxSpMiddle" style="margin-left: 72.0pt; mso-add-space: auto; mso-list: l0 level2 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span lang="EN-GB" style="font-family: "Courier New"; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-GB">In case of a total replacement
of the data warehouse, do we have the proper front end tools to make the business users
exploit the data or do they have to rely on data scientists and data engineers, potentially creating a bottleneck in the process?<o:p></o:p></span></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
</div>
<ul>
<li><span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><span lang="EN-GB" style="font-weight: normal; text-indent: -18pt;">How will we process the data?
Do we simply dump it and leave it all to the data scientists to make sense of
it or do we plan ahead on some form of modelling on the Hadoop platform,
creating column families which are flexible enough to cope with new attributes
and which will make broader access possible?</span></li>
<li><span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><span lang="EN-GB" style="font-weight: normal; text-indent: -18pt;">Do we have a metadata strategy
that can handle the growth, especially from a user-oriented perspective?</span></li>
<li><span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><span lang="EN-GB" style="font-weight: normal; text-indent: -18pt;">Security and governance are far
more complex in a data lake than in a data warehouse. What’s our take on this
issue?</span></li>
</ul>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;"><span style="font-size: large;">Check the evolution of your business
requirements</span><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;">It’s no use to invest in a data lake when
the business ambitions are on a basic level and stuff like a balanced scorecard
is just popping up in the PowerPoints from the CEO.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;">Some requirements are very clear on their
data needs, but others aren’t. It may take a considerable amount of analysis to
surface the data requirements for semi-structured and unstructured data. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;">And with legislation like the GDPR, some
data may be valuable but also very hard to get as the consumer is more and more
aware of his position in the data game. That’s why very fine-grained opt-ins
are adding complexity to customer data management.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;"><span style="font-size: large;">Develop a few winning use cases</span><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;">“A leader is someone who has followers” is
quite applicable in this situation. You are after all challenging the status
quo and if there’s one thing I’ve learned in 30 years in analytics and ICT in
general: a craftsman is very loyal to his tools. Managing change in the
technical department will not be a walk in the park. It may require adding an
entire new team to the department or at least have some temporary professionals
come in to do the dirtiest part of the job and hand over the Hadoop cluster in
maintenance mode to the team.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;">To enable all this, you need a few winning
use cases that appeal to the thought leaders in the organisation. Make sure you
pick sponsors with clout and the budget to turn PowerPoints into working
solutions.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-weight: normal;">There certainly will be use cases for
marketing, finance and operations. Look for the maximum leverage and get
funded. And by the way, don’t bother the HR department unless you are working
for the armed forces. They always come last in commercial organisations…<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-46585070840534145412018-04-19T21:03:00.002+02:002018-04-21T12:25:27.777+02:00How to make progress in a political organisation<h3>
Why Business Analysis and politics don’t mix.</h3>
<br />
<i>After thirty years of practice in all sorts and flavours of organisations there’s one that stands out as a tough conundrum for any business analyst and by extension enterprise architect as well as project managers. It’s the political organisation, so eloquently described by Henry Mintzberg. </i><br />
<i>The problem with these organisations for a business analyst, project manager or enterprise architect is identical: setting priorities to determine the first iteration of the development cycle. This lack of priority ranking may lead to scope creep, projects that never deliver the product or a user community that is not on board, etc…</i><br />
<br />
<br />
<h4>
Forces in a political organisation</h4>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://4.bp.blogspot.com/-Ur2VuzBg80Q/WtjkrGo3qRI/AAAAAAAAIJE/lEZ_8SPLnRYzGQ5RZpkNSRX9aIo-IOX-gCLcBGAs/s1600/DecisonMakingContext.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="608" data-original-width="1051" height="231" src="https://4.bp.blogspot.com/-Ur2VuzBg80Q/WtjkrGo3qRI/AAAAAAAAIJE/lEZ_8SPLnRYzGQ5RZpkNSRX9aIo-IOX-gCLcBGAs/s400/DecisonMakingContext.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Wouldn't we all like to work in Tom Davenports Analytical Organisation?</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
In the paragraph “Decisions, Teams, and Groups at Work, Classification of Decision-Making Environments, I use a simple matrix to describe decision-making contexts for BI projects. But, believe me, you can use it for any project type.<br />
<br />
You don’t need much time to determine if you’re in a political organisation. Look for committees that make the ultimate decisions, look for a lack of accountable individuals, slow decision making processes and a track record of projects that failed to deliver the intended product. Of course government bodies are by definition political but you will also find them in the private sector.<br />
<br />
<h4>
How to recognise a political organisation before you’re even at the reception desk?</h4>
Maybe this table can help:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-9u_GdSCqkoI/WtjlLktMcMI/AAAAAAAAIJI/n--Bb8x5gnwOERIFJ71djHzAQSoKwWyzQCLcBGAs/s1600/Tabelherkennen.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="212" data-original-width="1104" height="122" src="https://1.bp.blogspot.com/-9u_GdSCqkoI/WtjlLktMcMI/AAAAAAAAIJI/n--Bb8x5gnwOERIFJ71djHzAQSoKwWyzQCLcBGAs/s640/Tabelherkennen.JPG" width="640" /></a></div>
<br />
<br />
Political organisations, by definition, don’t have shared goals. Each alderman, state secretary, each manager, wants to score his goals without letting the team take any credit for it. Because re-election or promotion matter… And political organisations always differ on the cause and effect chains which shows clearly in analytical projects.<br />
<br />
<h4>
Setting priorities in a political organisation</h4>
<br />
You can imagine that this is the toughest conundrum to solve; if you can’t prioritise “because everything is important” you can’t even start an analysis track. Unless you simply want to sell billable hours… And prepare for a debriefing and passing the buck, dodging any responsibility.<br />
<i>But if you’re a hired gun that may be exactly why you’ve been hired: to take the blame for the organisation’s ineptness to take responsibility and make choices even if they go against some members of the team.</i> (I use “team” for want of a better word in a political organisation)<br />
In this post, I am giving you a few tips and tricks to force the “team” to come up with priorities.<br />
<br />
But first some context. The organisation is looking for a new way to analyse structured and unstructured data; Therefore it needs a modern data architecture. Your job as business analyst (and by extension project manager and enterprise architect) is to know what the strategic priorities of the organisation are. This needs to match with the available data and information needs. You need to check the feasibility and then choose the first iteration to deliver analytical results. A best practice is to check the organisation’s strategy, its initiatives to improve the organisation’s position in case of a commercial entity or the level of societal utility in case of a governmental or non for profit organisation.<br />
Imagine the first intake with the project sponsor, the product owner and any other stakeholder who has been identified in the project structure.<br />
<br />
Here’s the dialogue:<br />
<br />
<b>Business Analyst:</b> At the kick off of this analysis track, I’d like to determine with you the first iteration: where we start analysing, designing and building the first deliverables.<br />
<b>The “team”:</b> (silence)<br />
<b>Business Analyst:</b> Do you have a project portfolio and do you use program management to prioritise the management actions? Do you have mission and vision statement for this project?<br />
<b>The “team”: </b>We thought you could formulate the vision and the mission for the project. And no we don’t have a project portfolio. We do have an Excel sheet with a list of all the projects and their status.<br />
<b>Business Analyst:</b> Could we infer from the status what the priorities are?<br />
<b>The “team”: </b>No.<br />
<b>Business Analyst:</b> What if we look at the budget per management project. Maybe the size says something about the priority? Or what if look at rejected project proposals and the reasons? Maybe that says something about the criteria.<br />
<b>The “team”: </b>Not necessarily. First of all, all management project requests are answered positively and funds are allocated to these projects. Some projects may have big budgets but that doesn’t indicate anything about their importance.<br />
<b>Business Analyst:</b> What about the number of full time equivalents allocated to each project?<br />
<b>The “team”:</b> A high number may indicate something about the complexity or the scope but that doesn’t tell you what priority the project has.<br />
<b>Business Analyst:</b> I think this one may help us out: have you indicated the origins of leakages and losses in your business processes and could those numbers give us a hint of what’s important to the management team?<br />
<b>The “team”:</b> Leaks and losses are handled by the management team and as such are equally important.<br />
<b>Business Analyst: </b>Does the amount of data, the connection with business processes and the variety in the data give us a clue where we should start the project?<br />
<b>The “team”:</b> That’s we are hiring you as a Business Analyst.<br />
<br />
Now it gets tricky and you make the call, as The Clash sing: “Should I stay or should I go”<br />
<br />
Here are few of the killer questions and remarks that will lead you to the exit:<br />
<br />
<ul>
<li>What projects will get or got the most press coverage?</li>
<li>What if you had to choose, right now?</li>
<li>Do you expect me to deliver a successful end result if you don’t know what you want?</li>
</ul>
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<img border="0" data-original-height="340" data-original-width="215" height="320" src="https://1.bp.blogspot.com/-vEo36skv9Rk/UZ3eAhEypzI/AAAAAAAAAMs/MOWXDSPfR54pDd1Vlqr34TQ1BJXl951DgCPcBGAYYCw/s320/Book.png" width="202" /> </div>
<br />
<a href="http://bit.ly/BA4BI" target="_blank">More on decision making contexts in the book “Business Analysis for Business Intelligence” p. 203 – 213 </a><br />
<br />
<br />
<h4>
Is there way out? Maybe.</h4>
<br />
<br />
The only escape route I can think of is to start with a stakeholder analysis. Try defining the primary stakeholders and map them on a RACI matrix. If that works, you can develop your first iteration with some confidence, knowing that danger is always on the road ahead..<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://2.bp.blogspot.com/-rbO9dnp_qpI/WtjmN0Od-fI/AAAAAAAAIJY/yyAA1iDMk8g22LzsE-ykyD_S1gS1Nz_fwCLcBGAs/s1600/GridRACI.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="262" data-original-width="586" height="286" src="https://2.bp.blogspot.com/-rbO9dnp_qpI/WtjmN0Od-fI/AAAAAAAAIJY/yyAA1iDMk8g22LzsE-ykyD_S1gS1Nz_fwCLcBGAs/s640/GridRACI.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Example of a stakeholder analysis that turns out well: the CEO’s desk is where the buck stops.</td></tr>
</tbody></table>
<br />
If a stakeholder analysis is inconclusive, there must be someone who’s not involved in the official decision making unit (DMU) who is the primary influencer. Now you’ll have to get out of your comfort zone as an analyst and start thinking like an account manager.<br />
<br />
I was lucky to have training in the Miller Heiman Strategic Selling method as well as the Holden Power Base Selling method. It sharpened my skills for identifying and influencing these hidden decision makers. So here’s my advice: check out these two books. They will increase your efficiency in political organisations with an order of magnitude.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-TS-9OWSZo1I/WtjmaapxubI/AAAAAAAAIJc/hmqic-n-XcUHrtazRIrNf5IHqkZDQgYzACLcBGAs/s1600/boeken.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Target account selling; Fox hunting" border="0" data-original-height="535" data-original-width="865" height="246" src="https://1.bp.blogspot.com/-TS-9OWSZo1I/WtjmaapxubI/AAAAAAAAIJc/hmqic-n-XcUHrtazRIrNf5IHqkZDQgYzACLcBGAs/s400/boeken.jpg" title="Miller Heiman Strategic Selling; Jim Holden Power Base Selling " width="400" /></a></div>
<br />
The new strategic selling is an update of the original, worth reading for any novice in business analysis and project management.<br />
This is Jim Holden’s original book. Of course, as things go in this business, there were many to follow up on his success. Start here anyway.<br />
<div>
<br /></div>
Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-29560231480304201482017-12-31T15:10:00.001+01:002017-12-31T15:12:09.361+01:00<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Xm5ALJfpDIc/WkjvvXJH1jI/AAAAAAAAHdY/v1iPQm2DYTEGG_V9X0QPesN5xoxPgJjcACLcBGAs/s1600/new-year-2018_27099893.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="950" data-original-width="1436" height="422" src="https://2.bp.blogspot.com/-Xm5ALJfpDIc/WkjvvXJH1jI/AAAAAAAAHdY/v1iPQm2DYTEGG_V9X0QPesN5xoxPgJjcACLcBGAs/s640/new-year-2018_27099893.png" width="640" /></a></div>
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0tag:blogger.com,1999:blog-4520438534640699001.post-35595525377806157452017-12-24T18:09:00.001+01:002019-10-15T16:10:38.215+02:00Getting practical: How Analytics Can Drive the Information Architecture Development<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Does the theory presented in the previous article work in practice? That is the theme of this post where I present an (anonymous) case from a project I did for a customer.</div>
<div class="separator" style="clear: both;">
But before I proceed, a quick reminder from my book “Business Analysis for Business Intelligence”.</div>
<div class="separator" style="clear: both;">
What every organisation needs to know boils down to four C’s. It is information about the customer, the cost, the competition and the competences of the organisation, the latter also represented by a higher level of abstraction: the capabilities.</div>
<div class="separator" style="clear: both;">
The illustration below shows how these four C’s are the foundation of a balanced scorecard. But a balanced scorecard measures only the intended –or planned- strategy, not the emergent strategies. Therefore, this 4 C framework has a much broader scope and includes decision support for emergent strategies. </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-HeN5og4DzaA/Wjjk-hv1b9I/AAAAAAAAHZE/YVHZI7Fdi4s1p0rWj4SdTU2UvYPLGLyWgCLcBGAs/s1600/Fig1%2B4C.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="308" data-original-width="431" src="https://1.bp.blogspot.com/-HeN5og4DzaA/Wjjk-hv1b9I/AAAAAAAAHZE/YVHZI7Fdi4s1p0rWj4SdTU2UvYPLGLyWgCLcBGAs/s1600/Fig1%2B4C.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
To develop a shared knowledge of the customer, this organisation needed to embed a business rule in the data namely that contacts are associated with an account. This, because the organisation is an exclusive business-to-business marketing machine selling to large corporations. A contact without this association was registered and kept in a staging area, waiting to be associated with an account. In other words: only contacts related to an organisation were of use to the business. At least, in the present context. </div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-bq-624nzCcc/Wjjk_dFof5I/AAAAAAAAHZU/2cy4uQVQ56IjKzvgQjl4KxfF7sOu-B5ggCLcBGAs/s1600/fig2%2BCustAcct.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="459" data-original-width="508" src="https://2.bp.blogspot.com/-bq-624nzCcc/Wjjk_dFof5I/AAAAAAAAHZU/2cy4uQVQ56IjKzvgQjl4KxfF7sOu-B5ggCLcBGAs/s1600/fig2%2BCustAcct.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB">Today, this rule is cast in stone in a
monolithic CRM application but the CIO wishes to migrate to a service factory
in the near future. This way, when the business rule would change or when the
company would move to a B2C market, the CRM processes would be easier to adapt
to the new situation. A transition plan for all customer data needs to be
developed.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Lingua Franca used the following phased
approach:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
</div>
<ol>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Mapping the customer data in a
data portfolio</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Study the ASIS</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Link capabilities to analytics</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Map the capabilities on the
data portfolio</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Define the information
landscape</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Make the mapping analytics –
transactional data</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Define the services</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><span lang="EN-GB" style="text-indent: -18pt;">Decide on the physical
architecture</span></li>
</ol>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/--yWeFSNUlPc/Wjjk-scdpkI/AAAAAAAAHZI/iM4i3DravpI2ney-Hc17681GkKgdkwjewCLcBGAs/s1600/Fig3%2Bmatrix.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="305" data-original-width="557" src="https://1.bp.blogspot.com/--yWeFSNUlPc/Wjjk-scdpkI/AAAAAAAAHZI/iM4i3DravpI2ney-Hc17681GkKgdkwjewCLcBGAs/s1600/Fig3%2Bmatrix.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h4 style="clear: both;">
<br />Mapping the customer data in a data portfolio </h4>
<div class="separator" style="clear: both;">
A lot of customer data is of strategic value and a lot isn’t. That led us to use a modified version of McFarlan’s portfolio approach to information systems which can just as well be applied to data.</div>
<div class="separator" style="clear: both;">
Variant on: McFarlan, F. W. (1981). "Portfolio approach to information systems.</div>
<div class="separator" style="clear: both;">
"Harvard Business Review (September–October 1981): 142-150</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
The analytics version of this schema translates the four quadrants into workable definitions:</div>
<div class="separator" style="clear: both;">
Strategic Data: critical to future strategy development: both forming and executing strategy are supported by the data, as well as emergent strategies where data might be captured outside the exiting process support systems. The reason is clear: process support or transaction support systems are designed and tuned for the intended strategy. </div>
<div class="separator" style="clear: both;">
Turnaround Data: critical to future business success as today’s operations are not supported, new operations will be needed to execute. These data are often not even in scope of the emergent strategy processes. They may be hidden in a competitor’s research, in technological breakthroughs, in new government regulations or in consumer outcries against abuse to name a few sources.</div>
<div class="separator" style="clear: both;">
Support Data: Valuable but not critical to success</div>
<div class="separator" style="clear: both;">
Factory Data: critical to existing business operations: the classical reports, dashboards and scorecards</div>
<div class="separator" style="clear: both;">
In this case, the association between account and contact was considered factory data as it describes the way the company is doing business today</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
As the illustration below in the Archimate model shows, there is a cascading flow of business drivers and stakeholders that influence the business goals which in their turn impact the requirements that are realised by business processes. These are supported by legacy systems and new software packages or bespoke applications. The result of this approach is a dispersed view on the data that are used and produced in these applications. What if not processes but data would be at the base of the requirements? Would this change the organisation’s agility? Would it enhance responsiveness to external influence? That was the exercise we were preparing for. </div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-WVoOeNoCz9M/Wjjk_vNB0RI/AAAAAAAAHZY/61p2gLegCQMErRMSR4MpLPHy8UMXYfo-gCLcBGAs/s1600/fig4.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="Generic enterprise architecture" border="0" data-original-height="311" data-original-width="546" src="https://3.bp.blogspot.com/-WVoOeNoCz9M/Wjjk_vNB0RI/AAAAAAAAHZY/61p2gLegCQMErRMSR4MpLPHy8UMXYfo-gCLcBGAs/s1600/fig4.png" title="Generic Archimate model" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Data dispersion in a classical IT landscape</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
.</div>
<div>
<br /></div>
<h4 style="clear: both;">
<br />Study the ASIS</h4>
<div class="separator" style="clear: both;">
Today, the business process of account and contact registrations is as follows:</div>
<div>
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-7q4PMegYGrE/Wjjk_7CrC1I/AAAAAAAAHZc/oNaTLYYnRC8e5QEygCPrT54GpkCdG3OgACLcBGAs/s1600/fig5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="578" data-original-width="605" src="https://1.bp.blogspot.com/-7q4PMegYGrE/Wjjk_7CrC1I/AAAAAAAAHZc/oNaTLYYnRC8e5QEygCPrT54GpkCdG3OgACLcBGAs/s1600/fig5.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The present CRM monolith supports this process but future developments like the takeover of a more consumer oriented business may change the business model and the business process drastically. Thus, the self-service registration process should make the link between contact and account optional and the validation process should only deal with harmonising data to make sure the geographical information is correct and contact data are uniform as far as (internal) naming conventions and (external) reference data are concerned. It is already a great step forward that the company uses a master data management system to separate data management from process management. This enables a smoother transition to the new information architecture development method. </div>
<div class="separator" style="clear: both;">
<br /></div>
<h4 style="clear: both;">
Link capabilities to analytics</h4>
<div>
<div>
Therefore an extensive inventory of all potentially needed business capabilities is undertaken and linked to the relevant business questions supporting these capabilities.</div>
<div>
In this example we present a few of these present and future business questions:</div>
<div>
What is the proportion of contacts from our B2B customers that may be interested in our consumer business?</div>
<div>
Which accounts may experience a potential threat from our new consumer business unit?</div>
<div>
Which contacts from the B2C may become interested in our B2B offerings?</div>
<div>
Which products from the B2C unit may prove sellable via the B2B channels?</div>
<div>
By listing all the relevant present and future business questions, it becomes clear that the account validation process as it is defined today may need to change and what is considered factory data today may get an “upgrade” to strategic and turnaround data to deal with the challenges.</div>
</div>
<div>
<br /></div>
<h4 style="clear: both;">
<div class="separator" style="clear: both; font-weight: 400;">
<br /></div>
Map the capabilities on the data portfolio</h4>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-60AEkYcu2kM/WjjlASQ46TI/AAAAAAAAHZg/-0OSWnKIRwE7HPQTu_VHxu7Xg1hwvtzNgCLcBGAs/s1600/fig6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="346" data-original-width="460" src="https://2.bp.blogspot.com/-60AEkYcu2kM/WjjlASQ46TI/AAAAAAAAHZg/-0OSWnKIRwE7HPQTu_VHxu7Xg1hwvtzNgCLcBGAs/s1600/fig6.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="clear: both;">
<span style="font-weight: normal;">In this diagram, the entire data landscape of the account – contact association is charted and managed via five methods. </span></div>
<span style="font-weight: normal;">Operational business metadata describe the context in which data is created, updated and deleted as well as the context in which it is used. A minimum deliverable is instructions and training for the people who perform the CRUD operations.</span><span style="font-weight: normal;">Process metadata relates the business process (present and future) to the business context to provide the process stakeholders with information and motivation: the what, why, when and who of the process and the data captured.</span><span style="font-weight: normal;">Business Intelligence metadata describes the decision support possibilities in the present and future clients: dashboards, reports, cubes, data sets for further examination,…</span><span style="font-weight: normal;">Process alignment: it describes what is often a mutual adjustment between a monolithic application and the business process it supports. Some market leaders in OLTP software present their process flows as best practice. As if all businesses should converge in their way of doing business…</span><span style="font-weight: normal;">ETL Architecture documents the lineage from source to target, the transformations, quality measures, as well as the technical aspects of the process i.e. parallel or sequential loading, dropping of indexes and rebuilding them, hashing, etc… </span><br />
<br />
<h4 style="clear: both;">
<span style="font-weight: normal;">Define the information landscape</span></h4>
<div style="clear: both;">
<span style="font-weight: normal;">Even in this simple customer – account relationship some thinking needs to be done about a holistic view on the essential elements defining the relationship. By “essential” I mean the minimum attributes and levels of detail that need to be shared outside the context of CRM to be used in other business functions like HR, operations, finance,…</span></div>
<div style="clear: both;">
<span style="font-weight: normal;"><i>Here are a few of the considerations to be made:</i></span></div>
<div style="clear: both;">
<span style="font-weight: normal;">How long is a customer considered as such? If the average buying frequency of your product is twice a year, for how many years do you keep the relationship active if for three years no order has come in? </span><span style="font-weight: normal;">How do we compare the account performance in case of mergers? Does an account always need a DUNS number? Or a VAT registration? What about informal groups regularly doing group purchases? </span><span style="font-weight: normal;">Discussing these and many other issues lays the foundation for a data governance process. </span></div>
<h4 style="clear: both;">
<span style="font-weight: normal;"><br /></span></h4>
<h4 style="clear: both;">
<span style="font-weight: normal;">Make the mapping analytics – transactional data</span></h4>
<div style="clear: both;">
<span style="font-weight: normal;">This phase is crucial for the quality of your decision support system and is very much like the business analysis process for analytics. Start with high level concepts and descend to the lowest grain of attributes and transaction records as well as external sources like social media, open data and market research data.</span><span style="font-weight: normal;">For instance: “customer loyalty” is expressed as “a constantly high share of wallet over an average historic period of time of three years and a projected future loyalty period of another three years”.</span></div>
<div style="clear: both;">
Can you imagine the data needed to make this definition work? </div>
<div style="clear: both;">
<span style="font-weight: normal;">The exercise at this customer’s site produced 87 different data types coming from the ERP and CRM systems as well as external data like Net Promotor Scores, contact centre chat data, e-mails and response to LinkedIn posts. It sparked new ways of customer interaction procedures: new sales and order processing methods as well as new aftersales initiatives, the organisation would never have come up with if it hadn’t done this exercise.</span></div>
<div style="clear: both;">
<span style="font-weight: normal;"><br /></span></div>
<h4 style="clear: both;">
Define the services</h4>
<div>
<br /></div>
<div style="clear: both;">
<span style="font-weight: normal;">To move from the monolithically based approach to a more micro service oriented architecture, we needed to decompose the monolith into distinctive reusable services and data components. This approach forces a strict quality management for the data in scope as errors or poor quality will reflect on an enterprise scale. On the other hand, this “do it right the first time” principle avoids replication of work and improves the quality of decision making drastically.</span><span style="font-weight: normal;">The schema below needs some explanation. The intake service triggers the validation service which checks the contact and account data with reference data, Chamber of Commerce data and, when finished, triggers the registration service which in its turn triggers the master data update service. MDM contact is now a superclass of this contact and will be used enterprise wide. Four services now ensure reusability for not just the CRM application but for all other use cases in the organisation. And the data quality improves drastically as the “do-it-right-the-first-time” principle is easier to fund for enterprise wide data. </span></div>
<div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://2.bp.blogspot.com/-sJExSv7Ibv8/Wjjk-qQeZoI/AAAAAAAAHZM/W2pX5laIpvckhPnHvEQRr9mrwWlnDg5fgCLcBGAs/s1600/Fig7.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="240" data-original-width="462" height="331" src="https://2.bp.blogspot.com/-sJExSv7Ibv8/Wjjk-qQeZoI/AAAAAAAAHZM/W2pX5laIpvckhPnHvEQRr9mrwWlnDg5fgCLcBGAs/s640/Fig7.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Data landscape for a CRM and customer MDM application</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h4 style="clear: both;">
Decide on the physical architecture</h4>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
The classical approach using at least two environments is becoming obsolete for organisations that want to stay ahead of the competition. The separation between transaction processing and analytical processing will go out the window in the next few years. Not only because of the costly maintenance of Extract Transform and Load (ETL) processes between the transaction systems and the data warehouse but first and foremost because of the lack of integration with unstructured data that are in Hadoop Distributed File Sets (HDFS) or streaming data that are caught in Resilient Distributed Datasets (RDD)</div>
<div class="separator" style="clear: both;">
The organisation needs a significant leap forward and is now examining the Vector in Hadoop solution, a database that combines the classic SQL environment with NoSQL. The reasons are supported by objective facts: a rapidly scalable full ACID SQL database based on HDFS. It supports modify, insert and updates using a patented technology developed at the University of Amsterdam: the Positional Delta Trees (PDT). <a href="https://www.researchgate.net/publication/228611733_Positional_Delta_Trees_to_reconcile_updates_with_read-optimized_data_storage" rel="nofollow" target="_blank">More on this in their paper which is published here</a>. The short version of PDT: a separation between the write and read store where updates are merged into the write store at run time using the row index for a correct positioning of the modify/insert or update. The result? Online updates without impacting the read performance. Since the database can also access Spark’s parallel processing capability combining Spark RDD architecture accessed from the SQL perspective so that queries that were previously impossible to consider, this system combines the very best of three worlds: ACID based transaction support, complex event processing and HDFS support for unstructured data analytics with a flexible approach to changing data influx –provided you do your homework and define the column families in the broadest possible sense to fit your analytical needs. </div>
<div class="separator" style="clear: both;">
Data loading – if that is the purpose - can be achieved at a rate of around 4TB per hour comprising four billion ‘120 column’ tuples per hour on a 10 node Hadoop cluster – or around 500 billion columns per hour in total! (many caveats apply but it is still a remarkable performance.</div>
<div class="separator" style="clear: both;">
The advantage of this architecture will be exploited to the maximum if the data architecture is connecting transaction data, which are by definition microscopic and consistent, to analytical concepts which are macroscopic, flexible and fuzzy. So here is –finally!- my sales pitch: do your proper business analysis for analytics well. <b>Because the cost of preparing for a well thought through system is a fraction of the license-, hardware- and maintenance cost.</b></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h4 style="clear: both;">
Epilogue: an initial approach</h4>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
A first attempt to map the various data ingestions to Vector H and the consumers of the data was made as illustrated below. This has a few consequences we will discuss in the next few paragraphs.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-LBtiOQbCYh4/WjjlAhs7FQI/AAAAAAAAHZk/1AU33r1VAR8bL0NPFde8YR3-grQvYHBMgCLcBGAs/s1600/fig8.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="254" data-original-width="605" src="https://4.bp.blogspot.com/-LBtiOQbCYh4/WjjlAhs7FQI/AAAAAAAAHZk/1AU33r1VAR8bL0NPFde8YR3-grQvYHBMgCLcBGAs/s1600/fig8.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h4 style="clear: both;">
A more in depth example of Vector H’s power</h4>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
One aspect will be along the Spark line – the ease of facilitating combined queries that incorporate data that is held in Hadoop with managed structured data in a way that standard BI tools simply query the database in the same way that they do a standard SQL database. I.e. the user does not need to use ETL or ELT separately from the actual BI query for ad-hoc queries once they have defined the external table as referencing the Hadoop data. It is hard to define the simplification this brings. In its simple form – it’s like the data really is inside the Vector database. This brings the advantage that current solutions – including off the peg turnkey applications can access this data.</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
This example shows the declaration made by the DBA, once this is done, the end users’ business layer will simply see ‘tweets’ as a table that can be joined to actual tables </div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
CREATE EXTERNAL TABLE tweets</div>
<div class="separator" style="clear: both;">
(username VARCHAR(20),</div>
<div class="separator" style="clear: both;">
tweet VARCHAR(100),</div>
<div class="separator" style="clear: both;">
timestamp VARCHAR(50))</div>
<div class="separator" style="clear: both;">
USING SPARK</div>
<div class="separator" style="clear: both;">
WITH REFERENCE='hdfs://blue/tmp/twitter.avro',</div>
<div class="separator" style="clear: both;">
FORMAT='com.databricks.spark.avro' </div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
This command will select tweets that are made which are from customers only, those from non-customers will be ignored:</div>
<div class="separator" style="clear: both;">
SELECT tw.username , cust.firstname, cust.lastname, tw.tweet </div>
<div class="separator" style="clear: both;">
FROM tweets tw,</div>
<div class="separator" style="clear: both;">
Customers cust</div>
<div class="separator" style="clear: both;">
WHERE tw.username = cust.username </div>
<div class="separator" style="clear: both;">
AND tw.Timestamp > TIMESTAMPDIFF( SECONDS, 120, CURRENT_TIMESTAMP )</div>
<div class="separator" style="clear: both;">
\g</div>
<div class="separator" style="clear: both;">
Similar queries can track non-customer queries.</div>
<div class="separator" style="clear: both;">
Where possible restrictions will be pushed down to the Spark ( Map Reduce and Scala level ) in Hadoop to be answered. The data never needs to be stored. Of course some data may be required to be added to the structured data. I already applied this in a customer analysis project where I illustrated how the results from Big Data analytics can be transformed to dimensions in the “classical” data warehouse:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-GdL_MGE--Ek/Wjjk_MWkGnI/AAAAAAAAHZQ/AE9xtBq7EEECJKqek5okdNgdd1KzTJQGACLcBGAs/s1600/Fig9%2BFactless%2BFact%2Bconcept.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="396" data-original-width="846" height="299" src="https://1.bp.blogspot.com/-GdL_MGE--Ek/Wjjk_MWkGnI/AAAAAAAAHZQ/AE9xtBq7EEECJKqek5okdNgdd1KzTJQGACLcBGAs/s640/Fig9%2BFactless%2BFact%2Bconcept.jpg" width="640" /></a></div>
<br />
<br />
<h4>
To conclude: will hybrid architectures make data modelling obsolete?</h4>
<br />
I can’t yet generalise this for all hybrid databases but at least from Vector H we know that there is a serious chance. It uses a partition clause that distributes data on the hash of one or more columns that have a minimum of 10X unique values evenly distributed as the number of partitions you are using.<br />
<br />
Vector H is therefore the most model agnostic data store I know. You simply create a schema, load data and run queries. There is no need for indexing or some form of normalisation with this technology.<br />
Whereas the need for 3NF, Data Vault or Star schemas may become less important, governing these massive amounts of data in a less organised way may become the principal issue to focus on. And metadata management may become the elephant in the room.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com2tag:blogger.com,1999:blog-4520438534640699001.post-30218085197516018882017-07-25T09:36:00.000+02:002017-07-25T09:36:21.257+02:00What if Analytics Drove the Information Architecture Development?<h2>
<span lang="EN-GB">Introduction</span></h2>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB">Information architecture helps people to
understand their work field, their relationships with the real world as well as
with the information systems which are supposed to reflect the real world.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Information architecture deals with
objects, their relationships, hierarchies, categories and how to store them in
and retrieve them from applications, files, websites, social media and other
sources I forget to mention…<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">With the massive expansion of sensor data
rebaptised “the Internet of Things”, social media and linked open data, these
semi structured and unstructured data are adding complexity to the information
architecture. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">On the other hand, hypercompetitive
environments force agility upon the larger corporations as the next garage
start-up may overthrow their business model and their dominance in an
incredible short time span. This agility is translated in flexible applications
with point and click business process reengineering. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">So how does all this affect the information
architecture development? That is the approach to submit to your judgement in
the next paragraphs.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<h2>
<span lang="EN-GB">Analytics, the classical chain of events</span></h2>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB">In many large organisations, the process
can be described in eight separate stages:<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">A business question is formulated, e.g. who
are my most loyal customers from the past that may be vulnerable to competitive
offers?<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">The data analyst starts looking for data
that can contribute to an answer by breaking the business question into related
questions, e.g. which customers have given proof of price sensitivity? Which
customers have shown a downward trend in their net promotor score? Which
customers are reducing their purchases of consumables, Etc…<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Gathering the data is the next step: in
transaction systems, market research data, social media, e-mails,…<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Manipulating the data: from simple cleaning
and conforming operations to very complex pipeline processing of text and web
URLs to make the data useful for analysis<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">But before that, visualisation may already
provide intuitive insights: histograms, heat maps, bubble charts and the likes
may show you approaches for further analysis<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Analysing the data with the possibilities
offered to analyse text, the old dichotomy between quantitative and qualitative
research has become obsolete. Modern analytics is about hop skip and jump
between the two extremes: quantitative approaches will tell you about the
proportion of clients that may look for greener pastures whereas qualitative
analytics will probe for reasons and root causes.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Interpreting the data may follow more
intuitive paths where extra information is added, opinions are collected using
the Delphi technique or other qualitative approaches to add useful meaning and
actionable insights to the analysis. E.g. developing a customer scoring model
that is broadly used and understood in the organisation.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">The hardest part is the last phase:
integrate the data and the analytics in the decision making process. To
conclude with our example: developing scripts and scenarios for the call centre
agents that pop up whenever a client with a potential defection risk calls the
company.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<h2>
<span lang="EN-GB">Architecture development, the classical
chain of events</span></h2>
<div class="MsoNormal">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://4.bp.blogspot.com/-BYkbGPfvz74/WJ7dj5cXtZI/AAAAAAAAFYE/yGN9Pl1QQ88RlUonzjsj4L9YfDL88lXbwCPcBGAYYCw/s1600/ADM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="553" data-original-width="571" height="309" src="https://4.bp.blogspot.com/-BYkbGPfvz74/WJ7dj5cXtZI/AAAAAAAAFYE/yGN9Pl1QQ88RlUonzjsj4L9YfDL88lXbwCPcBGAYYCw/s320/ADM.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TOGAF's Architecture Dvelopment Method</td></tr>
</tbody></table>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Togaf’s architecture development method
(ADM) also follows a structured path as the illustration shows. For a detailed information on the Togaf ADM,
we refer to the Open Group website: <a href="http://www.opengroup.org/subjectareas/enterprise/togaf">http://www.opengroup.org/subjectareas/enterprise/togaf</a>
<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">At the heart of Enterprise Architecture
development is the management of requirements. These requirements are
predominantly based on process support.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">User stories like “As a call centre agent,
I want to see the entire customer history when call comes in in order to serve
the customer better” are process support requirements. The data are defined
within the context of the process. In this comprehensive case, some level of
enterprise class da ta is attained but
what about more microscopic user stories like “As a dunning clerk I want to see
the accounts receivable per customer sorted per days overdue”. In this case, no
context about why the customer is overdue is in scope. Maybe the delivery was
late or incorrect, maybe the customer has a complaint filed with customer
service or maybe the invoice was sent late and arrived during the client’s
holiday closing…<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<h2>
<span lang="EN-GB">Yes, we have a shifting paradigm!</span></h2>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB">I know, in this business the paradigm
notion is an overrated concept, abused for pouring old wine in new bags. But in
Thomas Kuhn’s strict definition of the term, I think we do stand a chance of
dealing a with a paradigm shift in information architecture development. <o:p></o:p></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://2.bp.blogspot.com/-Jxi38nXHJN8/WXbz0iO4GmI/AAAAAAAAGQ4/LeG99q_5hXMjCjkBpcuYSE2IjZ4gvdj2wCLcBGAs/s1600/Kuhn.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="331" data-original-width="219" height="320" src="https://2.bp.blogspot.com/-Jxi38nXHJN8/WXbz0iO4GmI/AAAAAAAAGQ4/LeG99q_5hXMjCjkBpcuYSE2IjZ4gvdj2wCLcBGAs/s320/Kuhn.png" width="211" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A must read for anyone in information technology</td></tr>
</tbody></table>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">I see critical anomalies: <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">inconsistent decision making depending on
the flavour of the day and the profile of the decision maker, often based on
inconsistent information which is extracted from inconsistent data. With a time
to market reducing to smaller and smaller timeframes, the old process based
architecture development method may prove to be ineffective to meet the
challenges of new entrants and substitute products and services. Although every
pundit is touting that information is the new oil, not too many companies are
using it as the basis of information architecture development. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">The old top down view leads to
underperforming data retrieval which is no more sustainable in a digital
competitive environment where time to market is often equal to the time it
takes to tailor data to your needs, e.g. recommenders in e-business, cross
selling in retail, risk assessment in insurance,…<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<span lang="EN-GB"><b>There’s external pressure from the GDPR</b></span><br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB">By now every organisation doing business
with or in the EU will be aware of the 25<sup>th</sup> May 2018, date when the
general data protection regulation or GDPR, comes into effect which requires: <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">valid and explicit consent for the use of
any data that can identify a person, <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">data protection by default (anonymization,
pseudonymisation and security measures for data,<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">data breaches communication to the
authorities and<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">records of processing activities.<o:p></o:p></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-VpQ-PxytCko/WXb0nlHc40I/AAAAAAAAGRA/7d7RNkcDlBUmyyXSz8nD4JjmwwUuF7qMwCLcBGAs/s1600/GDPR%2Bcycle.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="428" data-original-width="640" height="214" src="https://3.bp.blogspot.com/-VpQ-PxytCko/WXb0nlHc40I/AAAAAAAAGRA/7d7RNkcDlBUmyyXSz8nD4JjmwwUuF7qMwCLcBGAs/s320/GDPR%2Bcycle.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Data management activities needed for compiance with the new legislation </td></tr>
</tbody></table>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">This requires organisations to manage their
data on individuals far better and more centralised than they did in the past. Data
requirements on persons will be at the heart of the information architecture
development cycles as dealing with those on a lower level in the architecture
framework will be a sure recipe for disaster.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<span lang="EN-GB"><b>Technology also contributes to this new
approach</b></span><br />
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">At least three technology evolutions enable
the data centric approach to information architecture development:
microservices, master data management tools and hybrid databases.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Microservices enable rapid scaling and
reengineering of processes. The use of consistent data throughout the
microservices architecture is a prerequisite.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Master data management tools are maturing
as each relevant player is expanding from its original competence into the two
others. You can observe data governance tools adding data quality and master
data management functionality as well as data quality tools developing master
data management and governance services and… you know where this is going.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB">Last but not least, hybrid databases will
enable better storage and retrieval options as they support both transactional
and analytical operations on structured and unstructured data.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB">In conclusion: modern information
architecture needs flexible and fluid process management support using
consistent data to facilitate consistent decision making, both by humans and
machines. <o:p></o:p></span></div>
<br />
<div class="MsoNormal">
<span lang="EN-GB">In the next post, I will use a case to
illustrate this approach. In the meantime, I look forward to your remarks and
inputs for a thorough discussion. <o:p></o:p></span></div>
Bert Brijshttp://www.blogger.com/profile/11439837670743990741noreply@blogger.com0