zaterdag 29 mei 2021

Managing a Data Lake Project

With the massive growth of online generated data and IoT data, the proportion of unstructured and semi-structured data constitutes the bulk of the data that needs to be analysed. Whereas a 50 Gigabyte data warehouse to facilitate analysis of structured data was quite an achievement up to now, this number dwindles compared to the unstructured and semi-structured data avalanche.

Data Avalanche?

Yes, because compared to the steady stream of data from transaction processing systems, we now have to deal with irregular flows and massive bursts of incoming data that needs to be adequately processed to provide meaning to the data.
New data sources emerge, other than social media and IoT data, like smart machines and machine learning systems generating new data, based on existing sources. Managing various data types and metadata in impressive volumes are just a few technical aspects which can be solved by technology. The HR- , legal- and organisational aspects are level more complex, but aspects these are not in scope of this series of blog posts. 
We are adding extra process and event based decision support to our management capabilities and that alone is worth the cost, the trouble and the change management efforts to introduce a data lake.

See you at the Webinar!

Wednesday 9th June you can tune in on a short webinar hosted by the Great IT Professional. You can still register via this link. The webinar will be followed by a series of articles on how to manage the Data Lake project. Stay tuned!

Bert Brijs Webinar on Managing a Data Lake Project