The Future of Big Data with Data Lakehouse

Introduction
Big data beginnings
New big data approaches 
Big data challenges 
Data lakes
Data platforms
AI and ML
Business Use Cases
Conclusion
Big data challenges 
–
Many companies that adopted these big data technologies have found that 
managing the various open source Hadoop tools was complex and expensive. 
Companies, such as Cloudera and Hortonworks, sold packaged versions of the 
open source projects to address that market.
Even with these packaged solutions, a major challenge for companies adopting 
Hadoop is managing the required data center infrastructure. Clusters must be 
able to scale to hundreds or even thousands of nodes for certain jobs. They 
must be available when needed but may be idle for periods as well. Those 
requirements make it difficult to find the right balance between economics and 
the availability of the cluster infrastructure. 
The cost of storage also becomes an issue as organizations struggle to keep 
pace with the amount of data they need to store. Tiering of storage to support 
production, backups, archives, and so forth is crucial to contain costs.
The emergence of public clouds in 2010 promised to address these challenges. 
With flexible compute and reliable low-cost storage, they became an attractive 
option for building one’s own data clusters.
05
The Evolution of Big Data and the Future of the Data Platform

Download 2,02 Mb.