Syncsort has released the results of its fourth annual Big Data survey, detailing the top use cases and challenges organizations face with Big Data initiatives.
Based on the results, there are five key trends that organizations should monitor in 2018.
1. The composition of the data lake has been shifting. The amount of Relational Database Management Systems and NoSQL databases increased this year. Relational Database Management Systems were chosen as the top source, surpassing traditional enterprise data warehouses. It also found that cloud repositories are gaining popularity.
2. Legacy platforms will continue to make significant contributions to the data lake. There was a 27 percent increase from last year in the amount of people who believe it is important to access and integrate mainframe data into data lakes.
3. Data quality and regulatory compliance were top challenges for many organizations. Forty percent of respondents cited these as significant struggles that they will continue to try to improve upon in the coming year.
4. Another trend is that data lakes will be kept fresher in order to support data use. Seventy-one percent of respondents listed ETL as the most compelling data lake use case, with advanced/predictive analytics and real-time analytics coming in second and third, respectively. Even though these case require up-to-date data, over 75 percent of respondents reported challenges with keeping their data lake in sync.
5. Finally, organizations will continue their investment in Big Data. Ninety percent of companies found that using Hadoop and Spark instead of legacy systems was valuable in saving money and driving data insights.
“We are seeing increased adoption of data lake initiatives where organizations are very focused on governance of the data in the data lakes, increasing benefits through advanced analytics and machine learning and deployment of hybrid environments including cloud,” said Tendü Yoğurtçu, CTO, Syncsort. “But those benefits can only be unlocked if organizations have access to enterprise data, can create trusted data sets and establish effective data governance practices. This propels them to a place where they can not only adapt to digital disruption, but take advantage of it so their businesses thrive.”
Source: sdtimes
Microsoft Corp. today kicked off Connect(); 2017, its annual developer conference in New York, with the introduction of several new tools designed to help companies build artificial intelligence applications and other modern software.
The first additions are rolling out for Visual Studio, the company’s popular coding platform. Chief among them is a package called Visual Studio Tools for AI that lets developers quickly get started with a new project using their machine learning engine of choice. The bundle supports Microsoft’s CNTK framework, as well as popular outside alternatives such as the Google Inc.-created TensorFlow.
Once a machine learning model is ready, Visual Studio Tools for AI can be used to ship off the code to Azure Batch AI. It’s a service within Microsoft’s public cloud that provides a convenient environment for training machine learning models.
Specifically, the offering enables developers to have their AI applications practice on sample data until they become accurate enough for real-world use. At that point, a team can employ one of the new deployment tools that Microsoft is launching alongside the machine learning package to roll out the model to the target platform.
A company building an AI-powered mobile app, for example, would use AI Toolkit for Azure IoT Edge. The framework allows for models to be deployed to connected devices by way of Microsoft’s public cloud. Another new framework in turn lets companies run AI software directly inside their Azure SQL Database deployments.
Microsoft debuted the artificial intelligence tools alongside a number of more general-purpose products that should appeal to even more developers. Most notably, the company has unveiled a managed version of the popular MariaDB database and Azure Databricks, a new service based on the popular Apache Spark analytics engine. It’s the fruit of a yearlong collaboration with Databricks Inc., the recently funded startup that was founded by project’s creators to commercialize their work.
Back in Visual Studio, a new tool called Live Share aims to ease collaboration for development teams. Microsoft says that the capability will enable multiple programmers to edit and debug code in a single view a la Google Docs.
The company has also announced several smaller enhancements across its development and cloud portfolios that are being spotlighted at Connect(); as well. A complete list of the features unveiled at the event so far can be found here.
Source: Siliconangle
