Amazon takes a step in direction of a zero ETL future with two bulletins at re:Invent • TechCrunch

Amazon made a few bulletins right this moment at AWS re:Invent in Las Vegas that helps transfer information administration in direction of a future with out the necessity for ETL.

ETL, extract rework load, is the bane of each information scientist and crew as they attempt to get information into form to place it to work. As AWS CEO Adam Selipsky defined, you could have information in various totally different locations like your utility utilization information in a database and buyer evaluations in your information lake. Placing them collectively has been a big problem up till now.

AWS launched Aurora zero-ETL integration with Amazon Redshift to offer prospects utilizing the Aurora database and the Redshift information warehouse, the power to maneuver information with out having to carry out ETL on it.

“We’ve been working for a number of years now and constructing integrations between our companies to make it simpler to do analytics and machine studying with out having to cope with ETL,” Selipsky instructed the re:Invent viewers.

“However what if we may do extra? What if we may remove ETL fully, that may be a world we’d all love. That is our imaginative and prescient, what we’re calling a zero ETL future. And on this future information integration is not a handbook effort,” he stated.

“So right this moment I’m excited to announce the preview of a completely managed new ETL free integration between Aurora and Redshift,” he stated.

Whereas he was at it, he introduced an analogous integration between Amazon Redshift and Apache Spark, the favored open supply massive information processing platform. It gives comparable capacity to maneuver information between the 2 platforms with out having to extract, rework and cargo first.

The Redshift-Aurora integration is in preview. The Redshift-Apache Spark integration is obtainable now throughout all areas.