Amazon Web Services, Inc. (AWS) yesterday announced the availability of Amazon Kinesis Analytics, a fully-managed service for continuously querying streaming data using standard SQL.
Using Kinesis Analytics, developers can write standard SQL queries on streaming data and gain actionable insights in real-time, without having to learn any new programming skills.
Today’s digital businesses generate massive quantities of streaming data from diverse sources such as website and mobile app click-streams, sensors embedded in connected devices and IT system log files.
Being able to continuously query and gain insights from this information in real-time – as it arrives – can allow companies to respond more quickly to business and customer needs. However, existing data processing and analytics solutions aren’t able to continuously process this “fast-moving” data, so customers develop streaming data processing applications – which can take months to build and fine-tune – and invest in infrastructure to handle high-speed, high-volume data streams that might include tens of millions of events per hour.
With Kinesis Analytics, AWS aims to make continuously querying streaming data in real-time as simple as writing SQL queries. Kinesis Analytics integrates with Kinesis Streams and Kinesis Firehose and can automatically recognize standard data formats within data streams and suggest a schema, which AWS says is easy to edit using Kinesis Analytics’ interactive schema editor.
Kinesis Analytics automatically provisions, deploys and scales the resources required to continuously run queries, delivering processed results directly to AWS services, including Amazon Simple Storage Service (Amazon S3), Amazon Redshift and Amazon Elasticsearch Service.
“AWS’s functionality across big data stores, data warehousing, distributed analytics, real-time streaming, machine learning and business intelligence allows our customers to readily extract and deploy insights from the significant amount of data they’re storing in AWS,” said Roger Barga, General Manager, Amazon Kinesis.
“With [this] addition, we’ve expanded what’s already the broadest portfolio of analytics services available and made it easy to use SQL to do analytics on real-time streaming data... ," said Barga.
Customers can get started with Amazon Kinesis Analytics by going to the AWS Management Console and selecting a Kinesis Streams or Kinesis Firehose data stream. Kinesis Analytics ingests the data, automatically recognizes standard data formats, and suggests a schema that can be refined using the interactive schema editor.
Next, customers use the SQL editor and built-in templates to write SQL queries and point to where they want to load the processed results. Kinesis Analytics takes care of everything required to continuously query streaming data, automatically scaling to match the volume and throughput rate of incoming data while delivering sub-second processing latencies.
MLB Advanced Media (MLBAM) is a full-service solutions provider that delivers digital experiences through all forms of interactive media.
“We capture telemetry and clickstream data from our video streaming clients using Amazon Kinesis Streams. We process that data in real-time to monitor the video streaming experience we provide our customers,” says Rob Goretsky, Director of Data Engineering, MLBAM.
“Insightful analysis of streaming data previously involved building dedicated data pipelines to derive specific metrics. We can now interactively develop queries in minutes... The familiar SQL API allows our data engineers fast and easy access to the data, and the interactive console provides immediate feedback.”
Customers can launch Kinesis Analytics using the AWS Management Console, AWS Command Line Interface (CLI) or AWS SDKs. They pay only for the resources that their stream processing applications use. There is no minimum fee or setup cost.
Amazon Kinesis Analytics includes technology components licensed from SQLstream. It's available in the US East (N. Virginia), US West (Oregon) and EU (Ireland) Regions and will expand to additional Regions in the coming months.
Article published by icrunchdata
Image credit by Reuters, Kim Kyung-Hoon
Want more? For Job Seekers | For Employers | For Contributors