Transmitting of data is complicated and painful. Remember few years ago, when industry just started to switch to the cloud, the discussion around how to transfer large CAD parts and assembly was very hot. Actually not much changed since that time. A huge amount of data lives on the desktops and produced outside of cloud environment. Think about variety of sensors that started to appear almost everywhere. Also, think about potential for huge amount of simulations – still many of these are run on desktop computers.
My attention was caught by Amazon newly announced Kinesis service. The description provided by Amazon is quite impressive.
Amazon Kinesis provides an easy way to process fast moving data in real-time. You simply provision the level of input and output required for your data stream, in blocks of 1 megabyte per second (MB/sec), using the AWS web management console, API, or SDKs. The size of your stream can be adjusted up or down at any time without restarting the stream and without any impact on the data sources pushing data to Amazon Kinesis. Once your stream has been created, you can immediately start loading your data, with simple HTTP PUTs. Amazon Kinesis automatically manages the infrastructure, storage, networking, and configuration to collect and process your data at the level of throughput your application needs. Within seconds, data PUT into an Amazon Kinesis stream is available for analysis. Stream data is stored across multiple availability zones in a region for 24 hours. During that window, data is available to be read, re-read, backfilled, and analyzed, or moved to long-term storage (e.g. Amazon S3 or Amazon Redshift).
If you don’t have time to read product description, watch the following 2 minutes video. The idea in a nutshell is quite simple – Amazon Kinesis will pump your data to the cloud. After that you can decide what to do and how to proceed.
I can see few potential applications of Kinesis service within cloud infrastructure today. One of them is massive data processing from desktops and corporate network locations. It can enable analytic applications, search and many other services that cannot live without data. Another application area is so called “IoT” (Internet of things) I covered few days ago here. By processing data from various sensors and mobile devices, engineering applications can provide new data input that can improve design process.
What is my conclusion? Kinesis service shows a potential of data processing from variety of locations to the cloud. Bringing data together can enable a number of a completely new ways to design, making analysis and optimization. Capturing a huge amount of real time data, can help improve quality and provide insight on how customers are using products. Just my thoughts…
Best, Oleg
