Start with the agenda of this post what are we going to cover in the next one are so the first thing that we are going to do is you know understand what real-time data analytics means you know what is this and why is it relevant in today's day and age and how is it different from the normal data analytics that you and I you know do post that will move on to the use cases where real-time data analytics you know is being applied today and you know why is it so you know critical to be able to do real-time data analytics and what are the use cases you know where it applies after that we will move on to you know understanding.

How we can write so every time you have a lot of data you have to you know be able to model it in a certain way so that it is easy to you know analyze right after that what we are going to get into is you know how do you analyze real-time data now that you're modeling it so now that your model data in a way it is easy to analyze how can we actually go about doing the real-time data analytics after that we will get into spark and a particular component of spark called spark screaming which is which can be used to analyze data real-time and in the end which is the which is my favorite part.

We are going to actually do a real-time analysis of tweets as they are happening during this webinar and you know we are going to find some insights from all those tweets okay so this is the agenda we're going to do so let's get started okay so what is real-time data it is essentially real-time data analytics is the ability to analyze data as soon as it is produced so data is produced all around this like every tweet we make every social media post you know every you know user visiting a page on our website user touches in the product on our e-commerce website all of this is essentially data which is produced at different points in time what real name return will it exists is the ability to you know will gather insights from this data as soon as it is produced so literally has zero millisecond latency now what is there is another term that is related to real-time data analytics which is near real-time analytics.

What this means is you know data gets produced but you are given a little bit of leeway you know you can analyze the data within a given time frame for example you know let us say we have an e-commerce setup where you know users are coming and purchasing products what I want to you know do is in the last 1 minute I want to find out how many products were sold right and this I want to do based on each event which is the order placed event so one after the other order is getting placed and I am doing the near real-time analytics where I am you know within I am trying to count how many orders or place in the last one minute so near real-time analytics is essentially the you know analytics where we analyze data within a given time frame after it is produced right.

This timeframe is dependent on the use these the use case that we just spoke about is you know users placing orders on an e-commerce website and there the business teams ask you to give the report every minute how many orders were placed so that use case of analytics which was came you know the one minute in which you had to count the number of orders let us say it will define the timeframe in case of near real-time analytics ok so now we know what real-time return will it exists and we also know near it looks produce now I would like to take you know a little bit of time and discuss what is the difference between the traditional analytics batch analytics that is done using Hadoop and MapReduce compared to real-time analytics so if you take any large analytics deployment in any company you know what happens is that you know data gets gathered like in this example orders get placed throughout the day in the order management system of that e-commerce company and drop will run in the night which will pull it out of the orders table in orders database whatever system it is and input the data overnight into Hadoop or into HDFS or whatever and then the next day you will analyze all the data you know think did the data that has come from the previous day or the previous week.

So different use cases based on the amount of data you want to you know analyze traditional batch analytics happens this so transactional systems produce data though that data is at some frequency imported into analytic systems like you know and or spark even and you know after that it is you know analyzed to the time billing which may be days which will be weeks whereas as we know we just discuss the real-time analytics so real near real-time data analytics is about the ability to analyze data within a very short time frame from which it is produced so having any you know borrowing any procedures from this traditional batch analytics where a nightly cron job or something like that gathers all the data and impulse it into your analytic system is not going to work.

I wanted to highlight this key difference you know battery batch analytic systems you know run once a day there you know have you use you school or they use some you know HDFS copy to is to which from which they copy data from these transaction systems into Hadoop and then the batch analysis runs the next day whereas the real-time data analytics which is a topic of today's webinar the system that we build we need to have the ability to analyze data as nearly as soon as it is produced okay so now just to summarize this we have learned what real time data analytics is what near real-time data analytics and the difference between traditional batch analytics and real-time analytics.

So moving on now let's look at a few of the use cases where real-time data analytics is required the first use case is very simple I don't even know if this is an analytics use case per se but it's a real time use case definitely let us say there is a system you know that is deployed in the intensive care unit beside each fed in the intensive care unit which is monitoring the key buttons of the patient right such a system you know is the main decision it has to make is to raise an alarm in case the patient needs immediate medical attention a system like this cannot work with the traditional.

Post a Comment

Previous Post Next Post