IoT Demo: Azure IoT Hub Egress to Event Grid

thanks for tuning in my name is Robert Robert icons here I work as a senior services engineer in Microsoft and today's session is all about iot Hub and data egress from iot Up this session we will talk specifically about event grid as you can see on the slide it's already chapter two if you're interested in chapter one and the whole series of recordings I have at the end of the deck a link just jump to the link and you see the whole series with all the information about iot services in production ready scenarios but today we want to talk about data egress that means we talk about data which is already interested into iot Hub and how we can get this data out of iot Hub and as you already know there are multiple ways to retrieve and process data from iot Hub in this session specifically we want to talk about the Eventing capabilities of iot Hub and when we talk about these Eventing capabilities you should always have in mind that there are two categories iot Hub can send events to other services and one category is for sure the Telemetry so more or less the incoming data from the devices the payload from the devices but we also have a second category and the second category is all about the state of the devices think about events like a device connected or a device disconnected or a device created or a device deleted and now when iot Hub sends the necessary events it is using another Azure service so more or less iot Hub is pushing those events into event grid and then event grid is pushing those events into so-called event handlers and when we talk about event handlers you can think about any functionality that could Implement for example a web hook or maybe an Azure function an Azure event or an Azure service bus or maybe an Azure storage queue and that push push functionality makes it super easy for you to create new real-time processing whenever needed it also allows you to connect let me say the world of iot with all the devices the signals from the devices the Telemetry from the devices with business functionality and easy to implement business functionality so that it can react on signals or Telemetry data from your devices but before we go deep there let's talk a little bit about differences between maybe an Enterprise messaging service like service bus and Eventing Solutions and when we think about service bus you can think of um functionality where inside of the messaging service raw data can be stored to be consumed or stored by receivers now when we think about Eventing functionality you can think about lightweight notification scenarios so think about a condition change or a state change something like hey a device has been connected or a device is now disconnected or a device has been created and like with everything in the world it's not that black and white there is always a grayish area in between and we specifically see this also the way iot Hub has implemented inventing because remember those two categories we have Telemetry and we have device events and when it comes to telemetric when iot Hub is pushing events to event grid for Telemetry you also will get the whole payload of the message which was sent from a device to iot HUB so for sure you should hear also respect a lightweight scenario so don't push events from iot Hub to event grid where you know there is a pretty big payload attached to it so it should stay lightweight but you can use it for Telemetry as well and now let's see a little bit how event grid itself is working so when you want to push messages or events into event grid those applications who push the events are called event sources and those event sources can be Azure services so really Services provided by Microsoft but not limited to those event sources can also be custom Solutions so more or less if you have a solution and application and you want to use the functionality of event grid to push events to other let's say consumers or event handlers how they are called in the Eventing world you can easily do this there are also a lot of software as a service provider who use event grid to really push events to other event handlers and the way this is done is an Event Source pushes the event to a topic and then the event handlers can subscribe to those topics and event grid will then push the events to those subscription and in the end to the event handlers and that could be again webhooks functions event Hub service bus or storage cues and to show you how this could work I have prepared here a pretty straightforward simple and simplified demo and what I want to want to show in this demo is really how you can bring the signals from a device together with functionality from a business application and in this case I used a logic app so a low code no code platform to really react on events on signals coming from devices so what you will see is we will ingest Telemetry from a device into iot Hub and then iot Hub will push this Telemetry into event grid and event grid will push this Telemetry into a logic app and inside of the logic app we will look into the payload of the message and then we will analyze the payload and here I have chosen a really straightforward simple example so the devices will send a Telemetry message where we have an energy consumption inside of the payload and the logic Gap as mentioned will look into this payload we'll check the energy consumption value and if it is above 20 the logic app will create a task in an Outlook instance and this is really a nice way how where you can see how easily you can connect signals or Telemetry from devices with business functionality and as already mentioned here even implemented with a low code no code application so let me jump to the Azure portal to show you the low code no code application and here it is so I'm here in the Azure portal and I have here the visualization of a logic app and as you can see here the logic app is waiting for an HTTP request so more or less for a push to a rest endpoint and here's the link or the URL which is created by the logic app let me copy this because I will need it in the next step or in one of the next steps and then the logic app is doing a straightforward thing the logic app as already mentioned will look into the body in the body it will check the energy consumption and if the property energy consumption is greater than 20 then it will create a task in an Outlook instance and the subject name will be straightforward device energy consumption warning for that specific device and it will put into the body of the task the energy consumption itself the sequence number stay here with me I will come back to the sequence number and the NQ time and the logic app time so more or less the time when the logic app has really processed this message so very straightforward simple functionality but also very easy to create all this logic all this functionality can be easily created just using here the visual designer from logic apps so now we have this logic Gap the next step is we have to connect iot Hub with this logic app more or less we have to tell iot Hub hey please Whenever there is a Telemetry coming in push this Telemetry message to event grid and then event grid should push it to this logic app and we can do this here also in the Azure portal and here we see the iot Hub instance that I've already created and I go here into the events plate and in the event events played I can say hey I would like to add an additional subscription and remember to remember the sketch that I have shown you previously the subscription is really something uh event grid is using to push it to the logic app so I'm creating an event grid subscription but inside of my iot Hub instance and the way I can do this is I can give this subscription a name let me say this is device to Cloud um um consumption check so energy consumption check you can select what schema you want to use to send this Telemetry information to the logic app so the event grid schema or a cloud event schema or a custom schema for my simple example the event grid schema is good enough here I have the topic type it's an iot Hub topic type I have the instance of my iot Hub as to source resource and I have the system topic name iotop events the next thing what I have to do is I have to select the event type which I would like to push to event grid and then event grid pushes to my logic app and here you have to be a little bit careful so we have here five event types so the device created deleted and device connected disconnected event types so more or less those device event types and we have the device telemetry those different event types have different schemas what they will send to event grid and then what event grid will push to the logic Gap so in this case I was using in my logic app the schema for device Telemetry therefore I will select device Telemetry and I will unselect the device events if you would have the two different schemas or different schemas here selected you will potentially see an error message in your logic app because the logic app the way I have created it the way I have created it it really waits for a specific schema and the schema is the device Telemetry schema so let me select your device telemetry then I select the end point I want to send it to a web hook because the logic app is providing me a web hook I can select the endpoint and here I paste in the URL which I have copied earlier from the logic app and I can confirm the selection and then I just press create and then I have created the whole pipeline so now iot Hub will push incoming Telemetry messages to event grid and then event grid will push this Telemetry messages to a web hook under this URL which is my logic app I've prepared this already and let's see this life and in action so let me go to my visual studio code instance and here I want to show you if you want to do this on in your own environment you can do this as well so here I have all the necessary scripts if you want to do this on your own in your environment so here you can add the necessary script to log into Azure select your default subscription create the environment using an arm template and the environment contains the event grid instance it contains the iot Hub it contains the logic app but I thought it could be also interesting if you do some of the stuff manually for example the authentication of the Outlook connector so that you can really authenticate the logic app and specifically the Outlook connector so that the tasks are created in the right instance so here you have a step-by-step Guidance with screenshots so more or less you can create the whole environment in your own environment and after we have created this um you really can create a device an iot Hub device using the script here so I said hey The Hub device ID is device zero one here you can create the device I have done this already in this instance and then you can say hey please give me the connection string for that specific device and I'm reaching out here to iot HUB and I'm asking for the device connection string and here it is and then I'm using this device connection string to simulate a device which is sending Telemetry data where I really have energy consumption as a property you can see this also in the source code that we provide for you it's a super simple straightforward c-sharp application where we are sending a bunch of telemetry messages to iot HUB and here we also have a property energy consumption and we increase the value whenever we send a message um by one so then we have a few messages which are above the threshold that we have configured in the logic app and if you remember the threshold is 20. so when we execute this code we should see a few tasks created in our Outlook instance where the task will be shown so let me now execute this c-sharp application I have provided the connection drink [Music] and now it has finished so now we have sent really messages to iot HUB iot Hub has pushed these messages to event grid and event grid has pushed the messages to a web hook to our logic app so let us go back to our Azure portal use the larger Gap and here we can see yep here are the messages here are the messages which have been pushed to the logic app and if I now go to my Outlook instance here they are here we have now the messages that we have pushed from our device to iot HUB from iot up to event grid from event grid to our logic app the logic app has analyzed this message and yeah if there is an energy consumption you can see it here on the right side above 20 then the application has created the logic application has created an entry here in my task saying hey Robert please take care there is a device energy consumption warning for device zero one and the energy consumption is above 25 and as you can see here I've also added the sequence number sequence number which I have provided from the device itself stay with me here I will come back to this sequence number in one of the next slides but let me go back to the slide deck so that is the demo and if you want to reproduce the whole thing in your environment just download the CLI script and the necessary source code from the link that we will provide you in the last slide so let's talk a little bit about what have we seen here from event grid and what properties functions does event grid provide so we have seen that iot Hub has two categories of events the device events remember device connected or disconnected device created or deleted and the device Telemetry category which iot Hub can push to event grid then event grid can deal with up to roughly 10 million events per second so this is already a pretty high number so it is also built for scale and event grid also has a 24 hour retry policy what does that mean so in our scenario we had a logic app with a web hook and event grid is pushing the message to this webhook if event grid can't push the message to this web hook because maybe the logic app is down we are deploying a new version whatever then event grid has a retry policy of 24 hours to really push this message to the configured endpoint and if event grid is not able to push the message it also has that lettering functionality so that this event doesn't get lost what I really like when it comes to this integration of event grid is the possibility of a web hook to push messages to a web hook because as you have seen I can use a local no code platform in my case it was a power app to really make it an integration a fundamental part of my business logic to analyze the incoming telemetry web books could be used from every other compute platform and it makes it really really easy to integrate business functionality and webbook is just one possibility to push events to but it is something which is really handy when it comes to integration with other functionality potentially even already existing functionality but let me highlight also some additional stuff here think about a scenario where a device is sending message one to iot iotr pushes it to event grid event grid to the logic app the logic app processes it pretty cool so far now think the device is sending a second message to iot HUB iotr pushes it to event grid event grid pushes it to the logic app and here you do not have a guarantee that message number one arrives before message number two it could be if those messages came in in a really really short time window that your logic app might get the message or the event number two before it gets number one so you have to be aware of this situation and potentially deal in the logic app or in your consuming event handler this scenario in some scenarios it's not a big deal it doesn't matter if the sequence is not guaranteed but please keep in mind that event grid does not provide you here a sequence guarantee another thing I would like to call out when you go into iot Hub and you create a new event subscriber and you save this new event subscriber the iot hub goes for a couple of yeah for a period of time couple of seconds in a transitioning State and during this transitioning state it does not accept API calls so heft is also in mind when you're designing your Eventing solution using event grid and iot hub and that brings me already to the summary so in this short session I talked a lot about iot Hub and the push push mechanism so where iot Hub really pushes events to event grid and then event grid can push the event to one of the event handlers I've shown you the functionality from a logic app using a web hook to really consume this event but there are other possibilities as well it could be an Azure function we could also or event grid could also push the event to a service bus to an event hubs instance and so on if you want to follow on your own or in your own environment the demo here's a link to download all the source code to download and watch the video the recording and in this link you will also see other recordings all the recordings are related to iot Azure iot services in production scenarios and please stay tuned um there will be soon another episode where we will talk the whole time about how do we pull messages from an iot Hub instead of iot Hub pushing events or messages to event handlers thanks for tuning in and stay tuned

Channel: Microsoft IoT Developers

Views: 1,812

Keywords: IoT, Microsoft IoT, Azure, IoT Hub, Event Grid

Id: -vj3rMiZy2c

Length: 21min 27sec (1287 seconds)

Published: Tue Mar 07 2023