Stackdriver integration with pubsub and Cloud functions - Event driven architecture

I guess welcome back so as a Solutions Architect you have to provide the best possible solution for a given scenario so one scenario that we're going to discuss in this video is commonly used in enterprises as part of the cloud architecture so in this video you're going to see how to trigger cloud function whenever there is an event that occurs inside the GCP console so end of this video you will know how to use stag driver logging not monitoring and then you will also see how to use pub/sub which is publisher and subscriber model in DCP and it also see how to invoke the cloud function through the pub sub when integrated with the stag driver logging so i will go to the console and explain you step-by-step how to get this done but the scenario which I'm going to discuss here is a simple scenario so whenever an instance will spin up using compute engine I want to label that instance with specific labels like you know environment is equal to test or something before we get to this solution in the console let's rewind and understand what is the scenario here the scenario is that as soon as a VM instance is provisioned we have to apply the labels to the provision VM instance now we have to come up with the best possible solution so you can think of like creating a cron job running it for every few minutes and checking if there is an instance and applying the tags that is one way of solving this or you can also you know do it manually of course so that is another way of doing this but we have to come up with the best solution and this might even appear in your rocket at exam so the way we're going to implement this by is by using couple of very good services in DCP so one thing is stack driver because stack driver is logging all the information related to your VMs and the second is pub subs so if I have to show you how stack driver log looks like you know I have provisioned one compute engine instance here and if I go to the log you can filter out other other stuff and then select only specific resources that you want to see the logs for and I have selected GC VM instance service and I've selected all locks you can also configure that like based on the error criticality and other stuff right now you can see that these are the events for the VM instance that have created that have provisioned right now so for instance one you can see everything is logged here so this log is very valuable for us for this solution and notice the next thing what we want to do is you want to pick specifically the log information that's useful for our solution right so so the scenario here is every time an instance is created we want to apply the labels so we just need computing instances insert scenario I mean insert filter so what exactly we need is this one right this is the entry that we are looking for if you expand this and expand the log it has all the information related to the instance that what spun up right so this in this information is good for us so what I'm going to do is I'm going to create a sink so matching entries and this is the filter you got all the matching entries for that filter and then create the sink sink name I can put this as a compute underscore create right it can be anything and the destination obviously is pub/sub right so the thing services pops up here and sing destination if you do not have the topic we want to create a new topic here instance label right so create sync okay the sync is created you can also look for the sync from here logs router and if you want edit the sync you can edit it here now if you go to pub/sub and refresh this you can see there is instance underscore label right so the the this is the top e that we have created right now so click on the topic and here we're going to select trigger cloud function so basically what we are trying to do here is we are trying to trigger the cloud function as soon as the messages are sent to this this topic so trigger cloud function you can give it as a function one or default any name that you want and I prefer Python 3.7 so I have written a small piece of code for this I will copy that from my code and then put it here so if you don't know how to use cloud functions I'm going to give you the link of this video of my previous video in the description or you can select from the top and you can see how to use cloud functions in this in that video so basically you know what it does is it picks up the event so the Heavin event here is the message that we are getting from stack tagalog so that's the event to it so the event comes as a encoded format and we're going to decode it using you know base64 decode and then we want to get all the data from the T vent so the data comes here as a JSON format so I'm going to load it here I'm loading it into my final variable and then I'm using I've used it in a very rudimentary way but you know I'm going to modify this to use use iterations in Python to iterate into the JSON dictionary and get the information properly but for this tutorial you know it's it's it doesn't matter we just have to get the right information from the JSON and apply the labels for that so here I'm getting am getting the resource name the project ID zone because these are all important for it's for calling the API I am going to give you the exact you know documentation of Google for calling different API so for four different things so we are going to use here set labels API for the compute engine and and this is the body so I'm passing environment test as a label for any VM that that is that gets one up or that's provision in my project okay so I'm going to save this we have a function named trigger DS cloud pub/sub and topic is the instance underscore label and this is the function to execute we have the same function here name and freaking create it's going to take a while I'm going to pause this video while it's getting created okay so the function is created as you can see there's a green tick here which is which means that it is active right now now what we'll do is we'll try to create a new instance and see if this is going to work create so while that is happening you can see for the previous instance there were no labels here right so they were where is the label section yeah as you can see there are no labels here right so for the new instance that is getting created right now we have to see the label getting applied automatically so if the labels are applied then the solution is working Oh as you can see for the new instance we have a new label environment callin test so if you go to the cloud functions click on the function one and you can see it got executed here around this time right you can also see the log of that when you click on view logs there are a bunch of errors here related to the import functions or you know related to the libraries and stuff but it for it executed successfully as you can see the status is okay here which means that it applied the label as soon as the main instance got spun up and this is this is an example how you can come up with solutions using GCP services and these are the types of scenarios that you will expect in the exam especially in the architects I'm not in associate exam this is more for an architect level where they'll throw you a scenario and then you have got to you have to come up with a solution the best possible solution I would say using the GCP services I hope this was helpful I'll give you all the links of the documentation in the description section and do let me know if you have a better solution for this scenario thank you you [Music]

Channel: Cloud Advocate

Views: 10,096

Keywords: Google Cloud, Cloud Computing, Cloud Technologies, GCP exam, GCP Associate exam, Google Associate exam, Professional Cloud Architect, Google Cloud pub sub, Cloud functions, google cloud functions, google stackdriver, stackdriver logging, integrating stackdriver and pubsub, pubsub and stackdriver integration, cloud functions with pubsub, pubsub and cloud functions tutorial

Id: wIVZ70ska8E

Length: 10min 23sec (623 seconds)

Published: Mon Dec 23 2019