06. Azure using Python SDK : Azure Blob Trigger Function in Action

in this video we are going to implement another variant of aure function and which is blob trigger so this is a high level diagram what we are going to achieve here we are going to have some files on our local we are going to push those files using our code to Azure and Azure is going to have the blob storage and the container the idea is to push all these flights to this blob container as soon as the files are getting pushed to this blob container we should be having the Azure function auto trigger and then it's going to print out some details about the file so this is what we are going to achieve in this video so let's go to our visual studio and see how it can be done in the first part we are going to implement aure function and this particular resure function is going to be blob triggered resure function to implement it we are going to follow the similar steps what we have used for sttp trigger function also it's just that few things are going to get Chang over here and I'm going to highlight what that would be so to create a function I'm going to click on this particular icon it's giving me all these options I'm going to click on this create function and then I have to browse a file path where you want to continue your code files so I will click on new folder and I will just name it funk storage trigger and I will select it after that I'm going to select python as my language then model V1 for model and here I'm not going to have any virtual environment so I will skip it and after that it's going to ask for a template and this is what is being changed so when we working on HTTP trigger one we have selected this but now we need to invoke our function based on the blob so we are going to look for for the blob trigger and here we do have at the top so I will just click on this and then it's asking to give it a name I will give fun blob trigger then I will just click on create new Local app setting and then it's as for a storage account if you don't have any storage account created you can create it by using this particular option but I do have a storage account created so I will just select that and after that it's asking which particular container it has to monitor so that can trigger this particular asure function I do have a container created so I will just name it common container so I want this functions to monitor whatever blobs or files are going to be stored over here in this common container and invoke this particular Azure function accordingly so that's why it's asking for a path so you can give your container name and based on this container activity only it's going to trigger that as your function when this is done I will just click on enter and after that it's seeing open in current window or new window so I will just say open in current window only and and you can see my blob trigger function has been created as soon as this function is going to get triggered it's going to give me a blob name like which particular file is being uploaded and what is the length of the file so it's going to give me those details if you want to perform any particular operation then we can also write it over here so if you have like any other requirement you can also write it down over here so few other things what I wanted to highlight first thing we can go to this function. Json file and here you can see the name it's taking as my blob the type you can identify okay this is the blob trigger function direction is going to be in so as soon as any file is being loaded or any operation has been performed so that is going to be in operation because we are going to place the files in the container that's where the direction is in and after that you can see the path it's going to monitor that is going to be common container and the name of the blob which is a variable because the name could be anything if I go to my Azure portal you can see under the storage account this is my storage 36 that's why you are getting that option over there and then under storage 3306 I have created a container this is going to be common container and as of now it doesn't have any kind of blob or file over here but as soon as any files are being uploaded into this one then the Azure function is going to get triggered we are going to monitor this and at the last you can see this connection property how this particular function is going to connect to this part you know storage for that actually there is this connection property storage 36 under storage and from where it's coming it's basically coming from this local. setting doj on and here you can see the storage 3306 storage have a connection string so using this connection string it's going to connect to this particular storage and the container to monitor the activities where exactly this particular connection string is being reflected on UI so for that so I will go to my storage and here you will see the security and networking under security and networking you will have this access Keys options you can click on this and here you can see your connection string being being stored so if you click on this show up it will give you the same connection string what is being reflected over here so if you want to change this you can see here we do have two options one is this connection string and another is is this one if you want to change anything you can take this and copy paste it over there it will work accordingly so there want be any problem as such so let's try to run this on our local first so I will try to run it without debugging it's up and running because as of now we don't have any file on our storage so it's not going to print anything let's try to upload something so that we can see this particular function in action so I will go to my container and then I will go to Common container and after that I will try to upload something so I'm going to browse for the files I will try to upload a sample file so I will take this sample will open it and upload it over here and I will go back to my function and you can see it has given me the name of the file it is also trying to print the blob side but this file is very small so it hasn't printed anything but here if you notice this it's saying new blob detected it was keep monitoring that folder whatever we have mentioned and after that like as soon as any file got got uploaded over there then you can see like it has performed all this operation it has triggered the function also so similarly if you want to enhance this logic we want to do something else then we can also do it maybe take the file you know do some raw processing pick out some particular fields from the file and print it over there that is also doable this is how actually you can test this particular function on your local so this is first part I'm going to stop this function now so I have stopped this function now now what exactly we need to do we have tested it out on our local and we know like it's working now we will try to to deploy it on our Azure and we'll try to upload some files programmatically rather than uploading one by one so let's try to deploy this one here we can click on function app and then you can see create function app in Azure Advanced click on this so we have to provide a unique function app name so I'm going to give fun app 30006 and then it's going to ask for an D stack I will say python 3.10 then is asking for a resource Group name what would be the plan I'm going with the consumption plan and after that the storage account again it's asking I will give the same then it's saying create new application inside resource I will say yes to the name I can give a name over here and then it's will try to create this function app so this part is required because we can only deploy our functions once we have a function app on Azure portal as we have seen in our last video use so we are trying to create a function app first so that we can deploy this function on Azure yeah so you can see this function app has been created we can refresh it and you can see uh we do have this funk app 30006 over here now we need to deploy this function you can just click on this and you can say deploy to function app so I will we'll just click on this it's going to ask for a function app name so I'm going to select this funk app 30006 and it's saying like Okay you want to deploy it this will override any previous deployment and cannot be undone we'll click on deploy and you can see over here like it's trying to deploy this particular function inside this function app it's seeing the deployment to Funk app 30006 is completed so what we can do we can go to our assure portal here we can try to search for function so you can see function app over here I will click on that and you can see here we do see this fun app 30006 function app which we have created via Visual Studio I will click on this so inside this we should be able to see our function and you can see the fun blob trigger at the bottom and this has been deployed so we can also confirm the code using this code plus test option and here you see we do see like similar code what we have seen in our visual studio so this part is completed we have created cre a blob storage we have created a blob container we have created Azure functions and we have deployed it on Azure also now what we will try to do we will try to invoke a method so that like it can take multiple files from here so for that we need to have a service principle created so that we can access this storage from our local this service princial is going to get created under Microsoft entri ID I will search for Microsoft intra ID and you can see over here I will just click on this create the service principal I will try to register this app and then I will go for new registration here I need to provide a name so I will provide service principal and uh I will go ahead with the default option and I will just register it so as soon as you register this application then it's going to give you this client ID tenant ID we still need to get a client credential so for that I'm just going to click on this and I will say new client secret and we'll add it over here so now you you can see the secret ID also this is the secret ID and this is the value so we need to take this value from here I'll just copy it I will also take this client ID from here and store it and then we need tenant ID also store it here so now we have the secret value client ID and the Tenant ID as per our diagram this part is completed we have created our service principal now what exactly we need to do we need to to take files from our local and have to push it over here one important part of service principle is like we have to take the service principle and give it access to this blob storage so that our code can connect to blob storage using the service principal credentials which we have just required so the next part is to provide the service principle access to blob storage for that we are going to go to storage and then we are going to go to I and here we will try to Grant access to this resource so try to go to add rule assignment and here I will search for storage and here you can see we do have multiple Options under storage we do have blob data contributor account contributor and all these things we wanted to have a limited access to this score to our storage so according to that we have to select our rules what exactly we want like we have to take that data and upload it over here so we need a WR access I think like this storage block data contributor would be enough because it allows for redri and delete access to aure storage block container and data so I'll just select this and click on next and then I have to select members and here instead of giving this access my email ID I will search for service principle and it has just given me the service principal I will just select it review and assign and you can see like now it's trying to add a role over here and it has added it now actually we have authorized this service principal credential to access this blog storage as well if anybody is using this credential then it will have access to right to this particular blob storage the last part what we need to do we need to write a code so that it can take files from our local and then push it to this blob storage we have already covered how to access blob storage in one of our previous videos so I'm just going to to use that we can take the data from our local directory which is going to be this input I do have these three files then we have to provide the container name here the container name is common container so I will change it accordingly after that actually we have to create a blob service client have to pass account URL so in this case I will go to our properties we do have to provide the client end secret what we have acquired earlier so I will take this client ID and replace it over there then we do have the tenant ID and at the last we do have the secret value we also have to pass our account URL and this is coming from AO storage URL so we can go to environment file here we don't have AO storage URL so we need to add it over here and then we have to provide our AO storage URL and for that I can go to my portal again and here I will go to storage overview and I will click on this Json View and here we do have the storage URL so we'll just take this and place it over here so using this like you should be able to connect to storage to connect to that particular account we are going to use this blob service client here we just need to pass this account URL and the credentials once we have the account connected after that we need to get a container name and for that we are going to use this get coner client and at the last we are going to take all the file from our local directory which is going to be input in this case have to take all this file and after that actually we are just going to Loop over all these files name then actually we are passing this upload blob method to push all these files to our blob storage we'll try to run this now and let's see like if the files are getting uploaded on the Zur portal or not meanwhile I will just check it one again whether we do have any file created over there or not go to container and go to Common container we do have one file so I'll just try to delete it from here and now we will try to upload all these three files via code so let me try to run this so this code has been completed and you can see all these three files got uploaded on this container now ideally because of function is running so we should be able to see something over there so I will again go to my function here I'll click on function blob trigger and then I will go to monitor it's taking some time still the blocks are not available over here but it's going to provide similar output what we were getting when we trying to run it on our local and if you want to see how it's going to publish over here then you can also refer our our first res your function video where actually we're trying to trigger the sttp function and the logs of those functions are coming over here so that's it for this video thank you for watching

Channel: TechyTacos

Views: 1,774

Keywords:

Id: HN3tUbEjgb4

Length: 14min 51sec (891 seconds)

Published: Thu Nov 16 2023