Azure Logic Apps connect to Firewall Protected Blob Storage

hey everyone this is Adam and in this episode I want to show you how to solve a challenge of connecting logic gaps to a firewall protected blob storage so that you can still use your logic gaps and maintain all the cool features but keep your data sensitive data secured all of that in today's episode stay tuned [Music] today I will start my demo by showing you the classic approach of connecting logic apps to storage accounts and because I want to show you what happens underneath the scenes in a classic approach so that when I explain the proper way to set it up with firewalls you will see clearly the difference and be able to set it up on your own first of all when you're connecting a logic up to a blob what you first do you create a blob trigger in the block trigger before you can connect and start sourcing your data from the blob storage what happens underneath the scenes is the creation of a resource called API connection for the blob storage this API connection will contain an accounting and an account key and this is the way that your logic app will authenticate and authorize to the two blob storage and create all the requests using this connection what you need then to do is to configure and set up a schedule for this block trigger so how often does this blob will be checked for the new files and the rest is pretty classic you get the blob content using the next action and you utilize the same API connection to get that block content that's fairly standard and that's normal thing that you're gonna experience when using logic apps let me go to our portal and implement the scenario very quickly so we can see what happens when the firewall gets enabled inside the virtual portal I have my research group already created in which I have two logic apps one which I will use to show you the scenario without firewall and one to implement that will work with firewall enabled and of course a storage account that I'm gonna use to present the scenario and upload files inside of my storage account I'm gonna open this in the next tab I have a one container called demo we're going to use this container to upload all the so let's go to the logic up to the first with no firewall and set this up what we need to do is create a blob trigger I'm going to start with blank logic up and search for outer blob storage since I have it on my recent ones I don't need to search and I'm gonna use that trigger when the blob is added or modified properties only and I need to give it a name notice something here manually enter connection information if you press it notice that you can specify a storage account name and a storage account key yourself or you can use just this wizard give it a name and select the storage account that you're gonna be working with this case is my logic apps blob firewall demo hit create and as I said underneath the scenes right now we created a connection you can actually verify this by going to a resource group in which you're gonna find new resource with the connection if you hit refresh after half a minute you're gonna find other blob connection here and this is type of API connection so right now we need to select the container which in case it's demo an other new step also from the same connector are two blob storage and in here we need to find get blob content and one we need to pass here is the path and we're pretty much done what we also can change here is the interval just for the demo purposes I can change it to one minute or I will be able to force this trigger myself so I will go now back to the containers open a demo container and upload one file form quickly this is just one of the JSON files that are prepared for this demo containing some arm templates since we uploaded the file we can go back to our logic app and open here the history right now it's not going to be there yet because it's checking only every three minutes but you can force to check it right now by hitting run trigger so it will now go to the blob storage and verify what are the new files since we created and enable this logic app in just couple of seconds you should be able to create press refer to find succeeded run of your logic up and as you see we were able to find a new file that we just uploaded called zero five man Jason and get the content of that file now let's go and enable the firewall on the blob storage so let's go back to blob storage good off firewalls and virtual networks and select selected networks hit save and firewall has been enabled in just couple of seconds this will take an effect and no longer this logic app will be able to work properly so let's test this by again uploading some file to the Container so let's go back to overview containers demo and as you see the firewall are ready to connect and it says I cannot connect to the demo container and I cannot upload any file because neither does my IP is allowed so what we need to very quickly check is go back to farm walls and add your own client IP as you see there's a feature for this for to do it very quickly which checks your public IP so hit save and it will add your public IP to the list of allowed IPs so that when you go back to overview alright you need to hit save don't forget to hit save go back to overview go to containers demo again as you see we can refresh and see the files upload and let's upload one more file in this case example p3 we can now go back to the logic app and again force the refresh after a couple of seconds if we're gonna refresh notice that we didn't find any new runs even though we forced the run but if we click on a see trigger history we're gonna find a run that failed because it was not able to connect blob storage and the status was forbidden so we were blocked by the firewall so let's go back to the presentation to talk about how we can solve this issue first to solve this issue we're gonna replace that trigger and we're gonna use event grid even grid is a fantastic centralized service for events in our and one of the features is supporting blob storage events and thanks to that whenever there's a new file on a blob storage we will be able to execute a logic up using even grid events and lastly what is important here it's invoked from the storage account therefore it will go through firewall because it's an unbound connectivity that is actually directed from the storage account into even grid itself so let's go into secure logic app scenario now again we're gonna have a blob and in logic app but this time we're gonna add a new trigger which is even the grid trigger and to do that let's go to add report all to set up the second logic gap so I'm gonna go back to my resource group and I'm gonna open option to wave firewall again I'm gonna start from blank logic up and search for event grid I will use even grid when a resource even occurs event now I need to sign into other Active Directory so that I can actually submit a new subscription on the events of my storage account so I'm gonna press sign in and use my personal account here now I can create a subscription for my events I'm first selecting our show subscription next I'm gonna select a resource type so this is a resource type that is supported by event grid for the events we need to use Microsoft storage storage accounts next we need to select a resource name so this is all the resources that I have access to in which case it's logic apps blob firewall next we need to select what kind of event are we submitting to and what kind of evens were gonna be listening to previously we're using the blob created and blob modified and this event is fine for that we can actually save this logic up right now to verify this trigger works we can go back to the demo container and upload one more file let me choose this one this time let's override this to ensure we're working correctly with the blob storage let's go back to designer let's go back to firewall demo and let's refresh and as you see instantly we have a result we doesn't even have to force the trigger because one of the big advantages of using even grid is near-instantaneous events and as you see if we click on this we're gonna find even grid with the information of the new uploaded blob and you can find the name of that file in here inside of that URL SEC main Jason alright so we have covered the event good trigger one thing before we move on is I'm gonna go back to the firewall go to edit and in here there's one thing that I very often say everyone to do because the main difference between even great trigger and blob trigger is that notice that you never ever specified that you're triggering from only specific container which means this event will occur from any container on the storage account which is very dangerous if you're gonna make a logic up that copies from one container to another maybe to a backup and you're not going to specify even grid here you might create infinite loop executing a lot of logic gaps and burning a lot of money so if we're gonna go to history you're gonna notice inside of that history when we the even occurred what you can do is use the subject to filter and as you see block services default containers demo if you're gonna grab the string you can use it to filter the events only from the specific container and to do that go back here at the bottom press add new parameters prefix filter and just paste that URL in here now only files uploaded to the demo container which is specified here will be triggering this logic up it's very important thing about very often forgotten all right so we have that trigger done we can now go back to the presentation to talk what are the next steps all right what is a trusted service if you've seen us setting up the firewall on the blob storage you might have noticed that at the bottom there's an exception called allowed trusted Microsoft services to access the storage account so what is considered a trusted service in Microsoft well first of all it's a service that is using manage identity to authenticate and manage identity is simply set a service principal at Technical Account a user but the technical user in order Active Directory that is tied to a specific application and its full lifecycle is managed by our ad that means if you're gonna delete this logic up that account is automatically deleted if you're gonna enable this it will live as long as the slot gap exists and all the requests are authorized through our bot which is role-based access control standard rules that you use in Azure for pretty much any user or groups you can also use it for application accounts and setting up this for logic apps is fairly simple just go to identity plate and select this as on so let's go to our report hall and start setting this up so inside of our report all we need to go back to our logic app and find the identity blade which is here in here simply press on and select save which will ask you if you want to enable this and yes and that's pretty much it you already did create the manage identity as you see there's a new blade which allows you to do roll assignments from this but I'm gonna explain that in a second so what do you need to do right now is to use this manage identity and to do that that's the thing that I wanted to set up this first demo in the first demo you've seen that we were using API connection and this API connection had account name and account key and unfortunately right now you cannot change it you cannot use the standard connector swift manage identity and to leverage manage identity you need to send a simple HTTP GET request to a blob URL container path and the blob path we some standard headers and setting authentication to manage identity and you need to use HTTP action within the blob storage so the next step on our journey to set the secure authentication with logic apps we need to add manage identity which we already did but we additionally need to add HTTP request which will leverage this manage identity so let's go back to logic ops let's go to firewall demo let's go and edit it and in here let's add the new step which is HTTP select the action select the HTTP action and write down as I said this is a simple get request so how do you know how to build this URL what are the headers and everything else so inside of Microsoft documentation this is very well written if you go to get blob so if you've gone a google-like how to get blob with REST API it says there's a get blob action there's gonna be the pretty much the first thing that will pop up on Google you're gonna find so you need to create your like this with my account my container and blob path so let's grab this container path so for instance we've noticed that in our firewall demo we had a path in the body so if we go to firewall demo go to history of our execution inside of the body request of the event grid if you scroll down you're gonna find here that you have a taste and body and inside of that Jason body you have data and inside of data you have the full URL to your uploaded blob so if we go back to the designer we need to get it and as you see there's no body here that you can use but if you click on expression plus any kind of expression like concatenate and go back to designer notice that right now you have actually more options and you can use the data object from the body so if we go back and remove this this is what you need a trigger body question mark because it might be empty and the data from it and of course from this data object you need a URL this is pretty much the path to your body so as you see data URL and prisoner sang here theta you roll and press ok so right now we passed a get method we have the URL to our blob storage we need the headers again our documentation says that there are three required headers authorization header which is required date or X and a state and required an X version if you're gonna go deep dive into documentation you're going to find values for that inside of the versioning you're going to find the current version is 2019 zero seven zero seven so let's add this header which is XMS version you need that header and the version from the documentation is 2000 1907 and we additionally have seen in the commentation we need XMS date which i already prepared a small expression to get it because x ms date returns a current date of this request so you need to go to expression and I prepared format date UTC now which is great current date and in standard format and press ok all right so we have the HTTP requests done and it's not yet using manage identity so to use manage identity if you're gonna go all the way to the bottom of this HTTP request you're gonna find add new parameter here if you press it there's gonna be authentic Asian option unclick it and now you have option to select authentication method which on the drop down thirst manage identity and inside of manage identity we're done for now so what else do we need this manage engine he will be now used to send HTTP request but what happens underneath the scenes is that it will connect to other active directory to get a current token and this token needs an audience and audience is a simple URL string of an application a service that you're going to be connected - for the blob storage this is HTTP storage Arthur comm so that we generate the token for this specific service it's one of the security features right so let's grab this token audience and we need to paste it here in the audience section and we can actually safe because we're pretty much almost done here and we just need one more step right now so the very last thing that we need to do is to add a role because what blob storage will do when this request comes through it will try to validate whenever this is a proper audience but additionally whenever this manager identity has a proper role which in minimum is a storage blob data reader so a reader role for this managed identity to read data off of this blob storage so let's go to our report all and set this very quickly up let's go to logic app just grab the name of the logic up we're gonna need that now go back to the blob go to overview in here you can actually go to access control roll assignments and add a new role assignment new role assignment select a role and if you scroll down you don't need a reader because reader is about resources in Azure you actually need to scroll down and find storage blob data reader as you say you have additional other roles that you can use contributor if you want to write data to blob storage owner if you want to manage everything or you can even read here about additional permissions this give you for now we need just a reader because we're just reading data and now in the search just paste the name of the logic up and as you see if you enabled manager identity you're gonna be able to find a logic up identity here in this panel it's safe after couple of seconds and role is added you're gonna be able to find it here storage blob data reader for this logic up and now we'll pretty much done we can go back to the logic up and if everything works correctly we can start uploading files and see the results so by going back to the storage account go to container demo and let's upload a file let's hit upload upload it successfully we can go back to load recap refresh and we see a fail if I'm gonna go here and open HTTP request notice that it actually says it's a forbidden because authorization promises a mismatch a request is not authorized to perform this operation this sometimes happens because there's a little bit delay in propagating those permissions so just give it a couple of minutes you can try resubmitting here to see if it already has been fixed after a couple of minutes and just refreshing if it's gonna fail again either wait or what you can also do is go edit and in here remove the manage identity for a second so simply select to none hit save and select manage I Denis again and hit save this should fix it so let's go and rerun the previous run and as you see we were able to fix it very quickly I'm not sure what exactly is happening here behind the scenes but I've seen that disabling manager identity from this drop down and enable it again it fixes the issue if you propagated the permissions afterwards so as you see we were able to get a successful run if you review the contents of the run you're gonna see that we are able to trigger from the new blog upload and in HTTP we were able to return that blob and to get the contents as you see this is the contents of the file that i just uploaded it's a simple arm template it's fairly straightforward now if you go back to firewall logic up and upload many more files just to prove the demo is working you're not gonna find any issues overwrite the files if they already exist upload it successfully refresh the firewall demo and as you see many successful runs even though the firewall is enabled we are able to use the logic ups and connect successfully few things that I also want to mention as part of this demo are additional benefits because we were using even greater you're able to achieve faster triggers because you noticed that after we uploaded file we instantaneously got our files and executed the logic ups there is no longer this free minute which ton that you had previously and additionally you pay less and you pay less because let's compare options in first option we had two connectors using the blob connectivity we had a blob trigger and get blob content if you would executed hundred thousand of times for each new file you would pay $25 because both of those connectors are standard priced connectors in the option the second one that we created using event grid even grid trigger is still a standard connector but HTTP called is so-called built-in connector which means you're gonna pay almost half of what you did for the original logic gap so not only it's more secure but you also pay less and it's faster all the benefits of using even grid with HTTP requests so few things to consider before we close off what you should think when using this approach first of all if you have no exception set on that firewall that we've talked about this trusted service access the only other option you have is integrated service environment which will need to be used and that's going to be topic for the future in my videos but for now if you have that far no exception you're ready to go additionally when using manage identity what we use in this demo is system assigned so each logic gap is getting its own identity but you also can use something called to user manage identity which is separated from logic gap but what its benefit of it it can be shared so if you're using one identity and want to set up permission only once so that all the other logic apps can reuse the same permissions then consider using user assigned manage by then and lastly there's a cool feature in logic apps which allows you to obfuscate your data and you should use it because all this scenario were talking about is that this data is secure and you want to secure it but as you see the one problem that we can highlight very quickly is that if anyone has access to this logic app they can go here and review the logs and inside of the logs they will be able to review the contents of the file and we're talking about this being sensitive data so what you should do at very end you should go to http setting go to their settings and there's a cool feature called secure output which is the obfuscation of the data which selected to on and head down will allow you to change the security configuration when you hit save right now and reupload some of the files so let's reload let's say five files and go back to our logic up and hit refresh what you're gonna find right now inside of the logs is that you cannot review the outputs because it's also hidden make sure that you're not gonna expose your sensitive data within logic ops logs anyway because there's a firewall for a reason right so keep it like that other obfuscation but don't do it after the development phase so you at least can debunk very well your logic gaps as you see the results are pretty good we were able to maintain high security and functionality of our logic gap but we additionally got some benefits like faster execution and lower cost all of that in just couple of minutes and I say that's pretty good that's it for today he that thumbs up leave a comment and subscribe if you want to see more I definitely see you next time [Music]

Channel: Adam Marczak - Azure for Everyone

Views: 7,197

Keywords: Azure, Logic Apps, Logic App, Security, Blob Storage, Blob, Trusted, Service, Trusted Service, Connection, Connect, Event Grid, HTTP

Id: xVFmpszXmC0

Length: 26min 12sec (1572 seconds)

Published: Tue Apr 28 2020