Github Copilot produces Javascript with Regex to let me parse some difficult JSON!

have you ever had a need to parse some Json that the parse Json action in your flow just couldn't handle today we're going to look at a way that you can do that by using JavaScript in Azure logic apps and using an action that's unique to logic apps which is called execute JavaScript code it actually lets you execute a small snippet of code within your flow and it can accept Dynamic content into that JavaScript and then you can get the output back out of that action to do other things Downstream I'm going to show you the use case that I had for this and we're going to tap into some AI by using copilot to help us write some JavaScript to do some manipulation to some Json to make it parsable foreign by the way I'm testing out a new format for my videos I'm going to move a lot faster for things that require a really long form video I'm still going to make those but I'm making the assumption that if you're watching a video about JavaScript and Parson Json and using logic apps and copilot that you have some familiarity with these Concepts so I'm not going to go as in depth as I have been in the past because it seems like based on the stats everybody that's watching my videos really likes a shorter format this is the logic apps designer I'm going to just create a compose action in here well now I can't show you the real example of this because it was for a customer's proof of concept it was a problem that they were having and so we're going to use this one instead so as you can see here we have a Json object not an array um with two nested objects inside where their keys are grids and these keys they change every time they receive this payload the key has changed and so if I tried to do a parse Json here and I upload a sample payload here and I use that same Json object what I'm going to see in here is that the schema that it reads has really static properties uh in terms of the key value uh pairs so it thinks that it's going to expect this grid to always be the key of these nested objects and like I just said that's not what we were receiving in the payload so the problem is that if I try to run this parse Json against their payload it's going to fail validation every single time and that's just because this the scheme is dynamic so how do we solve that problem well we can use inline code and we can use JavaScript along with regex to identify those grid values and replace them with something else so the idea here is I'm going to store the Json payload in this compose action the next thing I'm going to do is create a new Step I'm going to pick the inline code action so in order to use this you need to have an integration account established and set up with this logic app so I'm not going to go through how to do that all the documentation is available on Microsoft's website and go check that out in fact I'll put a link up here for you but I've already done that here so when I put this in it says here's some sample code whatever right hello world from and then it gives us the name of the logic app so in this case JS Dash regex-cop cool uh I don't want that and delete it so we need our own code well I know just from my experience in working with regex that it can find matches for certain types of strings and I'm going to just treat my Json as a string in this case you can do that there's no login set so I'm going to take my compose action here we're going to assume that it's going to understand that this is Json but I'm going to treat it as a string okay so what I probably would do in this case is I would have my compose action here let's rename this original Json and then what I'm going to do below that is do another compose I'm going to pick the outputs of my original Json but I'm not going to do it that way I'm going to Express them as a string so I'm going to write the word string and then pick my outputs from original Json I'm going to stick that in and I'll just call this one string Json so now we need code right well are we gonna get that I don't know how to just smash out some JavaScript for regex so I'm going to go over to vs code and I'm going to ask it to help me create some JavaScript to do regex against this string so let's go over vs code all right so here we are on vs code I have the co-pilot extension installed if you don't know about copilot go check out my other video there's going to be a link up uh top of your screen somewhere to take you to that one it'll teach you how to use it how it compares to chat GPT and some other stuff but basically the way it works is if you install it in your favorite IDE then you can just put code comments in and it will go and search for code that it thinks might work for you so what I'm going to do here I'm I have a new Javascript file open you can see that in the bottom right hand corner of your screen I'm going to put in a code comment and I'm going to ask it for regex to replace all grid values in a string with a new string called key string I hit Ctrl enter and I'm going to let it try to synthesize something for me okay so it gave me some possibilities here and these actually look pretty good not going to tell you that you know I got this on the first try because I didn't I had to go through and figure out you know how to make this work properly so I hit accept on this and it brought in all this junk but we really only need the first one so I started going through this to see if I could figure out what it was doing here I'm going to delete my original prompt as well so if I look through here I have a bunch of variables and the first one is called keystring and it has this word new key well I didn't want new key but I mean I could use that basically I can put anything I want in here so when I want to replace those with values in our Json whatever I put in here in the keystring variable is going to replace those Dynamic widths so I'll leave new key actually let's see if we can get it to work with just new key so this next line VAR reg X is a variable that holds the actual regex expression that's going to be used when this script looks through our string to try to find matches and you can see at the end here we have gi slash GI so this means global G and then I is a case insensitivity I think so basically it's just going to ignore casing then we have VAR Str now this is going to be our string value so this is going to be the Json string in our flow and we're going to have to replace this string with the outputs of our compose where we expressed our Json object as a string below that we have VAR new Str so new string this one takes the Str and it does a DOT replace on that it replaces anything that matches the regex with our new keystring and then it logs it to the console so if we wanted to test this in vs code we could do that just by going up to run and hit run without debugging but you can see down here in the debug console um we started with this string and we ended up with this one where we have a grid new key in place of the original wood value so we know that this works now how do we make this work for our use case well let's go back over to power automate and play with this a little bit so I've copied the code that copilot came up with and I'm going to bring it over here to the logic apps designer and in my code section for execute JavaScript code this action I'm going to paste it in now I'm going to kind of clean it up a little bit for it first and foremost console.log is just going to spit out whatever it is you're asking it to write to the log so in our case we're writing new string to the log new string is going to be our string variable with any grids replaced that's great that's what we want but the console.log doesn't actually give us what we need to get things out of this action so if you go and add another one of these actions you remember earlier I deleted what was in there because I didn't need it turns out there's a little hint in there so we go here to inline code execute JavaScript right here on line three we have return text let's go down to our our original action and instead of our console.log let's say return and then this time let's do new Str and you can see that it's actually Auto suggesting that for us which is pretty cool it realizes that that's a variable within our code and so it's okay maybe this is what you want it is so we're going to return new string so at this point we could go ahead and test this just to see if the execute JavaScript code is actually working the way that we want it but in order to do that we need to come down and add one more step so we'll add another compose action and in this one we will get the result out so you can see result becomes Dynamic content from our previous action our inline code action we'll put the result in there so what I'm expecting to see here is this string but with new key in place of any grid values that are in there and in our case it's just this one so let's go ahead and save the logic app remember you've got to have an integration account attached to your logic app before you come in and start building this and that's done in the workflow settings go check the docs for more information on that so let's go ahead and run our trigger okay it says successfully check the trigger and there we go we had it run and so now you can see down here this is a string with a good new key in it so that grid value that was there before is no longer there and if you remember if we look back that's exactly the same result that we got in our debug console when we ran this JavaScript in vs code so we know that things are working the way that we would want them to work the problem is we don't care about this string we care about our Json so that we can make it parsable so let's go back in here to the designer okay so we need to replace far Str the static string that's in here in our test we need to replace that with our Dynamic content and our Dynamic content needs to be the string Json representation of the Json payload that we're trying to work with so what's cool about this is that you can just cut this out and then you can just pick that Dynamic content and it's going to put that in there for you boom there you go and actually if you look at this closely there's a lot of different things you can do with with this if you remember um in the when we originally loaded the execute JavaScript code action there was some test code in there and it actually used this workflow context as well but it used it um at the workflow level so it said workflow context.workflow.name which gave us the name of the logic app workflow in this case we're trying to dig into the actions specifically the string Json action and the property we're trying to get from it is the outputs but it's very easy to just put your cursor where you need it to be in your code and then click the dynamic content over here on the right and it will stick that in there for you let's go ahead and run this again okay and as you can see we now have a new string and it is our Json but we no longer have the grid Keys all right we have new key instead which is exactly what we were trying to achieve how I've been playing a nasty little trick on you because this is still not parsable what we have is two objects with two different schemas both with the same name and parse Json does not like that it turns out it detects duplicate objects and it will only parse the first one that it hits when you try to upload a sample payload so what we just saw there where we had our replaced with values with new key it doesn't work it will only give you back the schema for the first object with that name that it encounters so so we need to make some modifications to our JavaScript code so what we can do here is we can say actually we want two keystrings we'll have VAR keystring two and that will be new key two so now we have another keystring that we can pass in and the really interesting thing here is if you remember on our regex we talked about global global what it does is it matches all the cases and it replaces them so what we can do there is we can say actually we just want to replace one okay so what that's going to do when we invoke this dot replace function down here is it's just going to replace the first squid that it encounters with new key the second WID will stay a grid until we do something with it so what I'm going to do here is say now I want varnu Str to be new Str dot replace still going to use regex but this time we want to use keystring 2. so because we've told it we're not doing Global here and we've got another dot replace function happening this time against the output of the previous replace action what we should see here is the first object showing up as new key and the second object is going to show up as new key 2. and then finally we're going to return new Str which is our our final output that we're hoping to get so let's run that and see what happens okay so now if we hit show raw outputs you see our new key here and again all these slashes and things that you see this is because uh it still thinks this is a string and technically it is but we have a new key with the created two column description to name two and then we have new key two with created um deleted the yada yada yada you get the idea so now that we've done all that we can finally get to parsing Json so I'm going to go ahead and do a new parse Json action now I can actually get rid of this compose now because I know that what I'm getting out of my execute JavaScript code action is what I want so I'm going to delete that remember it still thinks that our Json is a string so we need to tell it that it isn't by invoking the Json function here against our results for this schema I'm going to use what I've got in my clipboard so we have new key and new key too that's what our script is producing I'm gonna hit done and so now we can see we have type object with these properties we have new key and new key two with individual columns for each one right or individual properties for each one so if we do a compose after that we can get access to everything now um we can pull out uh the name value we want to which will come out of I believe new key two we can pull out type 2 which would come out of I believe new key one and we can actually validate that by just clicking one of these and hovering over it you can't see it on your screen but that tells me that type 2 comes out of new key what's cool there too is that since we didn't make this into an array in order to reference it that way it's an object we can just directly reference these columns and it'll pull off straight out of where it needs to come from the other thing you can do is if you wanted uh to grab the object as a whole you could say give me New Key and then that would just pull just the new key set of properties out for you and express those as Json for you to see why don't we just pick one here I think description two should come back as test so we pick description two we save our logic app and then we run it our final output there should be um test and there it is all right that's it for this one so things we learned you can run JavaScript and logic apps co-pilot can help you write JavaScript but you may have to get clever and add to it later and seemingly impossible is that a word non-parseable Json can actually be parsed if you're clever enough to figure out how to do it but now you know so you know tell your friends or whatever thanks for watching [Music]

Channel: Doing It With A.I.

Views: 265

Keywords:

Id: fWnJ1qrl-aM

Length: 18min 21sec (1101 seconds)

Published: Tue Feb 14 2023