AWS Step Functions: Workflows for development and testing

alright hey everybody its Rob I'm just gonna kill some time until I see my face show up on this stream because I had to jump between yeah there we are super it's Rob we're back for some AWS step functions today we're gonna cover workflows for development and testing you see on the screen already we've got a pretty full agenda so I'm gonna try to get all this done in under one hour still the priority is your questions you're here to get your questions answered so please ask them I've got some colleagues here on the mod to help answer them and I'll also be trying to track the questions we're gonna start from an empty folder we're going to create a workflow using the step function support in the AWS toolkit for BS code that we announced last week and did a show on that's still here in the highlights if you haven't seen that yet we're gonna use the AWS Sam CLI to create a service app we're gonna hook up some CI CD with circle CI which you may already be using and then we're going to show how to move through some of these workflows so basically the scenario is we've had a security analyst come to us and say hey I want to know any time at this AWS configural that we've set up is violated and that rule is SSH access right and so let me make sure my yeah so it looks like this right the AWS config rule triggers this the start and now I want two things that happen concurrently I want that instance to be isolated out of the subnet by changing its security groups so that there's no traffic in or out and at the same time I want you to kick off a snapshot of the EBS volumes right so that we can triage it and then once those two things have been completed I want you to notify me via an SNS topic so it's a pretty real world use case on the AWS config rule itself is going to hook up to a lambda and the lambda is gonna kick an AWS lambda function and that function is going to kick off our step functions workflow we're not going to be going into config or lambda today we're just going to be focusing on the workflow orchestration parts of this so since we've got that let's see we'll go here to our trusty console and the first thing I'm going to do is just to give us a folder to work in I'm going to use a Sam CLI we're gonna jump order a little bit and we'll come back so what we're gonna do is Sam in it that's our go one dot X I use go you can use whatever you want to use it's it's personal preference right we get this Quick Start template let's call it step function dev test boom and we'll go in here and the reason I do that is just so that we have a nice clean folder to start from like I said we're gonna be working with Visual Studio code I mean just a second to bring this into the monitor blow it up nice and big for y'all and I'm gonna put some screencast mode on so you can see what I'm doing so this isn't the important thing right now we'll how we'll hide this for now and if you remember last week we talked about creating a step function state machine from within Visual Studio code so almost everything that we're going to do today we're going to be able to do within Visual Studio code there's some configuration stuff that will have to come out and do like in Circle C I will need to add some environment variables if you already have that set up in your account then you wouldn't have to do it but for the most part we're gonna stay inside vs code today so we'll create a new step function state machine here oh it's not what I wanted to see all right and then parallel so parallel is the most similar to that workflow that we just saw right there's parallel and there's map parallel is when you've got I want to do task a and I want to do task B at the same time and then hold until they both complete and then move on the map state is dynamic parallelism like I have a collection of items that I want to apply the same task or workflow to each of those items independently and then when that collection is complete move on so we're gonna use parallel and right out of the gate we'll just go ahead and visualize this again that was that little icon right here or using the command palette render state machine graph and we see that this already looks pretty similar so let's uh let's pull in that graphic from our analyst and see what we got pretty sure I left it on my desktop yeah work flow to here alright and we'll hide that and so if we go back over here and we look at this workflow now we can look at these side by side right so this is on the left the workflow that we got from the analyst and on the right that's the workflow that we're building right with this state machine here so it's probably better to bring this back over here for a little bit we want to start at parallel yeah well that looks good parallel final state is the next one that's actually gonna be notify so we need to go down here and make this notify and we're gonna be leaving everything as a past State for now just until we we show this workflow of like all right I've got to get this right with an analyst the tooltip came up there if you if you missed that and then let's see we got two states here past that's not what we want we got two branches start at isolate and then our states isolate let's call it a we'll change it back to being a past state I'm moving pretty quickly by the way I know I'm making some assumptions here that's because I want to show you that this can be done in an hour and this is all gonna be recorded and chopped up to step by step later stored here and then on our YouTube channel as well so you can go back through the lessons as you need to so you got this tooltip here right value of startup must match an existing state so to solve that we named one of our state's start out and then just make that an end state right and so now if we save this let's call it our workflow and now we want to revisit it let's bring this back over here alright so that's it right like isolate snapshot notify would go to the analyst and we say okay is this is this your understanding of the process right like is this what you want to have happen when we get notified by that config rule so that we can move on and it's yes it's isolate by changing the security groups snapshot the EBS volumes notify us mu more complete and then move on so we've got this set up using just the toolkit inside Visual Studio code we're not going to publish it yet I'm going to show you how to wedge this thing into AWS Sam the service application model there's um there's not yet direct support for AWS step functions inside Sam there's a little bit of a canonical trickery right that I'm gonna show you here real quick if you've never seen a Sam template it's a cloud formation transform so this is just a superset of cloud formation that uses this transform to give you access to things like serverless function serverless simple simple table and other things in our case we're gonna actually have to make a let's call it let's call it isolator right and let's see step functions I'm doing this from memory which is always a challenge so let's see if that's actually right state machine properties there's gonna be a role that we need to add and there's gonna be a definition and the definition is the the tricky one so with this definition again the trick that I want to show you I'm cheating right now I'm going back to my Docs to look there's a there's a sub intrinsic function in cloud formation and what that does is it takes two arguments one is a string and the other is a map and I gotta get that it's I'm gonna look it up to make sure I get it right but it's something like this right and this second set of arguments is everything that we need below so like the hello world function so let me let me pop out to a terminal here and I'll let you watch me cheat all right so we'll all right say I ran through this rehearsing always rehearse right otherwise you're just gonna wind up stuck so if we look in here there's my there's my magic incantation right I'm gonna go ahead and grab the rest of it it is an AWS step function state machine the we give it a name here state machine name a roll a RN which we're gonna need to define and then here's this first bit of that definition string so I'm gonna put all this in here and now we've got these and right here is where we're just gonna take that workflow definition that we have we're just gonna drop it right there all right and we'll have to do some manual clean up again because right now like I said this supported one thing that can help with this if you can tell on the stream I'm using a plugin called rainbow indentation as well as rainbow brackets that lets you know when you got your your indents lined up but in our case we need to bring this in pretty far let's see right there and then this one is where we want that to line up like that and then when we go back over here you'll see that's that second argument to the sub function right we just need to bring that back so again what this has done this isn't gonna make sense yet and I'm gonna pull it out but I wanted you to see it what this is is a set of replacement values so anytime you do up here or something like if you just refer to it with a dollar sign brace notation like that it replaces that with your value from the list below so that's how you sort of wedge a state machine definition into the Sam template so for now what we're gonna do just pull that out of there because we don't need it it will actually solve nothing if there's nothing to sub which is fine and that's where we are so we have also added this role a RN and I think this actually needs to be this let's see I don't know we'll leave it we've called it branch or role so we need to create another resource and we call it branch a role and it's type AWS I am role this one again I'm gonna copy in here because that's honestly that's just how I build all my state machines and stuff is by copying these because they get a little complex but we're gonna go over it one at a time on line at a time all right yeah that's the end of it so this is the whole thing right here it's not super complicated but you'll notice it's tightly scoped right so state machines in the region where we're deploying can assume this role that's step function workflows right and then when you look at what the role can do all it can do is invoke this specific lambda function right so down here we defined hello world function already we're just getting the AR M from it and it's authorized to invoke that function now it doesn't actually invoke that function yet so we could cut this out but we're just gonna go ahead and leave it in since it's there and at this point what we have is a fully functioning Sam application right we've got one lambda function that's what you call a boilerplate code so it's just given to us by the Sam CLI sorta to get us started it's currently listening for an API we don't need that we're not gonna use that we're not gonna use environment variables we'll go ahead and leave X ray tracing active but it's already set up so we'll pull this API out cuz that doesn't exist anymore we'll leave the function there the role we're not concerned with we will be concerned with the AR end of a state machine later so I'm gonna go ahead and grab that so that we have it from the beginning we'll call it branch or role description workflow AR n value get attribute branch a roll-top AR n right so at this point we can bring up our CLI here we're still in the same directory I've got my AWS account set we can run Sam validate against that template so I've done something wrong let's see isolator hmm I don't know what you think this should be smoke test this should we just push it what do you think that's what I'm gonna do sometimes it's easier to debug up there let's see let's see what happens on a good old Sam build and a Sam to play ok failed two parts template that gave us a little more readable version right in Unicode string mmm while parsing a block mapping so I must have done something weird here AWS template format version transform alright so let's go back to the one that I ran before and let's see if we've changed anything out of there doesn't look like it this is why whenever you get code that works you always always always keep it around right oh it doesn't like something in the isolator either oh so I should have called this the isolator role that's one reason that's there the isolator role my isolator is indented thank you helpful robot that is correct so isolator role is weave out dented that correctly we need to update some of this stuff here isolator role state machine should be fine I'll call this isolator role and down here two hours get rid of that Sam validate it's a valid Sam template so thank you helpful robot that was very very helpful in fact but I do not believe that you are a robot yes as Reckitt says that's rainbow indent which we use for e mo i've also got rainbow brackets rainbow braces let's see what got lots of rainbows indent rainbow is the one that I use for that and rainbow brackets for C syntax languages are the extensions that I'm using thanks rackets for calling that one out okay so we go in we build this how we doing on time where it's 17 minutes so far we've got this built we've got it pulled into an AWS Sam template it builds next function is Sam deploy guided I'm gonna skip that today just normally if you're in sort of a dev workflow I would suggest that you use what we always suggest use a different account for each stage right so each step should have their own workflow and then I would push this up to my own account because I've created one demo account for today I'm gonna do everything in one space and so we're gonna let the CI CD push it into that as if it's a staging account the best practice would be to have a separate account for staging so that when your CI CD pipeline picks this up to deploy its deploying it somewhere else and you never worry about name collisions I know if I do this right now I'll wind up with name collisions from the two just because of the you know the demo spirits so I'm gonna skip Sam deployed for now the next thing we want to do is we want to wire up some CI CD right so to do that we need a repo but to do that we also need to be in an active git repository so get in it get at all get status dot it out it up alright let's touch it ignore yeah I know I'm in BS code but little little vim never hurt anybody you know what I mean workflow paying all that other stuff is good to go all right okay we'll get magic there on the line for you okay and get commit it's weird it's like I'm all hopped up like Mike I had coffee but my coffee was decaf but I'm still like I'm just so excited that y'all are here with me it's fun so we've got ourselves in like a local repo now right so we want to create a a repo that our CI CD pipeline can get to if you haven't checked out the github command line tool it's in beta but it's extremely useful because you can do things like github repo create and it defaults to creating a private repo for you so you don't accidentally expose your your stuff on the interwebs either one thing I don't like about it it defaults to HTTPS instead of get so I'm gonna do git remote and then search through my history there we go set URL or this is just changing it to SSH instead of HTTP so it doesn't ask me every time and then if you're a G so G Z shell user GPS u P is get pushed set upstream so it makes our matcher master match the upstream master tada and now we have a repo up on github right we can prove that to ourselves real quick we're going and checking this out what did I just do repositories here we go updated 17 seconds ago tada it's a private repo that seems to be the right one it's got our work flow it's got everything else in there right so that our work flow de slj some work flow top P and you got this nice pretty picture and everything alright awesome so we can roll over to circle CI and again you can do this with any CI CD tool I just we have code pipeline AWS code pipeline that's built into AWS I just picked circle CI because it's one that a lot of people use their partners of ours it's a very successful tool and you can see look this is this is what rehearsing gets you people see all that red all that red it's beautiful beautiful red all that red failed just to get us here to the success don't worry don't worry you get to benefit from the fruits of my labor and this was actually I need to stop building this I never disconnected it before so we can add a project here no add projects repo name sfn maybe I've already got it this is see this is the downside of rehearsing as you get cruft left over in your account so red her soul that's right bark I don't want to be in there that's the wrong org man that's why okay take me out of here and back to my organization that's why you know I'm just sometimes the answer is right there in front of you you know what I mean so we add projects here we go sfn dev test setup project alright this is their default go builder and pick that up from the repo that's nice I built one up using the AWS server lists it'll be a Sam serverless sorry aw service orb see what it's called here in a minute but I built one up already that we can just copy in the AWS Sam serverless orb an orb encircle CI is kind of like a module that you can compose your your file from and so what we've done here you got to use version 2.1 to use orbs so we can get rid of that we're bringing in that AWS Sam service orb if you hit links over here in the chat let me see if I can use my fancy button correctly I hit link so what happens nothing if you hit exclamation point links in the chats one of those links will be to this orb so let me just manually go over here see if I can type that look I'm in the chat there we go one of those is the circle CI orb that you want all right we use a docker image with golang 1:14 check out the code run Sam install run our build step oh this is neat we got to give it an s3 bucket I haven't done that yet I promise we'd stick to the CLI as much as possible so AWS s3 MB what I call that we're gonna change it will call it isolator demo 20 2004 0-7 that's today tada now we have an s3 bucket it's fantastic and put that in there in a couple places so we basically just install the Sam CLI build with some flags and run this use container false this is for my fellow Gophers if you're if you're building with Sam and go it doesn't like building inside a container so you need to pass it that flag and then we've just got the one step here building build and deploy so we'll click start building but we're gonna add this manually download that config file save it and then let's go back to vs code let's make ourselves a little circle CI directory move our config file in there yeah yeah it looks good to me all right so one change added circle CIA can take file push it all right now one thing is gonna happen here because we configured that project newly it does not have any sort of AWS credentials to actually deploy that into the account so if we go back it's gonna pick it up and it's gonna try to build it yes I have when we go to start building it's gonna pick this up for one new workflow it may have kept the project settings I'm gonna take this tab off-screen for a second just to see if I've got those environment variables there's also a blog post in those or there's not a blog post in those links let me copy these for there's four actually I'm gonna give you a blog post that we have Brett gets a circle CI has new next-gen container images for going we seem to work a little faster and have a few additional tools to support testing awesome so my fellow Gophers check that out as well I just dropped a blog post in there that is specific to the AWS circle CI orb that we're using the main reason that we need that right this second is step three modify your circle CI project settings where we need to put some of these in I'm gonna check that in the project environment variables they're actually already set for us so this may well this may work and then I'll be very confused nope okay well you know we're getting better and better with the demo huh all right that was started two hours ago okay so let's um let's check github real quick let's see the circle CI file made it up there it's good to go alright let's just touch something real quick and kick off a config yeah there is a cat that's that's my little girl Cara she stands up at the top of the stairs and yells at me when I when my demo is going poorly she knows and that's when she yells so let's go over here let's go to our template let's make a quick change let's call this isolator isolator save that push that all right now demo gods are with us let's see can I get a pipeline number sixteen please oh good okay so we got a pipeline number sixteen cool let me see our our job is proceeding as planned here okay so again the for environment variables that I had to set you you didn't see there and you're not gonna see there because it's my secrets y'all but they're listed in Section three of that blog post everything else they use that we needed to do you saw me do including setting up that s3 bucket and setting up the github repo so at this point we've gotten the workflow built we've got in the work flow poll into a SAM template we've gotten C ICD up and running our first build so and we're at 29 minutes right including a little bit of roughness so let me let me hit pause right here I'm gonna check the chatroom see if there's anything that I'm missing I here at cat yeah we got the cat hi bro I have a question he's still with trademark ie so with trademark I have an answer I hope demon 12 why no love for code pipeline I love code pipeline this is just this just happens to be the sample that we built right you can't you can't build it on everything all the time code pipeline is fantastic if you haven't used it and you don't have a CI CD workflow set up already and you're big on AWS I definitely recommend you check it out it does a lot it like it's very powerful for multi stage builds and roll backs and things that are dependent on one another it is it is a fully featured very powerful tool I've had a really good experience with it it's just that this this orb was out there so I wanted to show people circle CI as well but good question on that who speedy says christma that's right I would posit that my performance reviews should be related to how quickly I got to this point I'll a whole links AWS config custom rules that came from there yeah hello there alright red Herzl that's right time will tell yeah seat time I see what you did there I see what you did there okay so this is still this is building over here this by the way the first time it runs it's got to pull the container provision the instance and everything it's about three to four minutes so not too bad keep in mind this is probably only running when you make feature commits depending on your setup if you're doing the local workflow where you're just doing Sam deploy to your own account it's gonna be much much faster and here and now you're gonna see you see how quick that built because it's just one function in go waiting for a change set to be created so it's already gone up oh no oh no let's see what happened in a cloud formation over here y'all there's always something that you don't account for right branch or staging why is it called branch er because we when hard-coded that in the config file reviewing process what's a review y'all anybody uh anybody got one of these before no what's up a DJ geek click on the review you click on the review I was getting that's one of my told my colleagues he's the best if he does a sessions of Sam and happy little api's so I highly recommend you check out both of those that's I knew that he would be the one to answer that question because he's the one with all that cloud formation knowledge so click on the review now you need oh I did create chainset first you could check the events section for the cloud form look at the all y'all know fired up review in progress chain sets one or more subs don't specify expected arguments look I made a liar out of myself y'all so okay actually specify string as a first argument an optional second argument to specify mapping of values alright so let's go back to our template that should then it we see that from a CI CD perspective it did the right thing right like it made it all the way through to deployment push that cloud formation transform stack up and it failed because I gave it some bad data but the next step is to start iterating in AWS lambda functions in place of the past states that we have before and then we can work on our business logic one function at a time so real quick if we go back over here and look at our previewed function we see that we have isolate and snapshot let's take our hello world function and we'll just rename it to be our isolate function so there's a couple places we need to do that let me clean this up we're gonna rename this the isolator function which means a couple things well and we need to put that map there that we've talked about this can be isolator function a are in and that's just going to allow us to use this in line up here and then we said we're going to call this the isolator function now so we're going to do some light directory maintenance you know what I mean some cleanup will always start with what they give us isolator we got an output down here to some helpful notes to ourselves isolator function data area and the rest of this is unchanged and what we haven't changed is the actual path and we just need to change that name there the way it does everything in go is these are all set up just to be I'm neither in a module nor my go path you're a liar I'm actually in a module there don't worry about this down here so with this little set of changes we've done two things right one oh we haven't yet there's one very important thing that I didn't do start at isolate states isolate and this is where we go back to the toolkit from last week we need to go back to our definition here this is the definition for our step functions workflow we want to start at isolate we don't want isolate to be a path statement we want it to be a lambda task statement so here in the lines we hit control space brings up the code snippets we find the lambda task state boom it fills all that out for us right so we clear out our tooltips isolate is actually the function we want and it's going to be true for this branch inside the parallel execution get rid of our old state here type task resource that's gonna be the AR n we're not gonna use any of these other parameters yet we just want the most basic version here so now this branch has changed to look like this it doesn't make sense inside our workflow definition to change that AR n but it does inside our Sam template I just need to get the spacing lined up there we go Beauty eh okay so start out isolate states isolate type task resource and here we're going to do the string substitution we talked about earlier where we get this isolator function a RN and put it up here now I'm gonna zoom out for just a little bit it might be like a little more difficult to read but I kind of want you to see everything all at once if we can maybe then collapse this other branch and this branch and then yeah I'll leave this up for a minute so this is the entire definition of our state string and the operative function here is that we start with this sub command which takes two arguments one a string and so we've defined this using the pipe allows us to define this like dock your style string I'm sorry here dock style string and then the second argument is this map of variables up here to values right and again we're using dynamic referencing so that every time this gets deployed or if it gets deployed as part of a new stack or if the name gets changed from parameters and all of that that's actually gonna stay up-to-date because we're only using the reference so let me zoom back in a little bit that's really big still yelling I'm yelling at me about go it's gonna be okay all it's gonna be okay because CI CD is going to handle Bill in there and actually if you use Sam build and Sam deployed locally it's gonna fix all that so get status yeah move to a whole bunch of stuff around added the isolator function and we push it and that's it so we'll go and check and make sure that circle CI is building the way that we want it is we'll bring that up we know that takes about three minutes so I want to move on and talk about one thing invoking your step functions and this will make more sense once this finishes oh I said I was gonna stay in BS code didn't I so another part of the AWS Sam CLI is Sam local Sam local allows you to do things like invoke your lambda function locally or run an API locally it also allows you to debug your lambda functions one of my favorite features is that it allows you to generate test events so if you run Sam local help you'll see that there's a generate event command and then again just go down the tree right Turtles all the way generate event help and you get a list of services that can produce events that Sam local can generate mock events for you in our case we want config so let's see what's available inside AWS config we get an item change notification and oversize the item change notification in a periodic rule so if you're not familiar with AWS config you set rules and then whenever events occur that trigger those rules you get a notification it's called an item change notification some item that you're concerned about or that your rules are watching has changed so that's actually the one we want and I think if we do help it's just gonna tell me what are you doing you big dummy just get an event no it gives us some other options where you can model five the mock so you can pass it a specific region or a count ID we don't need any of those so let's just I'm gonna cheat a little bit here bringing to the pasteboard so that we can see it in vs code first it's easier when we call it config event dot JSON right so if we look at this we get an invoking event which is an AWS config rule we get rule parameters all this stuff right and all of this is gonna be passed in via our lambda function is passed through I just wanted to show you this so that when you're setting up some sort of test pipeline you know how to get well-formed events that are gonna match the form that they're gonna come in from the AWS services themselves so real quick let me say this we'll check in on our build let's get in there all right it's on the Sam CLI bit so it's close to being done the other thing no generated events for event bridge yet luckily it gives examples in the console good good call out there demon 12 so yeah we're missing like it would be lovely to have a little event bridge there I'm pretty sure that that that request is noted so sorry I think I said console in Sam CLI you're right but as you point out they're also available it is noted at Giggy picked up on that it is noted in the console in the AWS management console go back over your package Sam app for deployment Sam deploy this is very exciting this is very very exciting what's gonna happen I wonder it's gonna be great when it fails because I gave it the same name and didn't delete the old stack I don't know know create in progress resources we're starting a step function state machine create complete come over here we check out our definition and oh wow look at that tada start isolate snapshot notify and starts at isolate its properly transformed that reference in the sub that we talked about to the reference of the lay our n of that lambda function so hooray we're green at 243 so it's always good to be back to green now that we have this Oh back in BS code I cheat sometimes I use the terminal in BS code you know what I mean there's this command right here AWS step functions start execution it takes a state machine a RN and again we made this one of the outputs of our stack so we can go back I mean we can either copy it here right there's our a RM we can get it from the stack in cloud formation as an output oh no I got the wrong thing didn't I yeah so I passed the wrong thing here what we actually want is the isolator itself so I you know computers they always do exactly what you tell them to do right I'm gonna push this just so that it comes up later and then we'll move on with invoking that step function make sure I gave it a good name at least when we got here isolator workflow it's better I named it right I named it for what I asked for I just asked for the wrong thing corrected outputs and then that will build in the background while we're going about our business if we come back over here our state machine a RN is available in the AWS management console now if you're running the AWS Sam CLI locally for your dev workflow you're running Sam build Sam deploy Sam deploy that's why you add all those things to the outputs as well so that when it completes deploying to your developer account or to your development account you get that AR in as an output and you can pick that up and use it then to execute based on events so again this aw a step functions start execution command either takes the air in of a state machine so we're gonna have to plug that in here not either I mean it only takes that so you got to put in the state machine a RN that you want to execute for your workflow and then we're just gonna run it we'll throw it some empty input since that step function I'm sorry that lambda function isn't actually expecting anything right now and it'll tell us there's an execution AR in a start date so it's kicked it off and we can go back to the AWS management console here and we see that it did indeed execute just now that execution succeeded and we can take a look at our graph right we we get this input state that's exactly the input we expected and if you go into our AWS lambda function that's gonna be the output state this is the output from the snapshot pass state right so it's aggregated we didn't tell it where to put anything so it's aggregated that output side by side this is the isolate function and this is the snapshot function this all deals with result path output path input path and it's all based on JSON path we're gonna do a separate session only on that coming up it's listed on my twitch channel twitch.tv slash Rob Sutter I can't remember the exact date off the top of my head maybe one of my colleagues can find it from the from the schedule but it is coming up in the next couple of weeks right here during this series and then of course notify you see we get the same input passed all the way through because these are just past States so now at this point we can go back to vs code and we can begin to iterate on our lambda function until it actually performs the the business requirement which is to take in that it's gonna be an events dot AWS config I forget what the name of it is takes in that event and then returns whatever we predefined as the output after it's changed the security groups on that ec2 instance right so at this point it's just up to you to do the work I'm not gonna do the work here we're at 2:47 we've seen how to define a basic workflow using the AWS toolkit for visual studio code and how that let us match that workflow to what the business analysts provided us we've seen how to use AWS Sam CLI and how to bring our workflow definition into Sam CLI with that magic incantation using the sub intrinsic function we've set up C ICD using the circle CI orb again if you're not already using C ICD also check out AWS code pipeline it's a fantastic product we've shown how to iterate in some functionality for the step function and I'm gonna I'm sorry for the lambda functions and I'm gonna show you one more bit on that and then we've shown how to invoke the step functions remotely using events that are generated by Sam CLI and that forms the basis for your integration or into end testing right your ability to programmatically pass it known events and expect known outcomes so since we've got a little bit of time I'm gonna crawl the questions in just minute I want to show you one more thing in the template so we have the isolator function but we also know that we needed a snapshot function right so we can close this and come back over here and just duplicate this right and so what we've done right now is just add a snapshot function inside the Sam template that's all it knows about so we need to also duplicate the code oops we'll rename this to snapshot just like we had in our template super we probably want to copy the output because we'll need it for other stuff later we'll call that the snapshot function and that's a lambda function to initiate EBS volume snapshot but our workflow doesn't know about it yet right and it needs to know about it in a couple places one it needs to know about it here where we change this definition and what we can do is just copy this isolator state actually we can just copy the inner information here from the isolate state into the snapshot state but it needs to be the snapshot function a RN but sub doesn't know about that yet so you got a whoops too much too much so you got to go in here and give it another value in the map snapshot function a RN Maps want commas snapshot function dot a RN and then one very important place is in our I am role right because now our workflow knows about that function and it's going to try to invoke that function but it doesn't actually have permission to invoke that function we'd like to keep our I am roles scoped as tightly as possible so we don't use stars we use references right and that's it like now we've plugged in the second lambda function into our workflow quick commit and redeploy and that's all it takes so implemented snapshot function get push and then circle CI is gonna pick that up for us it's going to update our stack and then we'll have two lambda functions where they used to be none hold on those goes away early there we go build and deploy won't let that kick off over there and I'm gonna check in the questions let's see I know my people answer I've got to pick up my microphone and walk over here so I I apologize if there's any noise you're gonna you're gonna see stay blazing oh well nobody seems to really care about cybersecurity little off-topic but where would I learn cloud security I keep reading about how it's getting hacked and they need help every platform requires its users to build their applications in a secure manner the first place I would start is just with I am right so I would start with aws.amazon.com slash I am gain a good understanding of it and gain a really good understanding of the shared responsibility model because there are things that you can't touch that AWS takes responsibility for security but then there are also application elements that you own and then you're responsible for the security of those elements that's where you really want to focus your your efforts and your learnings edj geek time for Global's I'm not going to cover Global's today but I am listening for you at J geek so you can start here thanks @j geek for dropping that security link in there appreciate the questions stay blazing you up the lads down the cans I just deployed my group project to 8 OBS for my first time playing anything was very simple and lovely great glad to hear that I hope I wish you many more successful deploys and far fewer red deploys and I have had so many videos to watch another AWS channel I have two sub two from protobuf hey thanks for the shoutout appreciate it I hope that it's a it's good content for you Doug top and good example I learned a few things from watching your dev workflows thanks will be nice to have similar session and use AWS cdk instead of Sam since their support for step functions there says Christian Paul zero zero Christian Paul zero zero boy do I have some good news for you because the first week of May I'm going to be joined right here on this very channel with Richard Boyd who joined me last week for the announcement and Richard Boyd is a senior developer advocate for developer tools at AWS and the title of that session is going to be step functions workflows with the cdk so I know nothing about the CD the CD K for me is like Python it's a thing I know it exists I've seen it I recognize it I know nothing about it Richard's gonna join me and he's gonna take everybody through that through building step functions with the CD k so first Tuesday in May I believe I think it's May 5th definitely make sure you follow us and come back for that one Boyd is a rad fella he is indeed a rad fella he's my favorite co-host on Twitch looking at you Richard Boyd I highly recommend never doing resources star in iam policies demon 0 12 yes I also highly recommend never doing the resources star in I am policies sometimes there's a little bit of pain to get it right I guarantee you there's more pain if you leave it wide open in the long run so here we are we're down at Sam deploy we're racing against the clock we're modifying some step functions there's an update rollback in progress oh no state machine does not support attribute AR n so it's 255 what do y'all think you think I can get this fixed in time let's see update roll back complete events error outputs resource that's right it does not support that attribute different resources support different attributes unfortunately a RN is not one for the workflow it gives you that when you ask for the reference for it so a little bit of a gotcha there will redeploy we've got six minutes get at all get come in output fix get push and then we'll see either either a does or doesn't y'all I am confident that it will it's a race it's a race mr. bean anybody so please no all right build number 20 lucky number 20 number 20 is the the luckiest number for twitch I hear so ci is launching new virtual machine install salmon runs I mean it will create stack on AWS and it runs test and after that destroy stack am I correct or will it not delete stack so 28 mental thanks for the question that's all on how you set it up so in this case it is creating the stack is the final step that's the actual deployment so that's deploying your resources to your environment there is an instance that is created along the way that's installing Sam CLI that's all happening inside a circle CI or wherever your CI provider is from our perspective we never launch any ec2 instances we never create any VP C's subnet security group rules any of that stuff right unless you explicitly declare those so that's all largely dependent on how you do it the the install the testing all that stuff is gonna happen on your CI CD platform and then that final cloud formation deploy is what brings only the runtime resources into your environment so a good question thanks for that can't we do that simply within AWS why do we need circle CI you absolutely can do that only with in AWS using the AWS code suite AWS code pipeline code build code deploy code commit all of those if you are a a double us first or AWS native organization or bill then I again I highly recommend you check those out let's see what else we got cbk is awesome but Doc's need to be improved feel to aimed at javascript and feel weird when using other languages from daemon 0:12 thanks for the feedback we'll make sure that gets to the team helpful robot can you recommend any good tutorials on Sam oh I'm sure he has a few if you helpful robot if you're a golang developer if you're a gopher there's going serverless workshops io which takes you through all of the setup it is very specific to go so it's gonna like show you how to configure the debugger and all of that stuff but no Python developer ed J geek paging edj geek we have a Python question from helpful robot at J geek hey JJ geek so I in all seriousness I I'm lady with edge a geek on that one and I would also again suggest to you that you check out these sessions with Sam series that he's running he's gonna correct me on this if I'm wrong I believe sessions with Sam is every Thursday at 1:00 p.m. Eastern at J geek every Thursday 10:00 a.m. Pacific time 1:00 a.m. east or 1:00 p.m. Eastern Time I got it right see you're important to me buddy I know when your shows are and also every Tuesday every other Tuesday at 10 a.m. Pacific 1 p.m. Pacific he does happy little api's season 2 very API gateway focused highly recommended so make sure you check out both of those they're both here on the AWS channel on twitch Julia Norwood makes a good point you don't need any CI CD you can do everything from Sam build Sam deploy right on your machine I just wanted you to have an example of how you can take something and fit it into like sort of an enterprise requirement fit it into an enterprise workflow and then of course you can build it out from here for whatever steps you have in there on your own package Sam app for deployment it's three o'clock y'all but look change set created successfully create complete look at that just in time so I want to thank everybody for joining us today I hope you learned a lot again I will chop these videos up based on the topics that we covered basic workflow definition with the toolkit Sam CLI setting up CI CD bringing in your lambda functions and invoking step functions remotely thank you all for coming for sharing your questions thank you to to my colleagues for supporting moderation and you know be sure to follow the AWS channel and follow me on my channel send me your questions on Twitter and I will see you all on Thursday if you come to my channel same time or I'll see you here next Tuesday for more step functions thanks everybody

Channel: Serverless Land

Views: 3,345

Keywords:

Id: X79vSGi3qGU

Length: 61min 3sec (3663 seconds)

Published: Tue Apr 07 2020