Advanced GitHub Actions: workflows for production grade CI/CD - GitHub Universe 2019

[Music] hi folks my name is ed this is my colleague kala today we want to talk a little bit about github actions I hope that you've heard a lot already between yesterday's keynote with Jeremy the session between that the Jeremy did after that and of course today's keynote as well so I think that I hope at least that you've seen a lot about actions so far but we want to take it just a little bit deeper we want to show you a few more advanced things a few more advanced scenarios that you can do with github actions real quick introduction my name is Edward Thompson I'm a product manager for github actions so I'm part of the team that's that's been working on this really exciting product so I wish the whole team could be here sadly it's just me and Kayla for now but we'll have some more people from the team available after this if you want to hang out and talk and get your get your questions answered and if you don't if you're if you're feeling a little introverted or a little shy or you're overwhelmed with the number of things that you can do at universe that's totally cool feel free to hit me up you can find me on twitter i'm e Thompson it's Thompson without a P ETH OMS o n happy to answer any questions that you have and I'm Kayla I'm also a product manager on github actions and like Ed said you can find me after the talk at the hack space down though so diving right in I'm gonna start by covering a few concepts before we go deeper so I'll talk about triggers matrices and caching so here I have a node app it's for a basic calculator and let's just set up CI for it so I'll go to the actions tab to get started and github actions suggest a couple of different templates for me to use sees that I have a note app so let's just use that and template taking a look at this template we can see that it's going to trigger on every push it'll run on Ubuntu latest and then it's actually going to create a matrix of jobs for three different versions of node for the steps it'll check out the repository it'll set up that version of node and then it'll do the build and test so right now it's configured to run on every push and that may not be something that I want I might not want it to run on every push every feature branch for example so I can actually set up triggers to be more specific so I'm going to actually update this trigger to run on pushes two particular branches specifically master and release branches and here I can actually use wildcards so I don't have to specify a bunch of branches I can say anything that matches release slash I can also set this up for pull request validation so I'll add a second trigger and I will run this on pull requests into master great so right now it's running a matrix across three versions of node I'm actually going to update this and run a matrix across the three operating systems so let's replace this with the three os's we've got Ubuntu latest Windows latest and Mac OS latest and then I'll update the runs on above to use the matrix OS variable and then we'll also want to replace the node version variable that we had using we're using down at the bottom and let's just use twelve points for now so one other thing that I can add to my workflow that'll help speed it up in some cases is caching so I'm going to go over to the market place here and look for the caching action so I see here is the cash action I can go view the marketplace listing and I can go get some more information about it I can see you know who's contributed to it the number of stars it has and I can view some documentation so I can see the inputs it requires the outputs an example workflow and here at the bottom there are some other examples so let's click that and we can see that there's actually a node NPM example so at the top here there's a snippet for Mac OS and Ubuntu you can see it takes the path then it has a key it hashes the file so let's copy that snippet into our workflow now if you remember this was just for Mac OS and Linux so we'd want this step to be conditional if the runner OS is not Windows I could add another step here to do something similar for Windows but that's actually kind of clunky so you might have seen that below there's a snippet for multiple systems so let's grab that and stick that into our workflow all right so you can see what it's doing here it's actually setting the cache outputs directory and then using that in the next step for the path so let's actually go ahead and commit this we can create a new branch and then we will create a PR so you can see here in the bottom the jobs are popping in let me see all three OSS and then we can go to the checks tab and view the live logs so it sets up the job it pulls in the actions that it's going to use it checks out the repo sets up the version of node that we specified and then does the cache steps that we had and then finally it does the build and test so if we dig into these steps here we can see that it set the cache directory and then as expected in the cache step and there's actually a cache that it's not found this is the first time we ran it and then the end it actually saves the cache successfully for future runs so that's the first example pretty quick overview the next one I want to show is how you can integrate with github packages so again we have the basic calculator app we'll go to the actions tab again this time we will look for a docker template it's a scrolling down and there's the docker image on the right okay so this is actually a very basic workflow it's triggering on every push runs on Ubuntu latest checks out the repo and builds the docker image so let's actually update this to also publish to get home packages so let's update the name here and we will update the build step as well so we have the name of my package here my username the repo app and then to publish to github packages we'll need to log in so let's create a multi-line script here and we will log in so docker login and then provide my username and then for the password actually because we're using actions and packages together I can get the github token in the secrets context so it's super easy to integrate the two and lastly let's just push this so docker push add the image that we create in that first step there and then to make sure that those versions match we're actually going to create a version variable and we'll reuse that great so let's go ahead and commit that create another branch create the pull request and again we'll see the job pop in and like before we can go to the checks tab and view the live logs [Music] so it set up the job checked out the repo building the image publishing a ticket a packages so if we skip ahead we can actually go over to the packages for this repository and we can see that it was successfully packaged or published so now I'm gonna actually hand it off to Ed and he's gonna go into some more examples cool thanks Kayla thanks so much I'm really excited about the integration that github Actions has with other parts of the github ecosystem like github packages another part of the github ecosystem that I think is incredibly important as I think you saw on today's keynote is the security aspects of github I think that we have a real responsibility at github to make sure that your code stays secure and stays safe and so one of the ways that you can do that is by integrating github actions with the new automated security fixes that you saw earlier today so I'm actually gonna use the same repository that kayla was working on this sort of node J s calculator application and this is one that I've had for a little while I've been using this repo for a while and sort of organically I wish you know I had faked this but organically over time I've had security issues in this repository I've had vulnerabilities crop up in my dependencies so if we actually go up and take a look at my pull requests and go to the closed tab and then navigate to this one there we go so you can see that this is actually a a pull request that was opened by our automated security review process what we used to call dependable if we scroll down and open up this little commits we can actually see exactly all the things that it that it did it upgraded Milo - dependency actually several times conversion 4.17 not 11 to 4.17 not 15 something a little safer something that didn't have all of the vulnerabilities right so this is actually emerged pull request what I want to do is go to a different repository one that hasn't had this merged yet and we can see what our security alerts actually look like you can see this this yellow bar in the middle of my repository now this is something only I can see as a repository owner it's not that anybody can navigate to my repository and see that there's a security vulnerability but it gives me a lot of visibility into the problem so I can go I can look at that and I can come over to the to the right hand side and click view Security Alert and that will actually show me the information I can drill down a little deeper and it will actually tell me exactly what's going on I can click one more time and actually get to the pull request so unlike my previous repository this pull request is still open and I have github actions set up to do continuous integration builds on this repository so we've detected this security vulnerability we've opened a pull request that fixes it automatically upgrading to the latest version of lodash and we've run all of these CI checks so we know that the that everything builds and that all my tests pass and now in a perfect world what I would do is is just click merge pull request and I have a high degree of confidence because I've got good test coverage to know that everything's safe but in a more perfect world what I might actually be able to get is github to do this for me it's gone so far as to create this pull request what if we could actually just merge it and in fact you can let's take a look at what that looks like so I've got yet another repository and at this point I don't have automated security setup I don't have to pend a bot set up in this repository what I do want to do though is sorry let me pop back to that there we go and now I want to navigate to my workflows there we go and open up my workflow and this is right now just the default nodejs workflow that you've seen now a couple of times you've seen it in several demos over the last two days but let's let's change this up a little bit so I can click Edit and what I want to do now is change the trigger from on push to be on pull request and I want to scope it just to a couple of branches we'll just select the master branch and you can see that that that very nice yeah no editing experience because I don't know about you yeah Mel's not my favorite thing to work on having this autocomplete this wizard is a big help ok let's scroll down a little bit and what we want to do is add a new job that will run after our build so we've got that build job but I can add a new one let's call it Auto merge we'll give it a name and now what we want to do is actually add some conditionals we don't want to always run this auto merge we want to be very careful we want to scope it just to things that were opened by dependable and we can actually do that really easily we can look at the github context this is information this is the same sort of thing that you would see in a web hook but we can see the github context in our actions workflow execution so I can select NEADS right here first we're gonna run after the build this is this will run us sequentially instead of in parallel we want to make sure that our build succeeds before we go ahead and do that that merge and then we can say if and we can see all the things that we can do we can do a little bit of advanced execution but what I want to do is look directly at the github context so I can say github dot is master so if it's a going into master and I can say if github not actor equals dependable that's the name of the dependable user so now this particular step will only run on dependable pull requests and then I can say steps will do the merge and there are several options here at this point we could use run and type some - but one of my colleagues Jake LEM came up with something incredibly clever that I'm really excited about it's called github script so we can say uses actions slash github script we'll give it a version and basically what this lets us do is it lets us run JavaScript with the octo kit context and with the github context so basically I'm ready to go in an environment where instead of scripting with bash i'm already authenticated i'm already set up and I can just write JavaScript so I can say with this script and I can say github dot dot merge this is basically octo kit for let's merge a pull request and I'll give us some more information and I can get this out of the github context so we need the owner we need the repo that we're in and finally we need the ID the pull request ID the number of the pull request we want to merge so that's the current one and again we get all this information we get the basically the payload of a web hook so we've got all sorts of information about the the action that happened finally I need to add a github token and that's in our secrets that's github.com and so Kayla used this earlier this is basically a very short-lived token that allows us to work on our repository it's repository scoped when you fork it it's read-only so it's it's very safe very limited amount of time and very limited amount of privileges that this token has but enough that we can in fact merge this pull request so we've got this red squiggle here where so this is another nice thing about our gamal editor it will actually do some linting until you when it's wrong I've missed a required key so let's add runs on here we're gonna run this on just a Linux image there we go and now we can check that in so we'll go ahead and check it right into the master branch there we go and if we go up to the actions tab oh sorry let's go to the security tab first and let's do that again there we go we'll turn on automated security updates this turns on depend amat so now when I do a push or when I actually just by turning that on that'll that'll start the security scan so there we go now I've got a security update I can click on that and what it's doing now is it's in the background doing a creating a pull request to do that fix so it's analyzing the problem it's gonna try applying the newest version of lodash then the next four newest version until we get to the absolute latest version trying to make sure that they all actually work with my project and then it'll open up the pull request and github actions will actually do the test to make sure that it works with my project so there we go and we can look at this pull request let's dig in here and if we here's our pull request so if we scroll down we can look into the checks area and we can see that our CI is running actually our CI finished so let's go look at show all checks over here and oh oh oh it happened really quickly our pull request actually already got merged which is actually exactly what we wanted to happen we wanted our CI to run our tests to pass and then github actions to merge the pull request for us let's take a look at what actually happened let's look at our actions run its just finishing up its that top one right there let's click into that and so yeah so we had a matrix build on NPM eight ten and twelve on no date ten and twelve and then we did this last up the merge pull request step let's dig into that and open up the merge job there we go and that's just going to show us this github action script that we ran so very straightforward only a couple lines of code to get us from the point at which depend upon is opening pull requests for us to us validating those pull requests by running CI tests to finally just saying well if we have good test coverage we can trust dependable to do a good job and merge those pull requests so not something you want to do until you do have that good test coverage but once you're there I think that this is a great way to just automate security from end to end ok one of the other things that I'm really excited about in in github actions is the ability to use containers as part of the build so with github actions we give you virtual machines that you can take advantage of we give you Windows Linux and Mac OS VMs and we and we've got a ton of software installed on them but it's honestly not always enough you might need some very bespoke sort of tool as part of your build you may have some dependency that has a very special installation requirement or it's very large that you don't you know want to download and unpacked every single time so by using containers you can scope exactly what you want to your CI system you don't have to rely on the things that we give you so instead of my node app that is a very simple calculator I've got an even more simple node AB here this is this is something that we'll talk to a Postgres database it doesn't do anything really advanced I'm a I'm a product manager not an engineer so let's take a look at what it does do though I'm gonna open that up and you can see here up top we're gonna basically just connect to a Postgres database we're going to read out of the environment the database to connect to the host and the port and then just gonna connect and once we're connected we're gonna do a really awesome query we're going to select the now function yeah it'll give us the current time like I said I'm not an engineer here's the problem with this though we don't actually have Postgres installed on our runners that we give you so we've got all these these virtual machines but not a one of them has Postgres on it the nice thing though is that we can solve that by actually just using containers so one of the things that we can do is we can either allow github actions to run a container by specifying that in the build steps and so when you say run that run context will actually be executed inside a container or we can bring what we call service containers to the party and that's a this is a great use of that what I want to do is use a Postgres database as part of my test but I don't really need it for anything else so I can just actually spin up a a Postgres container just exactly the ones that I'd get off docker hub and use it for my tests so let's take a look so like I said here's the list of software that's installed on the github hosted runners I can scroll through here and scroll and scratch there's a lot of stuff look at all those haskell versions but if i if i search what I don't see is Postgres right so I've got all this stuff i've got dotnet core SDKs i've got several different versions of just 2.2 but yeah not a single instance of Postgres so let's fix that so here's what I want to do I want to go to my actions tab and I want to set up a new workflow and I'm not gonna take one of these out-of-the-box ones I'm going to set up a blank workflow I'm gonna check the set up a workflow yourself there we go and we can set up Postgres as a service so I'm gonna navigate here above our steps and I'm going to add services I'm gonna give it the name Postgres and this is arbitrary this is this is just a key so I I'm able to use this to refer to it later it's just the name doesn't have to be anything special I could have called it foo but what is special then is the image so I'm gonna specify image Postgres 10.8 and what's gonna happen when i specify this is that it's going to download this container image and it's going to start it inside my execution vm so like I said by default I'm running on a VM I'll have this container there running side by side inside my VM so I can I can take advantage of it if I were if I specified other services they would also be spun up on the VM and they'd be all networked together and finally I could even again use a container to actually do my run steps to run my actions workflows and then that would bring yet another container to the party all networks together so it's very straightforward to to set up actually complex build and test environment that I can that I can use so after that I want to provide some environment variables again we need Postgres Postgres user and we'll set that to Postgres Postgres password we'll also set that to Postgres because again I'm a product manager not a security guy and finally Postgres DB and we'll call that Postgres as well once that's there we're gonna add some options to the container I'm gonna paste this in what this does is it just ensures that the Postgres database becomes healthy before we actually start testing against it and finally we're gonna expose a port will expose port 5 4 3 2 on the container there we go and what that does is it Maps basically a port on the VM you know if you're familiar with docker I this is probably something that you're used to doing we're mapping a port inside the container to the VM we don't actually know what port though this particular syntax lets us we'll just pick an arbitrary unused port on the virtual machine and you might think that that's kind of weird and I concede that it kind of is but we'll take a look at what that how that works in just a second so once that's done let's come down here let's get rid of these dummy steps and actually do some work so what we want to do is run npm CI and then we want to run our little application client j s and we will set up some environment variables again we'll set up Postgres host and Postgres port and these are actually used by our application to to connect to Postgres that's that's actually in the java script that you saw now what port number do we use well what we can actually do is query the the data that we that that github actions has created as part of this workflow so when we set up that port we can actually interrogate it later using that kind of dollar curly curly syntax that you've probably seen it's not just for for the secrets variables or for environment variables it's also for variables that github actions will set that you can take advantage of so in this case I want to say job dot services dot Postgres dot ports five four three two and that's i concede a little bit verbose but what does it actually mean well it's it's hierarchical so it's actually pointing up to our jobs up top so up it will actually map eventually to this ports key right here services Postgres ports and then it'll actually expand out to five four three two so or I'm sorry it'll map to the what port five four three two on the container is mapped to in the VM so it'll actually be something random like 32 five 64 or something okay so let's commit this there we go and let's take a look at our actions run okay and then come down and look at this workflow execution cool so right now what we're doing is the first thing we do is initialize the containers so we're basically downloading that Postgres container image and then we'll start it up we'll map the ports so that we can actually get to port five four three two on the container once the container is actually started again we want to make sure that Postgres starts healthy will run npm CI and then we will run our our client our javascript and that will actually do the connection to Postgres as our tests so let's actually expand that out and see what happens well yep indeed if i look there and at my results i did select now and i got my query back and so that's the the current time so i think this is really great it's super straightforward to set something up that allows us to use dependencies that we don't actually have on our runners but that we might need for our our CI build so we do offer self hosted runners in case you need even more of a spoke things you know I know people who work in very or who work with software that has very strict licensing requirements and actually needs a dongle like plugged into a computer in the year 2019 and if that's true you can use a self hosted runner but if you have less complicated if you will licensing requirements then you can take advantage of docker containers and still run them in our in our runners finally I want to take just one minute to talk about creating your own action we've we've talked a lot over the last two days about creating workflows and running workflow executions but what we haven't talked about is actions and I think that's really important because that's where github actions got its name from actions are small reusable components that you can use within your workflow that you can share within your organization or that you can upload on to the github marketplace and share with others so so there we go so how many people have played untitled goo scam yes it's my favorite new game if you haven't played it yet the the conceit is that there is a lovely village full of lovely people just trying to go about their daily basis and you were a goose you are an angry goose and you sneak up on them and try to be difficult so my one of my friends one of my colleagues here at github has created an action that can that can help with this and so if I come to into an issue and it's a lovely issue and I come in and I want to add a comments and I do that he's created an action that will run on issue comments if we look at the camel we've got a trigger that will run on issues created and when that happens it will run his action it's called Jeff after Punk action if we go into his repository we can look at what makes up a github action it could be one of two things it could be a docker container or it could be a JavaScript application the nice thing about using a JavaScript application is that it's cross-platform docker containers are only gonna run on Linux if you build an action using JavaScript it can run on Windows it can run on Mac OS and it can run on Linux so in this case he's actually written typescript and let's take a look at what it does so we've got some imports this brings in the github actions toolkit this basically allows us to use octo kit and a number of other you know pieces of functionality that allow you to write cross-platform actions very easily the next thing we're going to do is actually inspect that github issue payload and so we'll see what happens basically again this is just like what you would see when you have a web hook execution we can get the issue information then we can get the comment what was actually posted the issue comment that was written that's comment body right here and what we're gonna do is see if the comment body matches honk the word honk is in the comment body if you are in fact an angry goose then we'll return we'll just stop doing anything at all if you didn't say honk in the issue we're going to delete your comment so this is this is an angry unfriendly action right here so again if we go back to our actual oh yes our our comment was deleted two minutes ago by github actions and it has replaced it with a honking goose well that is very unfriendly this obviously is in a very useful action unless you really hate your workers but it's just an example of some of the things that you can do with github actions you can trigger on any github event it's not just about CI CD you can look at issue comments you can look at when releases were created and ultimately you can use this github actions toolkit as a way to to take advantage of that so if you are interested in using pieces of reusable components within your workflows sharing it within your organization or sharing it with the rest of github and you can go to github marketplace to look at all of the actions that are there that people have contributed I think that that's a a great piece of technology to build it allows us to really work together to enable community powered continuous integration and repository functionality great so with that I'm afraid that we are just out of time for questions but we will be in the connect space Kayla and I will be there as well as some other members of the github actions team we'd be happy to answer any questions that you have if you want to come to us with your workflow problems we'd be happy to see if we can solve them and again if you if you have other stuff going on this afternoon that's totally fine feel free to reach out on a Thompson on Twitter and if you see Kayla or I around the event feel free to ask us so thanks so much [Music] you [Music]

Channel: GitHub

Views: 15,131

Rating: 4.9280577 out of 5

Keywords: git, github, github universe, github universe 2019, collaboration, programming, version control, open source, software development, octocat, innersource, github actions, github actions howto, continuous integration, continuous deployment, cicd, ci/cd, ci cd, devops, github actions devops

Id: 0ahRkhrOePo

Length: 40min 12sec (2412 seconds)

Published: Fri Dec 13 2019