GitHub Actions CI/CD with Digital Ocean! [DevOps Office Hours Ep. 09 -- Featuring Donny Roufs]

[Music] [Music] hmm [Music] hello and welcome to another episode of devops office hours here on the devops directive channel my name is sid and this is a series in which we bring on guests and i work with them on a variety of devops and cloud infrastructure related challenges today is a follow-on to a series that i've been doing for a while uh with a guest i'll bring him on now donnie roofs welcome back to the show hey i'm actually you said guess but i'm starting to feel like i'm not a guest anymore co-host okay i think actually yeah how's it going it's going good how about you i'm doing well doing well and yeah so i think this is part five maybe in in this series in which we took a project that you built uh and then we dockerized it uh we uh added docker compose for both a local and deployed environment last episode we worked on a process for migrating the data from your current instance which is running on digitalocean and then pulling that down and then updating it to a new instance and so today what i was thinking we would work on is adding updating the github actions to work with this new containerized based workflow and then potentially deploying onto a new droplet and trying to test that whole restore process out um does that sound good yeah cool definitely so i'll just share out my screen and if you're watching go ahead and let us know in the chat whether you can see us and hear us okay i see there's a couple people viewing so just always good to do a an av check and so that we're not shouting to ourselves into the void um so yeah let us know if you can hear us okay and where you're where you're viewing from and let's see so it actually reminds me of a uh of a uh a job um what's it called oh man we had a discussion at my previous job and this person wanted to explain something and it was like in the very beginning of cover 19 so we were not used to using zoom and i think he talked like 20 minutes straight about a problem oh no like we tried to tell him that that we couldn't hear him but he just didn't realize he was so focused on the brain yeah oh geez it's actually really funny yeah i think every every so often you gotta check and make sure things are things are working okay um cool so i've got the github pulled up um you have some actions already in place uh but i think on a fork they don't actually run um workflows aren't being run on this fork repository let's go ahead and enable them let me just zoom in a click or two as well all the cd yeah uh cool so i'm actually just gonna disable these i think and then if i just comment out this portion i think it'll just turn it off maybe so we got the production one we're not working on the master branch anyways so that should be fine and then staging on branch development let's make a new one we're going to call this uh yeah we'll just call it ci cd my macbook is struggling today i don't know if you can hear the fans in the background but i feel like it's it's really i don't know if i can use the slash in the name or not but i guess we'll find out thank you again uh we're going to trigger just since i'm working on this branch that's fine and then uh we're gonna use the github hosted runner so let me just i'm just gonna pull up the official documentation and then i'll probably cheat and go to like one of my repos where i have some simple simple actions right now it works right that's right uh stupid actions that's a good one hopefully hopefully we don't hopefully the stupidity doesn't bleed into our actions that we write today uh let's just start with hello world make it do something is asking or preferable is asking for the github link sure so this is the link to the github project that we're working on or the fork that i am using of donnie's repo and then this other one is for another video i did i wrote five very uh stupid github actions and so sometimes i use that as a reference to get started okay and i do have github copilot installed so we'll see how how good a job it does uh trying to help us out here so are you liking it so far uh so far it's pretty good i think it did really well on the leak code problems that i tested it with in my video that i released last week um it does a pretty good job of just like adding funnily enough it adds comments that i would i was about to write a comment and it like adds the whole thing like oh wow um in terms of code it saves a little bit of time but it's not producing full fully fledged functions that i just use out of the box right um so we're going to run on ubuntu latest we're going to just echo hello world and let's just commit that and make sure it's working okay so we want to commit three things the changes to staging and production to turn them off and then our new workflow the icd cool and we should see something pop up here in the actions tab there we go uh okay so it failed what's that looks like it's running yeah yeah um and so we got errors for these two because uh they don't have an on i guess that's fine or i could just do on never um i guess as long as i'm not on the two branches that it called out it would be fine as well okay um so our hello step ran hello world great cool um so that's good the the jobs that we're actually going to want to use here we're effectively going to need a few things we're going to want to uh trigger on master for staging trigger on release events for production um we'll need a state a step to determine what our docker image tag will [Music] be and so on staging that's going to be the git hash and then on production it'll be yes yeah it the co-pilot thought we wanted covet hash for both um that'll be the uh release tag then we'll want to build images push images deploy to uh drop up so that's pretty much everything we want to do um for the containers we could either push them to docker hub or to digital oceans registry if we're okay with them being public docker hub is free and easy so maybe that's the easiest thing yeah that's fine for me um cool so for this we'll use docker hub okay um so the first thing that we'll want to do probably is to set up docker um there could be there's likely a github action out there that does that or we could just uh install it let's see action docker build build and push docker images from the official docker organization that's one the nice thing about github actions is that there are tons out there that are pre-built and you can likely find one that you can either use out of the box or you can modify slightly or at least use a starting point for what you need to do um okay let's see uh this looks good we want to do this we don't actually care about kimu which is for virtualizing uh or emulating let's say arm-based architectures while on x86 or vice versa but we don't actually need that i don't think we'll need build x either we will want to log into docker hub so we'll add that step we will want to build and push uh we'll probably need to do something here to update this tag properly because it's just using latest but let's just get it working with latest first and then a spirit okay so for now we'll just keep pushing to the same docker hub registry that i already set up yeah and the seekers are coming from github now right uh these secrets we will populate within the repo itself yeah and so there's there's repo level secrets i think there's environments you can use and then there's organizational level secrets if you have multiple repos within an organization i will probably just put these at the repo level um docker hub uh hello to ismail welcome glad to have you here it's been a little while uh oh let me do that let me log into lastpass so i can get my password uh people keep telling me to use a password manager but i just refused but every time when i forgot my password i'm like why did i not just use a password manager all right oh wait what is this field supposed to be oh email oh i'm already i don't want to sign on i want to sign in no okay oh but here we have interesting so this this by default is just building and pushing a single image we actually have multiple images within our repo so i might actually not use um i might not use that just because we already have a make target that does that so if we can use the official image to get logged in and authenticated and then use our make target to build and push that's probably going to be easier what i'm not sure is whether this is also going to set up docker i think because of the way that they um had it documented it might let's see so if i go here build and push like they don't have anything yeah let's just see what happens i'm gonna add my github username and token off-screen so just entertain everybody while i'm doing that i would just stay out of debt so can you actually run the make files in the github action then yeah that's my plan it's basically our github action is just gonna be like make build make push so these are running on an ubuntu-based system um there's slight differences between my mac os system and that but usually uh it'll work out of the box sometimes when you're using a tool like said there's a different version on mac os than linux or if you need to install something specific you might need to do it a little differently but many commands you can just run directly and so what i'm hoping is that our github action is just going to be like make build make push make deploy effectively and set some environment variables along the way right settings secrets we're gonna add a repository secret we're gonna call it docker hub username okay let's basic palace okay now we're going to add a new repository secret this one's going to be docker hub token oh is that taking a token instead of a a password do i need to create a new token maybe uh account settings like versus just using my default login let's see security yeah here we go that's probably an old one that's fine let's create a new one uh league decks get hub action uh this one i think i do need to do off-screen as well it flashes it on screen once create okay copy and close come back over here add secret okay so now we have these two secrets in our environment and let's just to confirm whether or not we have docker installed let's just add another step that is going to do [Music] name oh we also want to we need to clone the repo is that by default in github actions i think we have to add a step github our top level make file what do we have we have a bunch of stuff but let's just do a build production client which should build both actually because we have build api as a dependency too many files open ci cd ammo uh so i don't know if that's gonna work we'll we'll find out here in a second and let's just do so do dvd just to get some information about where we are and what's in the folder [Music] looks good looks good uh and then i'm gonna uncomment these just so we don't get those errors every single time yeah as long as we don't push to those branches it's okay and actually be fine as well because i don't have credentials to your droplet so it wouldn't actually do anything it would just fail add that add that we've got our changes there um oh right before i went through and added all these phony uh tags just because that's what you're supposed to do to indicate that it's not referring to a file so i may as well commit that and that and then we don't need to make file in the docker okay add docker hub off to action i like how you used to commit stuff like i use the interface for committing but for pushing you're actually still using a terminal yeah i guess i don't know why i do that like the i like being able to go through the different files and see the diff in the in the uh window but for some reason i always do get pushed from from the command line uh ismail says that this turned into a 10 episode length of work yeah i mean it's been i don't know what is this five or six but i think the the key is that i wanted to show everything on stream and not go off and do a bunch behind the scenes and then come back and and magically everything works and so because of that it just ends up taking a little while but uh hopefully it's been educational for folks sure has been for me but that's good that's great that is great uh so we're here go back to actions okay we got a red that probably just means that make uh what hello let me close all these huh we don't even have hello anymore do we got rid of hello nope oh i named my job hello okay let's there we go uh this is going to be ci good catch cd um but where'd we fail here if we go into the job we successfully logged in we are in the present working directory uh that and then no rule make target um cool i think that just means we didn't check out our code right so that makes a lot of sense i think one of these we probably have a checkout yeah so you don't actually need any arguments you just add that step up front like so then you commit and i think there's a way to configure these actions to run locally whenever i work on ci cd stuff it's always a uh a pain to like keep pushing commits and triggering remotely um i should figure that out but uh for now we're just gonna do it this way and hopefully it's simple enough that we don't have a a string of 50 commits that are all broken yeah i feel that though but luckily this is all in a fork so when you merge it back into the upstream we can just squash all that uh oh we're over here actions looks like it's running the build command all right yeah we got our code we log in we are building um okay cool we did not set any tags so let that build in the background that should be fine so i ended up doing something similar in this project so let's go learn from myself a year ago okay so that's useful okay so here i guess i what did i do in terms of maybe i only had a prod and staging and i didn't actually use the commit hash let's look at the make file local tag oh i'm using the get shot so it looks like here i just use the github sha no matter what versus tagging with the um with the release tag which which would also work it is nice though in production to have a specific number that you can recognize versus having some shot that you have to go look up um okay uh but it looks like we successfully built our images uh let's we should be able to uh let's just then also push them make sure that's working as well while we then and i guess if it's just one line we can put it up here and what does that make target called push images uh and i'm actually going to change this just for consistency's sake then we're going to go back here we're going to have that way we can build them independently if we want add that add that okay so we're not going to do it exactly the same as i did in this project but i did like that step where we set the this environment variable so we're going to do that then what how do i remember this guys is just setting the same environment variable right uh yeah so we'll we'll set instead of environment we'll probably just set something like uh container tag right and then so if i go here or if i can close some other windows and make my computer run better yeah i had that issue before i upgraded mine i mean it it started to really annoy me i've got like a 2018 macbook pro with 32 gigs of ram it should should be okay but uh something about streaming just brings it to its knees um oh we got a bunch of comments so let me run through those uh in the meantime ismail says educational for sure folks interested in learning ci cd a lot to learn there definitely is there's also so many different tools so this is just highlighting one specific tool um docker learn by doing yeah i'm just i'm just trying to figure it out as i go and then hey eddie welcome welcome to the stream glad to have you here eddie has another youtube channel i've worked with him a few times on uh some kubernetes related things so definitely check out eddie's channel i think i've seen one of those yeah we did one on my channel i think you popped in at one point um so we were copying this one okay uh so if we are on master branch we want to do that we'll set staging i also remember oh there was something about how i was doing it initially that was insecure and github actually came out with a an announcement about it so i think i went back and fixed it yeah so there was this set end command um but apparently it leaked some information that github decided was insecure because you could access it from another action i think um right and so i went back and updated it what did i actually do yeah so previously i was using this then i went with this okay but we also want to then echo docker tag and this one is where we'll want to use the commit hash okay so i guess i can just do this uh oh github i don't need to be a bend how do i reference that later so i guess i just referenced that directly within the make file as the env environment variable let's just look up what github does [Music] uh hello to kyle in the chat glad to have you here by seeing all this yaml today i run a pipeline with iac step for the first time iac is infrastructure's code oh he's already answered in the chat um yeah yaml you use a lot of yaml when you're configuring things that's for sure okay so github sets default environment variables you can also set custom environment variables yeah so we've got so i think this is just the syntax for we're just writing it we're appending it to the github and file essentially and so later it will be available to us so here in production we want our tag to be the actual release tag number and so that should just be the tag itself yeah so github ref would give us i think this whole this whole string and we want just the number presumably right so that's why they're having to do something additional and then do we also need braces here probably here's a whole custom action for just extracting the tag it seems like you probably don't don't need a whole action just to do this but don't you want line 32 and 25 to be the same though oh uh my misunderstanding so the goal here is that if we're on the master branch we want it to be staging and we want the tag of the image to be the commit hash if we're on right if we're tagging and release then that is a production workflow and so we're actually going to tag with like v 1.1.0 or whatever um we could just tag everything with the hash and that would be fine but it's slightly nicer i think to do to do that yeah definitely i just didn't realize the if statement for some reason yeah yeah so let's what do we actually call the tag here uh just tag but let's call it docker tag and so because we have that question mark it'll get set if there is already something in the environment it'll be fine it'll override it oh it's not github you should be able to press f2 by the way if you select tag so if you undo it but you just got back left yeah yeah and then select all of them remove the docker prefix and then select the tag and then press f2 nothing happens maybe you don't need to select it but you can also need to be over it though because f2 stands for re-focus on piece code yeah i think i think vs code doesn't know how to do that with make files i think it does well with other languages for some reason oh that's interesting or maybe i'm just doing it could be user error but uh we're just gonna do that and we're going to search for tag skip that one skip that one skip that one and place that one okay so it's just those two so that is fine and so what we're hoping happens is we actually are not on master so we're going to do this we also should probably update this this is not really that safe to have the else condition default to prod we probably want to have a condition that is matching specifically on what we want prod to mean so like is it a release with a particular that matches a particular regex um or something like that i think this was me being lazy before um on that other project when i was trying to squeeze everything into one hour mm-hmm but we can't just have ice down it's just like the end of the if else in bash you have to add a i'm finished um oh i see okay so let's give that a shot and see if we end up building images with a github sha attached i think we've done everything we need to there uh ismail asks a very good question is there a way to a way or tool to debug these yaml files so there are tools um to lint them to like make sure that the syntax is correct um there i believe is a way to run github actions locally but i've never bothered to i've never bothered to get that set up this is always a challenge with these ci systems is your configuration locally is often slightly different than the configuration uh remotely and so it can be very difficult and slow to debug if every time you have to make a change push it wait for it to happen then check the logs some of these systems do allow you to like attach an ssh system so i know circleci does this where if you have a failed run you can retry it and tell it to start a a special step that allows you to connect to that executor via ssh and then you can poke around and look at things um a little more fluidly because you can have a shell session in the the ci machine i don't know if github actions has an equivalent or not but yeah it's working on these is always always seems to be a bit of a slog okay see how we're doing here add tag says it did it what did it actually do uh run so it doesn't tell us what the tag was but we'll find out soon i guess can i not scroll in those logs i guess like what i can do here is uh echo shouldn't you be able to see it when you push them though yeah yeah no it will um but right just for deepening purposes i guess yeah i thought i could scroll within this but it's not letting me alexander has a suggestion of using teammate which gives you an ssh terminal so that's similar to what i was describing on circleci where you can have a ssh session inside of the the workflow as it's running and so then you can if something goes wrong you can look at oh is that file not in the right place is the version of the tool that i'm using the wrong one etc etc okay so it looks like it did not work because this is still our default tag so why is that mm-hmm docker tag docker tag true it looks like what i'm doing right github echo and then redirect into github end and then echo it out in the next he says i think teammate would be useful for you right now it might be it might be let's just look at what that is yeah let's do it good suggestion from the audience i need to do anything else get the connection string up the checks how does it authenticate i wonder oh let's add this timeout because we probably don't need more than i don't know five minutes or so it gives you an ssh key when it executes okay um but is that an issue for a public repo like will everybody be able to grab it or we don't we don't care about that just run it he says he or she says i think it's a he um well i guess we'll find out i guess we'll find out but i assume you're supposed to put this like on the very top um it probably doesn't matter yeah i mean it would give us access sooner so that's a good idea uh we don't want to make it public because then if it is public then anyone can go into that action and for example grab the github uh the docker hub repo token potentially depending how well it protects sensitive things um interesting one thing i could do is make the repo private actually i don't know if i can because the fork of a public repo um i've actually no idea i don't think you can not sure anyways maybe i won't add it for now we'll keep we'll keep doing things the old-fashioned way i'll do some testing with teammate off-screen uh off uh off stream and then figure out what the limitations are there but where were we we to do this it didn't seem to actually do anything because we got the same uh values as before github ref uh let's just do echo github ref um is there any other information we care about let's also get sha just make sure we're doing that properly then we don't actually care about running these so we'll just comment them out for now you still need to add the other changes i think uh oh what did i forget to add you you commented them out when you uh after you committed that's okay well added the two stages yeah we'll we'll get that on the next commit just burning uh burning my github actions free minutes oh what does that like probably a syntax error oh uh oh i can't just run that i have to echo that oh there we go all right so we are on oh interesting because i have a slash in my branch name um this syntax is finding everything after the the final slash so if i want to actually get a match here i need to remove it from this if statement because i'm uh i'm comparing against this which is just dockerize app once i've stripped off uh everything before it it's like it'll be refs blah blah blah ref slash branch slash feature slash dockerize app and it's just grabbing dockerize app so that should actually fix that um we also didn't oh we didn't set either of those because we didn't match a condition okay so let's run that okay we're getting closer here getting closer that's for sure so hopefully we now match our first condition and set our set our tag or our environment variable accordingly weird don't actually need those quotes docker eyes app hmm yeah that should work all right i guess that's because of doing devops yeah yeah i'll squash them uh alexander's asking what am i using make for i just use make to store off a bunch of little commands that i want to run so things like my docker commands mostly here um i could just use shell scripts um there's also some other tools oh is that not the issue by the way what we didn't commit to makeful oh you are correct you are correct um i think it it could be the issue why we didn't get these echoes out um down there i'm a little confused about but let's let's see unless this github n gets read in at the beginning of the step and so they're not actually set yet could be the case but that is a very good catch on your part but we should see here whether or not we hit that first uh first condition okay we did hit the first condition that's a good sign yeah um i think that may be the case that it actually reads in that github end at the beginning of the step so it's not actually set as an environment variable at the beginning of this within this step but then in future steps it will be hopefully right um only one way to find out only one way to find out he says oh now i have to uncomment the other the other things um let's get rid of that um yeah but back to the question i always end up just defaulting to make because it's available on pretty much most unix and linux based systems and most of the teams that i work with are using either mac os or some linux variant so it ends up just being very convenient for not having to memorize a bunch of long sets of command flags for various things that you have to do regularly uh and being able to run each of the make targets independently is super convenient versus if you're doing just a shell script it's a little more clunky to be able to run individual functions like you can take the shell file and run source on it and then any of your functions will be available in your shell but then you run into naming conflicts across different files and it just becomes kind of annoying but if there are other alternatives that people would like to propose i don't know if you have one in mind i'm happy to happy to take suggestions there i know that windows users who view my channel get upset with me for always using make files well i mean if your windows it's there's a very high chance you're using wsl anyway that's what i tell them but they don't always like that answer then you should like i i even like i'm still learning c sharp on the side and everything like i'm doing everything in w asl even though it's c sharp so yeah no i think i think something like i don't know if it's 90 plus or 80 plus i think probably 90 plus of all the servers on the internet are linux based so i think it it's certainly the way to go and most mostly if you're using containers it can be linux space they do have windows based containers but i've never had a need to to explore them alexander says he he fits the stereotype uh not mad just ignorant of what it is yeah it's just a it's a actually a very old uh new tool that has been around for quite some time an old new tool so the only thing that i dislike about wsl is the fact that you sometimes have issues with files so like that it doesn't get permission or something like it it doesn't really happen but the moment you start using somebody else's extensions on vs code or something then then it becomes an issue um also if you're going to rely on you can access the application either file system right from one or the other but if you do if you're working with files in the other file system it's much slower i think is that true like if you're trying to access the windows files from the linux side yeah so you're supposed to have everything on the linux uh site makes sense in the very beginning when i used wsl i had everything on windows and i was like why the alice is so slow it took me like a month to realize so hopefully we get some new images here with uh a commit hash tag soon so i think they released uh wslg and the latest windows update which allows you to um run uh guise so you don't need a uh what is it called like the uh the window silver kind of thing like uh x11 forwarding thing yeah exactly yeah i remember it no no no it becomes actually native in college they had a system set up where you could essentially remote into a bunch of linux servers and then everyone had window forwarding set up so you could actually have a gui from your whatever laptop or computer into one of these linux boxes that was i don't know that was over 10 years ago now so i'm sure it's come a long way actually funny is that my loss today oh no the studio before that one we built a server basically from scratch and then we we made it in a way that everyone had their own linux environment so if you when you were at school you could just log into your own environment and then you would have your own virtual linux environment which was running on the server but you could access it from your laptop so everyone that had a really shitty laptop they could still do everything they needed to do because the server would handle it for them oh nice yeah yeah it was really a great experience to do though but it's not my thing uh so i think it appears to have worked so if we go into here we now have the 66e5 tag uh which is the commit hash that's great uh although i did notice something within this push within push it's actually building as well so i think we probably have a a make file thing going on so within push we have build api but we don't actually want that because we're going to build it separately i think that was just being lazy and wanting it to do it all with one command before that's fine um so let's actually just test our let's test the tag based uh branch of that conditional as well just to make sure we get that working otherwise we'll have everything working on staging and then it'll break for production um the wrong way around over here and then we want to match for now we'll keep it the lazy way um and just fall back to production as long as we only trigger on the right things then we should be okay um so we could either use a tag trigger or a release trigger let's just use the release trigger on created mm-hmm okay oh in order for this to get picked up though it probably needs to be on the master branch is that true like it's not gonna use my feature branch as the source of truth for when to trigger on a release so that makes sense yeah yeah so that i was trying to connect with those that one's a little tougher to test i guess because until we have merged that back into master the this workflow won't get triggered so maybe we'll put that off for now okay so now we've done that we've done that we've done that now we need to get the deploy step going so we'll do [Music] name to droplet hello my dog like she's waiting for someone to come home and like everything she hears he starts to bark uh yeah alex says using powershell for netapp um yeah so if you're if you're in the windows ecosystem then powershell uh should work great for that type of thing um comes to release you can try conditional automated release execution i'm not even sure what that is what do you mean by that conditional automated release execution if i had to break it down uh i yeah i haven't having trouble even parsing that sentence okay but we were going to avoid droplet i think i destroyed the droplet that i made last time let's let's make it again or make a new one again digital ocean careers that's not what we want good old droplets i think my digital ocean credits are about to expire oh engineering team says it's an issue yeah 50 bucks i to spend by the end of the month let's get a fancy uh oh maybe i just left it running okay i think i tore down the app but i left it running low cpu usage oh yeah last time ali was in the chat spamming our api with a with a infinite loop um cool so let's do we'll need to do we'll probably just create an ssh key as a github secret and then we'll have two of these droplets right so we'll need to pick which one we want to use and so i'm gonna do that here as well i think we're gonna do [Music] um droplet ip and this will be secrets dot so i'll store both of the two ips that we have as uh aging ip and then here secrets production droplet ip this staging droplet ip uh we can add that as a secret it's not actually secret but we'll put it there just for fun i guess it's sort of a secret if we're if we're proxying requests through cloudflare then nobody actually needs to know the ip of our virtual machine so it's less likely to get ddosed i guess if if it is secret so it probably makes sense to put it in in a secret uh secrets new repository secret we're calling it staging drop split ip add secret let's see if we can still connect presumably we can great um and so to actually deploy we essentially just want to run our docker compose command um we could specify there's there's a couple things we could do we could either copy our docker compose file onto the droplet as like the first step here i think we could also set up a docker remote host with the machine and then we don't actually even need to move any of the files look at that that's exactly the article we wanted how to deploy on remote docker hosts with docker compose first time i'm hearing about this actually yeah so here it's saying we can use what i just described we could either copy the the docker file over and then run it and that would work fine or we can use docker host as this huh interesting uh and so we could actually just set that up instead of doing this we can do docker host and then it'll be this droplet ip will get used instead of remote host get rid of that hopefully those slashes do okay and don't get something funny happened to him with the shell escaping but we'll see true uh let's just confirm that that docker host comes out in a format that looks reasonable and then we can actually probably do most of the development here locally with just the makefile and setting that docker host and then once we get it working we can just run uh make deploy uh okay oh what did i change about the makefile uh oh i took away that dependency so it doesn't build rebuild every time we added the release tag that's fine and then added deploy set our docker host environment variable okay that all looks fine and then we could use these context things but i think because we're going to automate the setting of that docker host anyways it's probably fine just to just use that back to actions ah what did we what did we not like broken pipe we're during oh doctor okay so we so because we set docker host here i think in the build step it's trying to use docker running on that machine right uh so interesting we can i guess we can just set it down here [Music] or we could just do docker deploy host docker cannot edit i need to need a real editor deploy okay so we're gonna avoid that naming conflict by doing this and then later when we actually pass it to our make deploy we'll do something like docker host equals docker deploy host make deploy okay so that should fix that issue uh we're also going to just to speed things up we're going to skip our build oh forgot to click the plus okay um okay so because it's a secret it's hiding it from us but i think that looks like we want oh except it should be root not uh great so now that we have that we want make run production let's just hard code that for now i keep doing that um docker host equals ssh root our ip address go grab it okay uh oh alexander says he submitted a pr let's go check it out live live assistance oh i see i see so basically automating the creation of the release process uh depending if you push to the correct branch bump the tag using the commit message from to determine whether it's a major or minor or yeah something like that could be cool uh for now i think probably just keeping it simple and doing manual releases um but then triggering on that manual release event is probably what we'll we'll do just to keep things simple but i think that that is a cool approach for sure back to here so we got our docker host we got docker compose our file okay nothing is running um cool so i think if we just do let's set our tag equal to like one of these tags from our action that we just produced so it looks like uh changing that environment variable name allowed us to avoid our collision this is our tag let's just set our tag equal to that here actually in our docker compose file that's one thing we'll need to fix is right now we just have these hard-coded tags um so let's just try running make what did i even call it make deploy make run production i think so you can make run production oh but i'm on there exit make run production successfully use my ssh key connection refused huh but if i just do ssh to that box root at i'm a little bit confused those are the same command aren't they root maybe there's just something weird i think grand production okay ssh okay that's fine does it not like something about our root uh what i'm not seeing it bob is there a space after no so that doesn't work but that does no that doesn't uh those look identical to me yeah is it blocking i'm using port 22 for both of those connections the weird thing is that after i run after i run the docker version it seems to get blocked and then i run it just with ssh and it gets blocked and then i wait a little while and then it seems to work docker host docker compose it does keep warning us that docker compose is now its own thing and doesn't require the dash anymore aha gives us a better warning or a better error make sure the url is valid uh what docker version are we on okay 1809 i'm on 20.1 that's fine uh what do we have on the remote host hopefully it's public key uh 1903 that should be fine right it said after 19. yeah so it seems like maybe something with our firewall settings are weird but the fact that we can ssh in normally on port 22. let's just confirm that all ports are blocked except for those okay that that's what we want uh oh is doc should we be connecting over one of those ports see if our uh trusty digitalocean tutorial gives us any hints [Music] um ssh user at ip no dice docker this is always the best part of streams is uh googling oh we'll point at the stairs right let's just look more closely at the uh error message that we get failed to wait signal killed when it tries to get the postgres image it doesn't like that do we have something funny in our docker compose no we're just specifying the image so it should pull that image okay that works fine now if we do [Music] what is the dash l flag ssh shell oh the login name okay so we're logging in as root we've got our ip if we get rid of okay that command works if anyone in the chat sees our our issue please let us know but this time it failed at a different place when it tries to execute that get command it's when it's trying to pull pull our images so always certain that you're putting the right image down um no that's a good good question so that actually seems like it worked let's see if i do docker version yeah so this is the docker this is definitely the docker version running on the remote host uh if i do docker run postgres um that will run locally and then what i don't know is whether when you use a remote host like this if we will need to have like the environment files if it will look for them on the remote host or on this or on the local host probably on the remote host docker run do dash d for it to run in the background okay so that's running locally and then if i do that we get none because it's running remotely so that's good let's just try to pull those images remotely is docker pull not the right command docker docker pulls the thing right uh sounds good to me manifest unknown and that's just doing locally maybe that tag doesn't actually exist yeah that's what i'm thinking too huh well we'll use this tag oh well yeah that's gonna be it for sure yeah i'm not sure where where i got that from then the okay so that one should at least exist and if i do docker pull okay so that works locally and does it work remotely it does okay verifying checksum uh okay so maybe let's just try again with make run production the weird thing is that it's giving us the port 22 issue seems like it wouldn't give us that i guess let's why don't we try copying this file over like we did last time make sure the docker compose itself is working okay and then we can debug this docker host thing if we had to we could even go this route for the actual deployment but hopefully we don't have to so we've got so we want to copy our rocker hose dot animal docker compose production oh we got to change use it to rotate yeah user root and then this is i don't actually care about recursive because just one file then we're going to ssh in actually let's copy those environment variables too i guess so it is server dot and uh it doesn't server should exist [Music] maybe by default it doesn't like uh no well we can just create it on the server if we need to oh not that one you want [Music] ssh do you remember how i said i hate devops those are the moments like i like everything about it but the moment you need to debug stuff that's just nightmare did oh maybe because it's running the older version of docker compose we can't actually do like they upgraded that in version 20 i think okay so we'll do touch root server and what can't touch actor server touch and then we copy these guys over once again that is a defunct riot api key so that is okay yeah and even then it doesn't matter it's a development key then for prisma we also have a secret here eventually we're going to need to set these via the command line that's what we want okay so that all seems to work okay we just have to get this remote host thing working and then handle our various secrets i wonder uh i think last time you set up this dns record so maybe it'll just work uh well we got some error but cannot start [Music] oh we did we did have to do some we didn't actually bring our nginx config or anything over okay um but the docker compose portion worked fine so why does it not like why does it not like us we're going to keep the old syntax because the remote version doesn't have that yet [Music] um all right well this might be a dead end for today um yeah we're coming up i've also been googling but i yeah i'm coming up on the two hour mark um we did make some good progress i've actually learned to make it like at least two hours so i don't have to go like after one after a little bit longer yeah yeah well we let's see what did we get done we successfully set up our ci cd to build and tag and push our docker images we have a staging and a production setup that's using our github sha for staging and a tag string for production we're pushing those to docker hub and then we got stuck on using the docker remote host to actually do our deployment last time we were able to copy our files over and run them run the commands manually on the droplet we need to do that from our github action or from our local system and so probably i'll do a little bit of debugging off screen and see why docker host isn't working but i think other than that we're pretty much ready to i guess the the one final thing is setting all of the necessary environment variables um the secrets yeah setting actual secret values dot and um then we'll need to log into the actual droplet we'll need to dump the database into a backup file i think we'll need to manually tweak it slightly because you have two databases in one database cluster um but it should be pretty minimal and then we'll we'll pretty much be ready to go yeah very close we're we're getting there we're getting there let's see i got a couple more comments here before we close out um alexander says he closed the pr but it's there if we if we want to check it out cool thanks uh appreciate the the input um uh perez is asking about some interview stuff uh ace the technical rounds rejected me hitting the projects are very average should i remove devops mini projects i mean i have all kinds of stuff on my github i don't think it hurts you to have small projects i think you just want to highlight uh any that are significant um so if you have a bigger one that you want to focus on make that show up in your your github profile itself is the advice i would give there any advice donnie as you go through the interview process yourself um not really just be you yeah all right well thanks for tuning in we made some good progress and i will catch you in the next one take care

Channel: DevOps Directive

Views: 1,010

Keywords:

Id: l54qzIIegpk

Length: 119min 5sec (7145 seconds)

Published: Fri Jul 09 2021