Docker, Kubernetes, and PHP: Laravel Edition | Rawkode Live

hello and welcome hello how's it going rick good how are you i'm very well thank you i'm feeling good i'm excited yeah it was fun last time it was we i feel we got quite far um but the requests were unanimous people want to see how we do this with laravel and all the bits that we didn't cover the bits we left to the reader's imagination if you will so are we assuming people saw the previous session yes yes we are assuming that people have seen the previous session although to be honest we are uh it doesn't matter if it doesn't uh we will do our best to keep things explained as we go along and we are starting with a laravel project that has no containerization at all so we are going to be doing a little bit of a repeat but hopefully much faster pace than last time with some of the core steps and how's your larval non-existent wonderful like a few versions ago i did the demo but um it's all sensible stuff isn't it and i think a lot of the things we're going to talk about the general way you containerize things how you build assets how you do various things are going to be applicable across all projects aren't they yeah hopefully this yeah they will definitely well i don't imagine there's going to be much super laravel specific stuff to how we how we can turn around yeah well i mean let's set the context down for the people that are watching uh okay first and foremost i'm going to look away from here so i can click over here um we are taking comments if you want to ask any questions if there's anything that we don't explain very well drop it into the comment section on youtube or twitter we'll keep an eye on it also if you need a nice easy way to find these games in the future you can go to la rocco.live and for anyone that wants to keep the conversation going afterwards there's a slash chat which is a discord server feel free to come and join us there and ask questions when we're not b streaming getting better at this i'll be there now we're going to set some more context about what we're actually going to do so i am just going to pop open the share here and we have my screen so how did i pick the project to work with today simple i went to get up.com topics laravel i filtered on php and most stars and i scrolled down this list until i found a project that had no containerization whatsoever so obviously we're not going to do larval itself because we're looking for a project that has all that um day two stuff migrations asset completion etc um i did consider using this coel project for music streaming it looks really cool but i wanted something that hopefully more people are actually using as a development platform rather than maybe as a tool to consume where you would just use docker files anyway uh there's a few library based projects on projects and then i got the vladimir i'm not sure how to pronounce it but i'm sure i'll change it randomly as i go now this project looks cool it's like a forum uh written a larval and it it checked all my criteria my criteria was no containerization you can see there's no docker compose here there's no docker file here we get to do everything ourselves yeah there's not much there is there no it's got a pretty solid amount of stars you know is pushing on 11k it's got you know over a thousand fourths who has maybe got a vibrant commuter a contributor list there's 20 contributors down here so php it's laravel i mean i think this is going to be a pretty cool project and a fun one to hack on today and how do they actually install it so have you tried this without have you like installed it no no no no no sorry but i have corned it so i just did this a few moments ago you can see it here all right all right we've got the files we're starting from a first a fresh light here but i mean i think i'm hoping this is an idiomatic very laravel steady technical application that we can just go from start to production and cover all those steps that we kind of fluffed around last time so i guess what makes sense first is to just copy what we did last time and that's what just get it running and understand the steps that are that we need to go through to make that happen that's so good yeah and i can see they've got a installation guide where we're going we don't need gates well it's interesting to know what how they say to install stuff don't they all right you go your way i'll just have a quick look and see there's any options in there i'm just going to run composer as well first like let's let's see how far through this we can get you take a look at the gate there that i'll pull up again too let's do it where's the installation going down is it here oh there we go stop flowering.org yeah yeah click that um so they give you installation on apache or nginx you need something you need mysql or mariadb and then specifying a particular version of php that's it ideally so we have our first worrying comment from alex who has said this does not look like a standard laravel application and that the composer jason doesn't even mention larabelle now i didn't notice the lack of larval and here now i'm hoping that that is just encapsulated in platinum core i know that an assumption but i'm going to clarify that now so if you do compose your info in your shell does it have laravel yeah there's loads of laravel stuff in core um so what was that matt how can i confirm that here well composer info yeah but i could still eliminate i'm pretty sure that's the larval thing i can see yeah i'm not saying laravel laravel so your concern is it might be too non-standard i wasn't concerned alex is now maybe concerned yeah within the first six minutes of this dream cheers mate so i guess the question for me is do you do things that a typical laravel project would do for instance to run migrations and okay so if i grab composer nfl for dash is mm-hmm so maybe they're just using a load of laravel bits ah that's frustrating yeah okay now is there enough laravel here for us to continue no illuminate is the database package so we can show migrations right even though they're not laravel specific hopefully that's not going to be too different uh uh right okay right alex human so alex has suggested we take a look at anterior pen crm so we're gonna do a last minute switch what's the worst that can happen uh and area ping crm all right alex we'll do it you have persuaded us i mean it's only got 670 stars what's the license the github license that just means anyone can use it if it's something there's something i can't show on screen yeah that's fine okay and we've got isolation instructions npm composer okay i'm happy with this so we've got some artisan here which i believe is the oim uh power flower pillow so we will we'll do let's go for it so i will clone this artisan is like the task runner so let's pop that open and uh it means i'm going to kill this live share but i'll send you a new link in case we need to do any painting i should automatically copy to my clipboard copy copy copy and we'll get started all right so i need to run composer install again wonderful and let's just uh get a darker composed file started for now and get some of the boilerplate out of the way yep so we're gonna go with version two i covered this in the last session about why i prefer version two over version three and it's really because we want to be able to orchestrate our we want to be able to use health checks to guarantee the order in which things are started i think i i spent five to ten minutes going over why that's important you know as much as we want party between our development environment our production environment is it the workflows with development and production are extremely different so what's the last video for more information on why version 2 is important we will call that service php i will just add a very base php image for now i'm not going to do anything else and i'm assuming we're going to need a database looks like it yeah all right we have another question from pierce so are we taking a laravel app and building docker into it yeah we are we did the session last week where we took a php application we showed how to build a development environment using docker and docker compose but then take that uh automation and use that to build a docker image that can be shipped to kubernetes and production the feedback that we got is that people want to understand how to handle all the the deployment steps that we cannot flush over database migrations asset compilation and a few other pieces running these tests and that's what we're going to do today so we're going to use laravel we're going to get a development environment running with docker dot compose again we'll do that relatively snappy and then we'll start to talk about the kubernetes aspect and how we run those migrations in a production-like environment yep it needs php 7.3 or better okay so we're just running page seven which covers the latest anyway so you know you don't you don't nail it onto a specific one but dave yeah i should do this is if i was doing this for production uh i will okay so what i would normally do is go to here docker hub go to the php image i can set the cookies tags and uh i thought we had a 7.4 but i guess that was wrong yeah there is seven four output but in a general sense do you not do not specify a minor version would would you do you think it's important no uh so i like to take as many uh updates as possible so i don't commit lock files and i don't generally painted minor versions in a docker uh in a docker container the reason why would be that i always wanted to pull the latest agrees and then i want to trust my continuous integration and deployment platforms to detect when something goes wrong and block that pipeline so that i can go and investigate and fix it right which means if i do pen to seven and seven point five came out and there was some sort of regression my ci or my test would pick that up and then i would be able to go and make my code work with that latest version rather than just penance for 7.4 and then 7.7 comes along i've got three releases of things you know that that surface of change it's just much bigger and i'd rather deal with really small incremental continuous updates in large ones but you've got to have the people in there yeah people do try and sort of lock everything to specific versions as a sort of safety net don't they but you're saying like the ci is okay to break the ci every so often um you know contract your continuous integration trust your test you know if you've got enough of uh if you're getting so much house issues there's or said one incident or anything like that because of that approach then your test coverage needs to improve the automation needs to improve you know there's there are symptoms there that can be worked on and it doesn't mean that everyone can take this approach that i have i generally i i think i spend and invest a lot of time in the tooling to make it work and it's a journey so it's a scale of zero to ten and how close you can get to that continuous automatic update okay it's not for everyone but try and get towards that stage i think is really important because then your team can focus on the more fun stuff right let's let's go with i'm just going to 7 0pm after that okay now let's take a look at these installation instructions again now we don't need to play them anymore there isn't much there so there's npm stuff so you run composer you run npm you do some database stuff that i think assumes you've got a database okay so we're not going to use sql lite that's the easy approach here let's uh manuscript idea postgres what's your poison canon what'd you want i can start filling in the environment that we need to configure this but let's just walk through those steps as well if i didn't know then i would come to the mirror db image i can scroll down and eventually we'll get to these environment variables that allow us to configure how this runs though uh we're going to do a random root password nice i've never used that yeah it just means that we're going to force our automation to use a very specific username and password that we also configure for environment variables this is how it would run in production we wouldn't run necessarily with the root user and our production db i'm not going to do that here which means we can now specify these two fields the username password and the database we'll do that now and we'll call the user and crm and the password is going to hang zero not very secure but instead it's okay what else do we need database okay uh we'll get the database of name ping serum yeah you got this let me explain something else is weird about this configuration the mariadb container image wants to be a drop in replacement for oracle's mysql which is why these environment variables are called mysql so you can swap these out as you wish and they just work now that's the database configured so once we've run that we'll have a an application server an fpm server and a database we need a web server correct well we will need nginx and we're going to make that available locally so that we can actually browse through it um we need our codes inside of this container we want to work under that code so these two go great together we can set the working directory which means if i exec into this container i'm going to be next to the code where i mount it which is also here and what else does fpm acquire i'm assuming we're going to have some cash and logs to deal with but i'm not sure what that looks like yet yeah we'll find out all right well why have i got a rare oh yeah because this is a list okay looks okay yep all right uh so thomas is reaching out with a comment using dockerboards b3 would allow you to use secrets and mysql password file yeah there are just too many disadvantages like if i were to use version i really recommend watching last week's video is that rockwood.live and the steps that i would need to take to orchestrate or script around starting my database container and then waiting for it to be healthy to receive traffic and then spinning up and your next net game it's just it's just too much work and as much as i would like to use the secrets locally um i'm happy to admit that for an environment that i can spin up with a single command and i think for developer onboarding and quality of life and working on the project at the end basis that is more important than consuming something like the secrets i think that's a good example of what you were talking about that version three is more oriented towards deployment into product as well because managing a secrets file is very much the sort of thing you do want to do in production but probably overkill for a dev environment you just set there's no point having a mysql secrets file that just says ping crm ping sierra and pings here credentials yeah my goal is to so our number one objective is never to break the parity between development and production as far as operating system dependencies extensions are difficult but when it comes to the dev site i want developer happiness and experience and when it comes to production i want security and resiliency and you know those i can't have both of those at the same time and i'm not going to settle for anything in the middle that satisfies devon prod so just separate workflows appreciate that and make it work cool nginx fpm moviedb uh i think we're got a certain point yeah we need so we need an nginx config file so nginx can find fpm and we need to know how the php app connects to the database so we can set some environment variables so it can form the database perfect so you're right let's get that engine x config here and they don't provide one do they no laram did just saying guys all right let's steal the flower one it was a dot failure it looks really complicated though have you got the one from last time uh i don't have the one from last time because you had a very cut down one didn't we okay so that was under slim hot engine x theos.com i'll just copy that there for now but we now have our v host file here which is very basic we might just need to change that do you need to change that route path uh no i'll just make sure that's where i meant but we're not delivering any anything yet we'll do that if we have to i don't think we i don't believe we have to just yet okay uh let's get this working there so if we do a volumes here we mount our vehicles.com that's etcetera engine x contraby default.com and we also need some dependencies so nginx uh uses as you can see in this vhost.com engine x uses this hostname of php which means if we remember from last time that container has to be created first otherwise that resolution will fail yeah because we're using version two we can take advantage of uh the my editor seems to know that we're going to do this syntax um hp operator condition service now we don't need it to be healthy we just need it to be available for the dns resolution however our php application as a dependency 2 which is going to be maria db and the condition here we're going to make service healthy because we're going to automate running our migrations potentially potentially which means we need to add a health check to this so we can do a health check the other test we use the shell syntax last time we said robash fc and this is a very quick health check but it's fast uh we did uh cats i was dependent p2p i'm going to get this wrong deeply and cut from cat you're reading from it yeah i'm just trying to i i could pretty much do a cat or an echo to it um it'll fail as that port hasn't opened i'll show how that works in a second we'll spin up the container and actually run that do you need an angle bracket in there cut two no it should be okay like that let's let's let's check it okay so container run you know interactive we're going to do maria mariadb 10 and i will oh we'll just run bash and start docker why do i always forget to start docker it'd be nice if the command line had a do you want to start docker yes no all right i'll just take a few seconds well this is because you've got a mac right like yeah i keep debating back to used in my linux machine for day to day but i just i'm trying to persevere with the mac right now and so alex has left another comment thank you for making this interactive there's more fun when you leave comments if you want standard nginx stuff for larabelle yeah next um i'm hoping what you're trying to say there is laravel possibly has an nginx portuguese if it does drop the comment and i'll i'll just use their nginx config straight up if they have any problems over i'll make changes and i want this to be super laravel specific so send me a link please okay let's run this again and the syntax we're using is bash that's the cat dev ptp localhost pg06 and not such better directory so i've probably just got that wrong again i one day i'll remember that syntax it's a really cool hack though so yes i've seen people running curl and stuff you know and having to put something like that into their image whereas this is bash built-in stuff isn't it it's like catching yeah is that tcp well but i did the necklace of the cap okay maybe not as uh so echo ttp localhost so yeah that's better so what bash does is bash provides a virtualized tcp stack through devslash gcp which means that you can put in any ip address or domain name like google.com3 and it'll test it as open if you don't get a response back like that it means listening and you can actually echo the response code if i want to see whether google is responding on 1443 then you know that's going to change but localhost you won't get a timeout you'll just get a direct error saying hey this is so only bash provides that it means if you're using alpine and as your base image you won't have access to it unless you install bash but it's a really nice grid way to do simple port-based health checks when that's all that you need and as we discovered last week mysql and mario db only open the socket when the database is ready to receive queries or commands which helps very helpful okay so let's make that an echo there and so now we have a health check on our adv we can depend on it here now what that means is that we now have a single command to spin up all of our services which should work and dc is just an alias i have for document pose i should be able to run up oh yeah so 2 doesn't work has to be 2.4 and it's not oh it's test right command or command shell no one ever remembers all these things okay cool okay so for the environmental stuff for php finding the database laravel i think uses often for config dot n files all right so yeah we have a env example nice so those are the there's the sql we'll need to look at what the look at so that makes it so almost trivial to dockerize our containerized application you know this is classic 12-factor manifesto stuff being able to configure each environment through environment configurations it just makes everything a lot easier but that's great i'm assuming i just copy this to be not a dot example so let's do that um okay so all that started that just means my health check is working engine x can reach php as a dns name and php is well that won't be doing anything yet so php probably exit i know fpm is ready to handle connections but i don't think we've we've told that where to run around like that so it's probably not doing anything so we should say what's the next step i'm going to say see if we got the hello world nginx um i'll just spin off in the background we don't need to watch the logs right now so i mean they can leave it running so comment from alex in the chat yeah i actually see not alex but i see people get confused about dot n versus real environment variables and yeah if we're using docker we can specify all the config with our environment our actual environment variables and not have that dot in file right so you can um but it makes it a little bit more uh difficult so let me clarify that now i can i'm here add environment and add every variable that we need now the first problem is uh let's just do this i mean this becomes a really a certain number of environment variables and if we take a look at this.enb there are fair few and also i cannot reuse this now what i can do is use the dot compose env fail where i have a list and i say.enb and that just means that if i need to consume those environment variables in another process whether the uh imagine if we have i'm not sure what laravel uses for and job processing if it has a different process that runs but you know if this was real it'd be like sidekick or something and it might need access to those environment variables too it just means i don't have to duplicate it but i generally prefer using the dot env approach um and then just linking it together with it now obviously wouldn't have these i can't conflict planet with um i think laravel does this i know symfony does this the frameworks then try and read the end file directly irrespective of whether they're in a docker context to find environment variables aimed at dev type environments so i'm not aware of any frameworks that would read the dot enb as part of their process what larva maybe what most recommend is running a source prior to running your commands to pull it into the execution context so i'm not on as some ground on laravel but symphony has a php component that will read the dot end file and set those environment variables in php using php set end uh no knowledge but basically it sets them in the framework and that some can sometimes conflict with what you're doing in your docker compose setup um let's see if we run into that i'm not sure what will happen so i tend to turn that stuff off basically is what i'm gonna get to and make it so that it's docker that's reading the environment variables i've pasted in uh it's pasted in twice because this sharing isn't working but you can see the commented out section is pasted wrong that's the mysql config delete the bit below let's put it in about eight times db databases specifically especially something like db host i'd be happier having in the compose file because then you know that service name yeah i think i think i'd be i'd be happy to put them in the docker compose um although generally what i would do is use it for dev i'd use the enb um right and so that i can set them once in fact that's a good example let's do that but here i've got this here it's better when the configuration uses the same names so i can say this is mariadb mysql username uh let's just copy this supernova i don't need to guess it okay so we have these now this also means that i can just consume this in env here even though it's got loads of stuff in it the only bits i care about are the only thing we need is just this one here there's no point no i don't think there's much point in putting that in the dot entry because it's not consumed anywhere else uh and in fact we get to split that out so we're not having that we're not loading on everything so let's just do dot enb dot mariadb that there and we'll do various v and then we can consume this file and the fbm one as well yeah and then there's a third option isn't there that docker compose will pick up that dot env file and you can use those variables in this config file oh i wasn't aware the docker composed with parser.ermv but you may be right it does so if you for instance you could have it in environment you could have my mysql username colon and then dollar what what is in there what the secret is in the env file okay uh cool that's good enough chat alex that's what i was about to do good good comment so i'm going to remove these ones there and there and we're going to change our configuration to use the mysql one so where did he say that there was conflict database awesome and advice yeah there we go so i'm okay with that one being the same but we want to reuse that's real cool report yeah great okay so mysql that's user isn't it gonna have to check this like 14 times okay mysql database user password user password database user password database you didn't do these ones because these are unique okay i think we're good which means we've got db connection the report we've got a reusable one here now we can confirm all of this work by spending all of that down so we can do a drugstore hb and what we would expect to see when we do an up now is that mariadb should still be happy because it's consuming the configuration from env and our laravel application when we run into our migration should be happy too because it's consuming the same all right we could change the interval what's happening here and why the maria db has taken five to ten seconds or however long that is is and our health check here we don't overwrite the interval or how many intervals we need for it to be considered healthy etc so we could speed that up by just changing that but i'm quite happy with it's not taking too long now if we run a ps we can see our health check now reports this is healthy we have our engine xlp and we have our fpm up sorry now what you wanted us to do was to confirm that's uh that's good i'm happy with that that's engine actually responding but we haven't actually given it anything to serve yet yeah and okay so uh now we have another comment from alex four more comments alex and we're just going to have you join the session so doing this obviously makes it harder to change postgres in the future it would take conflict changes but that's fine if you know what you're going to use no one actually changes database engines uh if i'll leave that up we can talk about that for a little bit if you want any thoughts on that queue uh i have changed database engine i had a cto say good news we've um we've decided to move from oracle to mongodb once it's a travel change either it's going to save us a lot of money there is the sale yeah licensing is a lot better um yeah i feel like we're changing things a lot of places um because i guess looks like laravel has that two layers of environment variables and then that config file is mapping the how you're using the environment variables to the real config and we've got another layer of indirection going with the docker compose so you could probably rip it all out and make it very simple couldn't you you probably have in the end file my my database password yeah and what pretty much you actually simplify by using a dsm so another nice way of handling this kind of thing is to make make it so there's only one config key for the database connection but having a url-like um protocol for it i've seen that for databases cues etc so am i correct and saying what you and alex are both worried about is i modified this file right yeah okay so let's let's not change it which is sounds like fine if you know what you're doing okay so let's not change that fail now the only reason i changed that is because the nr.enb doesn't really be we have these variables available to us so let's put this back to what laravel expects then and then we'll change it on the database side which i guess is actually cleaner because we're not um our cases are in a single location so what we can just do here is say this is equal to db username uh password password let me get that mysql database it's a db database and that should still work well without touchdown of a normal arables i think yes i think this is actually much nicer i think that's a good idea let's make sure it's not broken i can't see what it's doing you can't see me what you can't see yourself when there's a comment up oh that's a shame ah not very ah it's not just trying to pick up above the comment it looks like there's a typo though doesn't it db data database yeah it's not the best name all right so we got so maybe builds wrong in all the places okay so the database is still happy um and we can just confirm that we can connect to that now so let's do a dc exe i'm not going to spend any more time on this i think it's been a while now we'll go inside a really big container oh yeah you can't do it on top of compose exec we do it for you um we should be able to run mysql host 127 user peng crm password and then the database ping crm what did i get wrong there you've got database spelt wrong in some of the places in the variable as long as it's only in that place it should work you're supposed to tell me these things i did i tried to but said it wrong okay one more time so that python.n stuff is from docker compose trying to read the end file so yeah docker compo there's this really weird thing and i can't remember the specifics of it because i tend to avoid the syntax anymore but to use the docker composed one i think you need to export it's a bit weird um we'll deal with that if we have to the two hours as long as this works we can move on to the next uh all right so let's just check we have what we expect so i i wouldn't expect to see a bunch of these okay let's not worry but that's our challenge then and let's see do we have our db underscores no okay so that is that really annoying thing with docker then so i probably let me show you what i mean docker is a bit annoying and uh i believe it expects this syntax which i bet here is not going to work with the dot enp and the barbell sides rather than me fighting with it i think we'll probably just stick it all and the doctor can pull snail for now you haven't got you haven't got those variables defined in the env have you er defend here right and where's that referenced in my post file here that no that won't work so the way it works is your end file setting let's go back to the compose file your m-file setting means those environment variables will be defined in the container so i would expect the db one um the in your environment stanza your expanding variables those variables would come from dot end only yeah it works now yeah okay that's not working that won't work is what i'm trying to say right anyway we can you can't use the variables in the in the docker compose yaml the n-file is sourcing them into the into the running container it's not making them available while parsing the yaml if you want them available while the ammo is being expand being uh expanded you have to put them in gotcha okay but then also expect this export syntax rather than i've gotta be honest i haven't used info much okay so let's test the hypothesis oh no the problem is i had clones export probably wasn't i'm just being really tough today i had pbe user and i had the ammo syntax i would say there's not yet yeah right okay you're saying that we put this here we'll get the expansion let's just remove this don't want for now and let's confirm that these weird weird things all right did you just take it out of the end again i thought you said apart this one automatically no yeah without you putting n file in but did you delete are those variables defined in yeah they are okay i moved them i moved them all to the end let's just oh that should be here okay really going to change that interval now one second so what we just want to confirm here then before we actually do something a bit more interesting is those environment variables are set and i can log into mysql slashboard adb okay that works well i can do one two seven thanks here i'm paying for you around right right how can you protected that password even though it's ping zero okay okay so something that uh we didn't touch on last week is what is our okay we haven't configured fpm so let's go inside of our fpm container they call it laravel rpm html or something and that location is not correct so let's just that's the apache stuff you don't know okay slim let me modify it here rw html whichever you get against that nope okay slash one cat command line we use cat we'll call that's a really cool hack actually whenever you're in a container slash proc slash one is always the running or primitive process and if you just tap the command line file you can see what the entry point was saved my bacon a few times now so that is our default fpm configuration and we're just looking for what are you looking for uh just a rip which apparently is and uh maybe it's in this dot d directory are you trying to change the config to point to slash code rather than i just want to know how that is so yeah so i can get it directly just read the readme and put it in the place they tell me to and try not to look at how any of it works there's fun in that we've got a whole bunch of different things here uh that's the www.com you can see some environment variables being set there you can see cherry is not set no that doesn't mean it's the webroot camera whatever they call it let's even grab this what i'm lost what you're trying to do i want to know where expect the php files to live we saw that as well www.html but that directory doesn't exist which made me think it was lying isn't it just because you have to mount it in we'll copy it in oh you know let's just get the doctor okay i couldn't grab an fpm one that was unfortunate fbm worker yeah he's fired a little bit you know okay it just doesn't exist so we can mount our code we'll just leverage that so we're changing the working directory now we need to serve public h2 so let's see the same as what we did last week but we need to modify our engine x configuration loss and this will just need to match public excel right we will recreate what he's recreated it's got some interesting questions in the chat about how do you maybe i'll come on to this how do you run typical sort of development commands in a dev environment i think that definitely i use a pattern called uh d shell so i'll show you what that looks like okay good i mean it's it's failing but that's okay because we haven't followed the installation instructions which means we do now have commands that we need to run as part of the installation which means we need a development environment now whenever you type php or composer here to me that's a warning sign we want to avoid that as much as possible because we're running those commands in an environment that does not have parity with production but how do we change that what i normally do is build a development workflow it looks like so so my entry point is now going to be rash that we'll do let's say i make target a little bit nicer there's no mix file in this project ah that's where we prove that it's gonna make fail i generally add a detailed target and the goal being i want to be able to say make d shell on a terminal and get uh interactive development environment that i can work with which means what we're going to do is do a docker occur compose run service ports and i'll explain what these like do in just a moment keep adding those things um it's really annoying run service pool rm php is the service that we want to run i'm also going to override the entry point the batch and i don't think i need ik i do not forget so i don't know what service points does service ports yes so when i do a docker compose run which means i want to run an individual service here it does not publish those ports and when i tell it to when i add service ports it will just make sure that the ports are exposed now actually i'm not exposing fpm locally but i'm so just going to keep the flag there because that's important for most applications and may become approachable major and i override the entry point so it doesn't run fpm so what that means now is when i run make b shell the make bells are fun so we need a dot phoney b shell that's basics or tabs and there's no editor or console there it's we're going to there we make go pendant style tab and then size for receive this file i hate that it does have i really wish it doesn't uh have that pain point of tabs and spaces but doesn't matter so what happened here is we ran php it started marie db because of its independency it hasn't started in janae which i didn't think would have which i didn't expect to encounter but it makes sense so we'll fix that in a minute and we don't have anything running here which means which means i need to add one step here up engine x background just because we can't have a cyclic dependency so i can't start php um because nginx depends on it so we just have to make that a bit special let's run that again engine x is up fpm is up now if i run here it's going to time out and fail oh it didn't oh because i still got it running shut that down and don't make these shells so like in a fresh environment with nothing running i would run this command it starts engine exposure really push me on an interactive terminal i can go here either starting our engine axis turn it because of the dependency that's funny because you're doing compose docker compose up dash d nginx before your yes all right that's good okay doesn't matter so uh now we need now we have an environment which is production like webmite fails i'm not gonna have composer so we need to fix that so this isn't the is this the have we run a new copy of that service yeah there'll be two running right now so let's jump out here go to my directory php examples we're relying on the fact that they share a volume right but if i run bcpa but this is the actual service that was started by the nginx dependency and this is my run container um they're both interactive let's see consider right now it's horizontally skilled um we can still run our migration paths and asset compilations they're both using the same volume and so i don't foresee any problems right now and we could remove that this one here um by just having an exit zero in the docker compose file but as it's really not that important yeah it makes sense so yeah so we can do all the typical stuff because all the things you want to do are either creating files in the volume or probably doing stuff in the database oh yeah definitely so right now we need to fix the composer thing like if i'm saying i don't want to run composer installed locally because maybe there's dependencies on extensions that which would fail locally then i have to do that inside of the container and right now we don't even have access to composer so we need to fix that and the way that we do that is the same as last week two where we have multi-layer files so we're going to say particularly right we'll call this our base we're not we're not going to do anything with we're just removing the duplication of this image then we have dev where we add the tooling that we need so i can do uh apk and we're not doing all things i can do an app update and add install uh let's just stick in them there okay and we're also going to copy composer from the official image and if we remember correctly from last week it is available under user let's just type in composer to use their fingerprint that may be local but i can't remember we can come back out here and instead of using this we're going to say that we wish to build an image the context of the current directory and we're going to target that dead layer that has all of our wonderful tooling that we need make sense yeah okay so now it's also going to affect the running fpm right so we've we're making it even more explicit that our our compose file is for dev oh yeah totally yeah those files should never be used in production let's get to talk about i mean there are people out there who'll say don't post cabbies and production with version three and if you're happy with that by all means but um it's not for me so this installation of apps is going to be cached right because we pull it we're copying stuff from the composer image which is always going to be cacheable and then we do it okay so let's go back into a d cell we now have the ability to run docker compose install and now you can see the the pad problem right is that my local environment magically seems to have some extensions that are required by this application that aren't available in my production environment which is this container we need to fix that so we need these two extensions and the docker file provides a whole bunch of different uh helpers to do that so and we're going to run now we've got php what's it called i'm just looking it up docker php extension install there we go this is helper script where we can do extxs and exp gdp you spell xa from that's a lot of output i don't know about working didn't work stop refreshing i know i i don't think you need the you know access and gd there we go and then we can copy that command to replace image so docker pc control 3d uh what's that look what's that let's see so and does that help that's our dev package let's work out this dependency maybe i should just quickly google it um zed lib g zed lib 1g yeah that might be it we'll try that and if not we'll just look it up very quickly that's what google says and this interactive environment is good for this because we can test this works before we rebuild the image all right okay so like we got past that so let's add that first so our face image now needs to start update and at the install that load oh what was that one g yeah we also need some sort of png one so let's do like png and it's a dev a png pad i never codifying everything that we need it's good more works on my machine nonsense so in terms of um wanting the latest versions of things how do you how because this is something i've wanted to ask before actually when you've got your app update at the start that's going to be cached right so it's every time i build the container it's not going to do apt update no but this base image you know our continuous integration system is going to force our rebuild on this on a regular cadence either every 24 hours or maybe once a week depending on how frequently your deploys are you know that cadence is really coming down to each team but yeah i mean the challenge being is that if you have a base layer like this and you never rebuild it then your dependencies your security apaches all that become very dating very quickly so you have to build there you'd have your tooling force a sort of uncached build every so often yeah your cash bills during nine till five and then at three a.m every morning have an on cash build run that make sure you're always up to date right okay so that's it now yeah so the reason i say nine to five use cash bills is that you know if you're shipping and building regularly during the day you don't want your developers sitting waiting any amount of time that they don't have to and it depends how slow your bills are if your bills are under a few minutes never use cash yeah just make sure people are productive during the day you don't want them setting sort of fighting because of compilation stuff yeah but we've got that safety net that if there's a critical security vulnerability in z-lib we're gonna get the fix at some point exactly and the composer install hasn't filled with the extensions we now have a container and production environment that has everything we need to build this application and we can cache that composer step as well as we we did last week where we say okay um we don't modify our poser.json that often so we're going to copy that to our working directory and we can put this but i'm just moving these steps up because even if this does change we don't need to reinstall these two dependencies so we'll use the cache there and we'll do a composer install and that becomes part of our build process that means if you do modify your composer.json you just have to rebuild this image to get the latest dependencies okay but currently we're mounting our local vendor folder over the vendor folder in the image anyway yeah there's a little bit of hoops you would have to run around actually to dig that up which i do because i'm like i'm using a mac and the less files mounted the better so for some big projects i end up just mounting instead of mounting everything might mount the app folder we'll come back to that like i mean we could spend at least an hour just covering that especially on a mac because it is very weird uh but let's not we'll come back to that maybe next week another session let's try and actually get this running we're another 10 minutes and we haven't got it running yet so isolation steps so let's expect us to run this generate commands so we're going to codify that in our make file so let's call this cell what else do we need high grade collaboration expired right and we'll be okay with that we need to see the database and we can run stair that's the web server isn't it i think so so we've nginx already we don't need that okay so let's go make sale so we're still inside our d shell environment which is our interactive dev environment hopefully all these commands just work which of course is never going to be the case so if i run make setup from the host it's going to fail well i wouldn't be able to think of database but because maybe i haven't got php installed or really be running you're always running mate d shell and then using it inside the d inside the d shop right so it says it wasn't able to find our driver uh that makes sense we haven't installed the mysql driver as part of our dockerfile setup because it wasn't it must be an optional dependency and the composer.json because you might not be using a mysql compatible database so we just need to here at i think it's called mysql you think that's right is that what the error message said yeah i couldn't find the the driver for connecting to the database uh pdf we need to mysql that as a downstream product here and i know what database technology we're going to use i could come in here and just enforce that at the compilation stat uh where's the extensions to the top line 30. so we could just quantify that like so let's just turn them and we come back into our d-shell and we can run set up and hopefully it's a little bit happier it is happier but it's still not working so connection refused so let's check our configuration uh so fpm doesn't have any config is that just a full path on our part oh yeah we haven't set it we only set it inside the microscope let's make the shell one more i got baby that's the one okay okay uh database whoever host i think alex said that in the chat about an hour ago 450 alex said we're missing dd host uh okay host not host name you saw nothing okay okay so the migration ran our database was cd and so asset compilation is that what that means is that what makes this right yeah i think so so the in the instructions there was a couple of npm commands ahead above what we did all right okay so we need an npmci to install the dependencies and an npm run dev to build our assets okay so there there's half a dozen ways to do this so a lot of these things we're treating them as ad hoc one-off things you do in your dev environment so you know proceeding the database running there's actually let's split them into two there's things you only do in dev seeding seeding the database isn't something you're doing in proj right running migrations is maybe something we're going to do in prod i'm building the pipeline at that kind of build step that feels like something we should be doing as we build the image almost uh yeah there so i think the two ways we'll try and cover them is the one you've just mentioned which is whereas maybe as a php developer i'm not going to touch the the front end assets or anything like that and i can make that into the image because i'm not going to change but i think there's the other end of that where maybe we are changing assets as part of our development workflow we still want that to work too okay so let's do uh let's go with the naive approach and that we're not going to change these assets and if i'm not going to change the assets that means i can ship them in the nginx image if i want i mean if anyone's got any strong opinions and like again we're both not hireful developers that's maybe maybe our experience here is not the one to listen to what i suspect going to happen is that mix like symphony web back encore there will end up being some interaction between the php framework and the static assets like it'll check they're there or do you know what i mean it'll it'll want to have access to them in the php image that's my prediction okay so let's make that's the easiest way is that as part of setup we run an mpmci and we run an mpm run there this needs to be naive let's get it going and then we'll see how we can optimize it now the first challenge is we don't have access to npm inside over dev based layer image but we can fix that i believe click so so uh install node.js and npm uh obviously i'm not entirely comfortable with uh leveraging or damn damage as uh i super repository or at least a super image with loads of stuff in it so we'll maybe try and see if we can clean that up but it does mean that i can run mpmci it should just work and i think what we'll do is we'll just prove this works we'll take it away and we'll break this into the engine x side of things like i really i think i do feel quite strongly that engine x should be serving the static assets the build process should be part of that image builds and that's a change i'm just going to force a rebuild i mean the development workflow for working on php versus working on the static assets would probably maybe i'm just not the person with the experience then definitely how the workflow should work yeah the forum developer is going to want you know what it wanted to be watching the file system and rebuilding if things change and all that stuff which might lead them towards running that stuff locally so we can we can we can make it work for me i just want all the stuff built so i can work on the php so would that be uh a node build step i'd probably add another container that handled it at all this npm rundown seems like it's actually running a web server unless it's going to start doing something in a moment these things take time it's a lot of fun so it's probably quite slow with volume mounts that's true i wouldn't have that problem on linux what'd you call them host mounts post volumes yeah a horseman what a pained mount or a horseman because there's been some chat about slow slow max it's that right it's having lots of files mounted oh so let's see what happens maybe uh i think something it was doing something all right we're getting some sort of 404 so let's just see what's feeling css okay yes oh let's check our engine x config so where is that gs fail come from app.css now i can't see that there it will be probably there yeah okay so i'm swimming there is some sort of redirect magic on the fpm site they should be looking for that stuff in that directory let's go to index.php i wonder if laravel should be handling this you'd not want it to would you all right if it just outfits everything under resources well now because the cs is under app css but the path is just here let me check that no no css apps yes yeah so we want to serve that resources to retro as a root fail system from nginx probably that's kind of the vape i'm getting have a look in the public folder and check there isn't a sort of sneaky sim link or ah there's an hd access there's a little css folder there isn't there but is it it just pushes ever heard through index.php which makes me think larabelle is trying to handle that and then it's failing no that that's in that in that htaccess it's only applying it's only requests that don't hit a file or a directory you're going to php yeah but a request for css app css is going to go be routed through index.php because it doesn't exist that is that css folder in public empty oh right good oh okay so what's going on here then i'd like to publicly ultimate and resources so maybe that's just what they left and then it gets compiled into public css alex says that laravels does not handle the css jx oh let's try css uh but we're serving everything from oh no right okay so i think the problem is is that request is coming in for app.css our engine x configuration at the moment is configured uh through everything through the index.js and then that's failing now let's just see if there is a uh engine x for laravel rather i was trying to so you think all requests are going through to php it's definitely yeah right we just want some we just want our production engine x kind of thing yeah looks good or we could just make that those assets available and and the nginx site and we serve them over internet i still feel that that's the right way i would make more sense from let's have internet service okay so how do we do that we need a volume yeah okay i was naive approach why not thanks and that and the nice responsibility of course that should just take everything from here the the challenge would be the index of php would exist very nice approach all right let's see if that fixes engine x uh let's just use a second terminal so i want to restart your next uh what that should mean is if i go inside the engineering container but i'm not right now for the wwe html rm engineering oh it's still there down there we go stop s stop it and force it right okay and then start and uh okay so now it should it was a little bit of that there uh let me know we have the 403 problem uh which we got last week as well and then i believe it's just because that public directory um the permissions on it i don't think it's permission i think it's that it just doesn't exist see what happens if we do this those are going to be trial and error rmsf engine hey i'm not back definitely a 403 and the problem is that is for betting so it's trying because the public folder does exist does that mean it's not passing things on to fbm is our conflict so this should be running as alex has some pertinent comments in the chat did i mention to the wrong location no what is our vehicle conflict like can we remind ourselves oh wait that's right because that's still the fpm i can't get a directory list and then that which makes me you know the florist seems like his permissions um i can access that and so i'm assuming i could probably act there that so the challenge is just a php fail so what i need to do oh it's close what what's the problem the problem is let's add uh pick it up okay so adding the index.html at least gets rid of the error the problem is now the fail exists um so when you go into slash it's because the public folder because the folder exists in the nginx volume now it's trying to serve the directory index of that folder rather than asking fpm for it correct and then something else has turned off directory indexing is that right yes but it's trying to deliver it's trying to find the root fail which doesn't exist and we should be going through the php but i think we need to add something that's specific so what would that explain the root file does exist it's the folder on the end of the root public he's saying that we do that it should work a good thing people watch this thing all right let's see dc's rm so we're going to restart that now let's confirm yeah that doesn't work so we've broken that but it's not that we want to add a public here is that you stay and just use the official engine x thing well let's do that and we can look at why it works and us doesn't okay i wonder if it's that little cheeky ah no is it ah that's what it is about one thing take that back then i'll copy the failed but you know i don't i need to confirm recommend them the problem is looking for an index.html it can't get a direct list but we can just say the index is a php file which forces it through this root thank you since okay that works and this will work all right i'll probably fail copy copy the whole file we're getting bogged down in nginx if this works let's never talk about it again that's not important you're right it's not an nginx stream not using a unix socket we're using i should have kept that lane uh i think that should be all right okay gs is say i'm not found here don't and come on it's good that we've got that confidence each time oh see this is why i didn't want to copy this nonsense okay okay it's wrong primary script name unknown and because it should be okay does the server name have to be there to i feel like i was closer with my own fail css works for betting is because it's trying to access the root tail so that should be enough to tell not to look for an index.html and if that doesn't work we'll fix it okay it's never again a fail not found let's try written adders through php which we had an example up here yeah we shouldn't use the public we just want to force it to go through php so it goes to the fpm endpoint probably that's not working just make sure i'm getting the right up to the error message okay so the cpi editor now is saying yeah okay so the path is important here so the log ah all right let's try and do something interesting okay so now we've got engine x serving our static assets we've got php surveying the php application we've got our database working for development environment we have an interactive d shell which is over here so as i change my php code i can run composer commands if i need to to cache clears do we have a development environment don't work yes yeah i think so if you're a front-end developer you might want to have some clever watch magic going on but it works um should we pivot towards production just thinking about the time available because this is very devi isn't it uh yes this is a development environment so and be interesting to talk about how we'd build the npm stuff for broad okay i just let's just make sure it's cemented a few times so like what as of my the development environment would be let's say i want to do database migrations maybe like i just want to make sure that people understand that this is the way that i would work if i were to change my database where the migrations were yeah cool if i were to change one of these files my development workflow would be that i know from this detail environment just run these commands and it worked well that's teachers complaining whether they already have seats that's not highly important so maybe just only run a migration i don't understand how the seeds work in there though i'm just going to remove it there must be a clean command right here then baby white let's see what yeah okay that's better uh what about tests how do you run tests and php it has some test targets i think um it's basically php unit stuff with some extensions for laravel pest agreement lots of tests okay so i think the development environment is fine we could definitely clean up the asset stuff but yeah let's talk about the production aspects of this and then i'll push alex comment on the screen so you can do a migrate fresh suit seems like it's a bit nicer okay um how do you want to apply this to production thank you well you tell me um so what's different right what's different is that the composing store would take place in we're not going to have as many volume mounts right we're going to do the composer install inside the php side of things we're going to copy the files into the images instead of mounting them in and we wouldn't we'd want to build those assets without having to as part of the image built so that's a question it depends how we want to serve the starter cassettes like do you want an engine nginx image with them baked into the nginx maybe in real life you'd be pushing them to some sort of cdn instead yeah both are very uh both are good workflows so the the challenge is just um if we wanted to build an engine x image with the assets we would ship them together as a deployment with two containers and a pods and probably share the same tag so they're always deployed together but we can talk about that and then we can also talk about the cdn approach where as part of your ci pay plan you would just push them um to a cdn and then ship your php image with the correct uh urls whatever so well actually i think i think there's two engine x's there's two roles in this in this app right one role is serving static images which doesn't necessarily have to be static files it doesn't necessarily have to be in the same pod as the php application right the the other nginx role is so i i i would just because they have to be deployed together in a sense there because the static assets are deployed as engine x would be very unique to that build and wouldn't it's not like you would run older version of the assets with a newer version of your php and fpm stuff so for that i would deploy them as a deployment with two containers yeah i think i think that's very much then depends on the application and the teams involved right if you've got um yeah is that conway's law thing where the structure of your teams determines the structure of your application if you've got different people if you've got a detached front end and back end workflows where maybe there's a single page application is the is the app and the php back end is like an api that it hits they may well be separately deployable because the fact those two groups are working separately would mean the api would have to stay super stable whereas the front end team could iterate quickly and make lots of changes right on the other hand if you're talking like a traditional php lamp stack team where there's a couple of front endies and a few back ends and they're all working together on the same stream of features then yes 100 you'd want to be deploying them together because they wouldn't have had to bake into their way of working the fact that the api has to be forwards compatible backward compatible they'll break stuff all the time um in a coordinated way so which one should we which version should we pick so in that separated out team right you might have the front end assets being completely deployed to either a cdn different set of static static servers static asset servers to when the deploys to the php back undergoing and in that case you might not even need nginx for the php um i mentioned it to you that i didn't know about this before someone called lewis kobuchi mentioned that kubernetes can have like an nginx ingress point so you ship your ph you ship your fpm and sort of trust kubernetes to expose that to the world which wouldn't work for static static assets but if you have this if it's a microservice and doesn't have any static assets maybe that's a good option okay so we let's pick a pic pick an approach for this app all right so what we're going to do is we're going to create an ob directory and we're going to start creating our kubernetes manifests we'll try and tackle a few of those different scenarios that you mentioned so this is our deployment we'll call this a peng crm and we're gonna and work out what goes here so i think because the cdn bet is not something that can show off right now we can definitely talk about that there's only one thing that changes in the deployment pipeline so let's go over the approach in that we have two containers in the deployment one is running nginx we want to run an fpm for our demo nginx we'll have the assets and we'll build an image specifically for that and we'll see how we deploy them together but at the same time in your ci system you can just not deploy the engine x1 and just have a make cdn target that does whatever your cpu ending is yeah so i think this is the best way for now which means that we're going to have an image called ping crm and we'll call this engine engine x and we'll give it a version of the latest which i would not recommend you do but i just wanna yeah i don't understand what the latest is i thought i did at one point and then all my illusions got busted uh we'll have that so now we need to build these we also need to make the ports available so of course international lesson on app let's not make that event and now we need to build the images so let's assume this makefile is also our ci which means that we want to build internet stuff and build every entry stuff let's start with let's get engine x delivering assets first so if we come into our deployment let's remove let's just comment out fbm and let's just get this working first there may fail we can just say that we want to build an image and we're going to call this peng crm engine x lasers and we're going to need a target here because we're building engine x and this could be an echo one for that so it doesn't break when i run it we need to satisfy this build set so we're going to continue to use our multi-layer build and what we're going to say here is that from engineer well we just need we ignored we need to compare the assets first yeah yeah i think 10 may be an image yeah as assets we need to be able to distribute them so you're right with the engine expert we'll just leave that without a tag and this is going to be our nginx layer that we actually targeted what we want to distribute here comes from our tooltipster9 method build and we need to copy something to our www.html and we want to run npmci the dependencies yeah what what i found is that um front end build stuff is much less likely to have any sort of system dependency so you can you can just you then use the node container straight off you know without having to like we have a php of how to install a bunch of extensions and system stuff you don't really get that with it yeah i would just put that to work the only thing i want to check is does npm run dev i don't know if they have a specific production task that maybe does minimizing and yeah so let's call that the production one i might make it slightly difficult to validate it but we'll see what happens and then we're going to copy this from slash code yes it's going to be in public yep so we could be a bit more sophisticated if we wanted to dig in about which files were copying in mostly from a caching validation point of view so that that node build target um is ephemeral right it's gonna it's gonna get built it's gonna do all the compilation and then we're kind of throwing it away we're not gonna use it for anything we're gonna just copy the final assets out but so in terms of image size you're not too worried about copying everything in but the cache will be invalidated anytime you change any file at the moment so you'd probably only copy the files in that you knew had to be there for the npm build we don't we're not going to do that now because we we're not going to dig in and figure out which ones is it just a resources directory we can we can we'll give that a shot and if it isn't we'll just you're definitely right like we probably only need and package json in the root i think you can do this sometimes right you can just um you don't need to do that you just um you list them out you have multiplied and the last one is the place to put them right i'll maybe do multiple lines are you i'm pretty sure that'll work let's test it we're going to use our cia assets i'm slightly concerned it will lose the directories things are in under code it'll lose some of the structure actually it makes sense to have multiple lines sometimes if things are gonna change at different cadences yeah the package don't do so i'm probably better being copied in first yeah uh so there you are this project is using some for lock validation which i generally discourage but so you wouldn't use that using docker right sure it just gives you consistency guarantees that you don't particularly require so you do require but don't forgive you right right right so we're using um build kit when you're running the docker but compose doesn't do that yet no it doesn't uh we're packed up mix we need yeah i'll add that and run it again and then it doesn't work i will add everything for now yeah right what's up don't mix the json it's dotmix.js uh yeah i think we let's just copy everything that's what viewers got the idea of what we're doing right something that annoys me about npm is it often shows things as errors that are just sort of more like warnings or it seems to when i'm building stuff i don't use that enough to really really know hmm he's telling us to run npm update that's interesting okay now what we have here is thing crm nginx let's just go inside of this container firew earlier html and there is all of our metaphytes dot cool should there be directories no yeah i think my star here is probably not what we want what we want to do is copy let's try that one more time if i can start flatten the directories which we don't want yeah and that's invalidated the copy because we edited the doc file we shouldn't have so we need docker ignore right uh oh no because i changed oh yeah oh yeah if i had done a docker ignore take note the docker file ah yeah okay all right let's see what we have in this directory now that's better perfect okay looks good let's deploy that i annoyingly have the kubernetes disabled i think we could have excluded the index php as well right if we were just gonna adjust the stuff that's well that wouldn't be that like if we if we were going to spend the time to work at the olympics and we need the copies but it would have been better but because we're just trying to get this working and not streaming for three hours let's get that quickly enabled i disabled it for yesterday's stream we were looking at the latest version of kubernetes and this is an older version so i had to so simply apply is we're going to apply it with the nginx container to make sure that we can pull those assets you still there that was weird so we have a kubernetes one kubernetes or one kubernetes and everyone get pods appointments there we go image pull back off click ok let's fix that um original quality and not present right so we'll only pull it if it doesn't have it already yeah so when you locally build images it can be a bit frustrating but the default policy is always which means it tries to go and pull the image and so if it's working on the official registry for this but if it's locally we're going to sell enough to do that which means it runs and so i'm going to put forward to it and if i hit here you get engine x but really what we want to see is what was it css absolutely cs6 oh it won't be called css anymore because i bet you it's minimized so let's just go and check that out again um what is that cso okay i guess we're debugging again let's go ahead and support did we copy in the config file or have we mounted in the config file uh that is a good point let's fix that we did not please uh at least in the default one i already spent ages working on that i felt it from here so i've got my old complete where's my build target engine x we're calling it ncrm engine exploded i really need to stop embodying that cash hopefully it's the last time we have to build it and then we'll delete the part force it to restart hopefully that is enough for us to hit up css then we'll build laravel and deploy that'll be good we'll get there when i'm debugging like this i often make sure i put my stuff at the bottom of the dockerfile even though even if it would logically be better higher up yeah and when it works resynchronic resync it and hit build and walk away and get a cup of tea uh why haven't we got crashed back off uh the upstream php doesn't exist and that's because in production we're not using an upstream php so let's create a vhost prod where we're going to use vocals i'll come into a docker fail and at least how many times i've had to rebuild this image this is my first stream that's when post two hours is that good or bad as far as people find it valuable it's good right i don't mind debugging things i think so i think it depends on your viewpoint when you're watching something like this either learn from seeing people debug or you get frustrated yeah i'm sure we'll get both again upon it both our audience will have both it's what makes it different from a slick presentation isn't it you get to see the frustration yeah frustrated definitely we get to see that even raw code has to debug things and makes typo i am sure i make more typos and debugging than most i break things a lot all right so we're delaying this part to force the new image to come into play hopefully that removes our failing correct we're going to do the port forward once before and crm we have our css being delivered yes and we'll just ignore that yeah that's better that's just cash okay so it's now trying to forward this request to fbm and it's failing we need to add our second container to the deployment manifest we want this which means we need to add a ci for peng crm fdm so let's copy this fpm and we're going to target means we now need a new layer uh we'll call us do the other php ones all right put it up near the other php ones why it's always a bit i always find it a bit weird having the notes you know a lot of different um images in one dockerfile at least put the ones that are php fpm together okay so we wanna from we're gonna do a copy dot code first and we can probably we want to do the composure install yeah yeah before you copy code oh it's build fpm and then from build fpm we'll do a test layer which we're just going to have to run the echo test for that and then we'll do it from build fbn as fpm so this is a production image and what we want is coffee i think we could just copy everything from slash chord to this last chord and then just shut that image uh okay so the uh inside build fpm you should swap you should move the copy of everything until after composer install yep yeah and we probably want to do composer install optimizes the autoloader but that will work sorry it optimizes the autoloader slightly faster and there's there's no dev or dash dash prod i think it is yeah prod so it won't install the dev dependencies like php in it okay right isn't it not having the tests uh so our test run is just like we'll just stick it in anyway let's see what happens so let's go back to our make file and we're going to build fpm um but first we'll run our tests which just means using a different target so yeah can you please check the location we're copying the code into on the fpm container is the same as the volume we were using in docker compose yep and in the fpm uh okay let's okay we're happy with it next we're gonna run make which is our fake ci spells fpn this would be our production php ci stack and we've got our extensions going composer install i mean this is probably going to take what one and a half minutes you got a joke for makita um what's really loud and sounds like a dog go on a dog it's attached really composer another thing i would maybe do here is just i know i go a bit crazy adding the layers and this file will become a little bit difficult to read because it'd be nice that the soundtrack's highlighted and recognized the from and made these like rainbow highlighted in some fashion well like yeah are collapsible um but i would probably encapsulate some of this like requiring composer and provide like a ci layer or something like that but for now i am okay with that prod doesn't exist you liar though let's go to fpm oh i said no first okay so the outline is that our ci would run the tests then ship the image off to a registry all right so there's there's a couple of ways i do this um there are different types of tests and you can talk about that if you want what i normally do is for unit tests is make them part of the image build process and so that the build just stops if they're going to test fields there's no point shipping an image anywhere yeah and then once i've got once a unit test pass and i have an image that works and then use that as a target for my acceptance tests or enter interest it's a bit more complicated isn't it because i don't i don't want to have the unit testing tools in my image no they wouldn't be in the image you would do that as part of the orchestration so you would run the image with an override of an entry point that you inject into it that installs the messaging dependencies that you need right right yeah that makes sense so i've done something similar where we build what did we do we built the prod image then we built another target off it that added the unit tests and its entry point was the test which is different to what you're saying you're saying run the test as part of the build maybe the last step of the build yeah just doing it and then there's all your acceptance tests which aren't you know the non-unit tests that you would then run on the after you've built the image you then run a set of acceptance tests on it before you push it out to the registry which would be things like ai level tests or um browser automation if you want to do a little bit of that or whatever i didn't see the error message what was it it's saying i don't have pdo hmm in bass when you're extending from bass let's see the error message again see that's something you added to composer.json so it's quite possible we've typoed it or it might be x-pdo underscore yeah annoying all right any more dad jokes no there's a comment in the chat from robert's roles cons my wife always says what's the problem what's the problem of upping the project and serving it to the public i should show this stream to her the part of actually seeing buttons and forms running is crumbs debugging and getting things to work takes most of the time i think i agree with that yeah definitely development is 90 debugging and actually i think it's quite an issue um a lot of people don't like doing it because it leads people to only ever write new code on nice green fields projects and the real valuable stuff is sometimes like bashing your head against the wall until you figure it out and being comfortable in that mindset yeah well that bill just failed there because we don't have get though i had to get oh you need gift for composer that makes sense i have excited the composer was sold locally which means we never had to see why you shouldn't run any of the commands in your local machine because you need to confirm that automation works yeah we didn't do composer install in your d-shell did we we did but because all the dependencies were there it didn't ever have to uh shell exec out to the command whereas in this environment does that's an interesting one okay that's a good point the docker uh the docker ignore files should have the vendor directory so it's never shipped with the context for clean builds and node modules uh locker speed it all up right modules uh it's called vendor yeah yeah docker file everything there's probably some cash stuff uh well do we not need storage database yeah we'll put it up for now but once that's built hopefully the tests pass an image we can add this we can deploy the kubernetes deployment again have those containers running side by side uh oh it's complaining we don't have the zip extension hopefully it still passes but we'll see i'll just add that just now anyway so that's it and i believe that actually just means that oh no it's just a command if you do prefer dist it won't need that so it needs to zip it because for some dependencies it will um download a table of the release you can force it to always do a git clone which might isn't what you want on prod because um that would also there's a git attribute that excludes things right and that's used to exclude things from the tarble version so some dependencies will end up shipping you their unit tests and their documentation and stuff if you forced this did you prefer the debt command was available in the extension before i started this build it wouldn't have pulled for forget anyway yes i think it might be falling back to doing a clone in all those instances and it'll be fine yeah i think it is but i don't know i don't know maybe you don't care about whether that's in your image or not i know some people who have their own scripts post you know composer posts and source scripts that goes through and deletes loads of stuff from the vendor folder to keep the package sized down tests and that sort of thing this is slow we should have used composer yeah i don't know if it's in the docker image yet but it does parallel downloads well hopefully it will be well we're close we're almost there i mean once we get this once we get this deployed you know this code is all available uh so git lab rock code php examples and uh it's on darker kubernetes does the well i haven't pushed it this is the slim stop from last week which is now in a slim directory uh and i will push all of this code with the laravel directory or the pencil m directory and i think maybe another good session for another time would just be maybe cleaning this up and optimizing it for production i don't think we're going to have to change to to do that today but we will get it working there's a lot involved in containerizing and deploying something to kubernetes it's non-trivial we've only been doing it for half an hour it might be interesting to deploy it to some cloud somewhere in a future session work through some issues pick a cloud well i mean i work for a cloud so we'd have to use them obviously so he's different we'd use it but that's the promise of kubernetes right is that you can deploy to whichever cloud you fancy oh yeah i mean urbanize is an api and if you deploy to that api the cloud becomes irrelevant unless you're consuming some managed services in which case of course you're going to get a level of vendor lock-in with each client yeah the kubernetes api being your deployment api gives you that freedom to move clouds if you wish and take on managed services when they do or as you said last week just do a heroku deploy and skip all this i know you're saying that sarcastically but um i think there's a lot of small teams don't need to be doing this stuff no no just yeah if you're looking at kubernetes i talk to you're deploying more than a laravel application yeah more than we'd apply today because i mean this repository's got a profile in it so you can just push it to a cloud provider this is really slow that's over five minutes now so should we talk about why max is slow uh well that's not why this is slow that's it in this instance though right because oh yeah this is just this is the build context which is a tarzip of the directory excluding the docker ignore files sent to the docker engine and the virtual machine so this is just slow because composers i mean we are using the get option because zip wasn't available we should probably slow it down uh it failed no so why is it failed um something to do so that's the class map optimizer that runs at the end having a problem why did i have a problem you know why i think that isn't a file or a c isn't a folder or a i fold the raw file weird it should be we copied everything oh no we haven't copied the code in yet so the optimizer requires the code to be there yeah just take out the dash show then what we can do take out the dash oh there but we have to rerun all anyway because this step failed but take out the o anyway because this is the way you should do it take out the that step and optimize it later and then after there's a dump autoloader dasho as a second step which i think is better in terms of cash and validation well i just hope that is much faster with that zip uh binary available otherwise you've got five minutes of dad jokes to tell i have to go soon i can leave you to finish the stream again oh you should you should put the zip on a separate line yeah those are quick though i mean that's going to be done in 10 seconds been running for 26 seconds see and hopefully this is much faster that it doesn't have to go over the get hub api yeah and i know some people who are making this a lot faster even in the uncached state by doing things like sharing the composer cache as a volume and using cache cache from to point to init at the previous image which has its composer cache populated and that sort of magic so there are a way of um speeding up composer installed yeah definitely it's a good way to do it using like the cash rom isn't very handy for stuff like that and we're committed we're committed now like why is it still updating dependency i mean it's not even at the fetch stage is it oh there we go yeah so these installs are probably the zips yeah these are much faster you can see it razing through the state and one of the improvements in composer 2 is that um resolution step is faster much faster and i guess even if it was using concurrent downloads and composer 2 i don't even think i expose many cores to my docker demon on mac so it might not even be able to take advantage of them i'm assuming it does the concurrency by spinning up multiple threads and having them each download one so it'd be parallel downloads would be how many cores i have again it depends if it's cpu band or network bound if it's io bound if it's sort of network io bound it might not matter how many calls you're using having multiple downloads at the same time i mean at the other end of the network well it was faster but unfortunately this thing why in composer json is there a post install hook yep i guess that's being true is the problem right yeah it's that deep flag to one so us leaving the flag out hasn't helped uh it doesn't really matter at this stage that's it let's just take the flag out of the thing the viewers will get the point oh but then we can't leverage the build cache when we don't modify the composer.json and i still think that's important so i'm just going to disable the optimizer and we'll do it explicitly because it's a deployment concern i don't think the composer.json should be set in that anyway i got root i will decide when to optimize my autoloader which means this is going to take 30 seconds to do update dependencies i don't know where that was the extension thing so this is going to take 60 seconds based on the last run well not too long hopefully it won't do the optimizing of the other loader and it'll do it on our manual step now just so it's really clear why we're kind of fighting with this if we had moved this line here they wouldn't have this problem but because composer install only needs to run when the composer.json changes we're trying to make sure that we leverage the build cache because your component updates on probably won't change often which means that we only need to do this yeah and that's what the reason it's not defaulted it scans all the files for php classes um and then builds a big class map i think there's a bunch of other stuff we could do to optimize php optimize fpm you can pre-warm the build you can pre-warm the up cache you can now uh actually i don't think you can do anything to pre-warm the new pre-loader cache but other things we could do to make the container more ready to serve stuff yeah you know the op cache the the in-memory cache you can give it a file based backup before that so if that file based back file based uh sort of second layer cache is present in the image it'll start faster and first start good for workers right let's just move it so we copy everything in we're not going to get to the bottom of this one of us can figure it out later uh it's almost therapeutic right maybe not quite all right i need to go in about eight minutes what should we achieve between now and then we'll get this working let's assume that works so next we need to handle the migration step at this deployment yeah this is how this works now there are two different um approaches to doing database migrations depending on what your team does so the first one which is probably the easiest one to get started is to use something called an init container within kubernetes which works like this we give it a name we tell it the image that we want to run now this is going to be our fpm one in this case and we can tell it to run a command so i'm just going to do echo because uh we have not deployed the database of kubernetes and we're running very low but you can imagine that what this was actually doing would be this runs in the container you've got a guarantee with the kubernetes api that and no containers for line 24 down will run until this executes successfully so does that mean in in terms of deployment what does that mean for downtime how does that coordinate with an existing running system so with our deployment in kubernetes we also have access to the update strategy where i can then say rolling update and then i can tell the max search let me just spoiler playlist for that so when we specify the strategy we can tell it that we're only going to accept you know one to be unavailable at any given time it also takes percentages so you could say that we're allowed to scale above uh if we've got rhetoric as four if we can scale above by 100 we'll launch four new ones and do the rolling upgrade that way or we can specify the maximum unavailable and say that we want to do an upgrade this way where we can we'll take one out deploy it don't work this also requires this approach requires that your that your migrations either do database level locking so that you don't have multiple containers trying to run at the same time or that there's going to be it also has to be idle important which is very important most database migration systems are if you're uh if you've already made the pre-meditated effort that your database migrations are always going to be additive you know you're always adding columns which is a really good way to handle database migrations which means that your older version of the code will still work even if the migration hasn't run yet which is a really nice way of handling this then you can remove i'm just going to comment these out so they're available when i push the code and then you can remove it in a container and allow this just to deploy um as is but before you do the upgrade in this you can use a job there's a special kubernetes type where we would use a job here and then that job would also have a spare um where i'm not going to deal with it i don't think i can all complete job stuff but and that let me pull up an example from one of the helm charts that i maintain so i use a job and this to configure the authentication and influx db so you set the api version and the kind to be job and then we just specify a container with a command that we wanted to run and that can just be a job that spins up asynchronously to you or before your deployment and runs the migration that way so let me try and clarify the two different approaches and why you need where how to pick or how to decide which one to pick so the only reason i would ever use a job approach would be that my migrations do not do any better database locking to make sure that two don't run at the same time in this case you have to as part of your ci run the job object first then do a deployment update if you're in a position where you do have the database locking which i'm assuming probably laravel has already got sorted with the orm then go with the edit container approach and tweak your max unavailable or search depending on the failure condition that you're willing to accept for that and whenever possible regardless of approach always try and use additive changes in your database migration uh yeah i think there's a point where you know that that's okay like we can take that like and the cache is there so it should be okay there's there's a point where if you've been adding columns consistently some of them are now redundant and you know there'll be a point where you know there's no running code using that column so you can you can then go out and take dump some old tables and you can even look at even in a massive system you can look at some of the access logs this table's never been queried in the last year let's delete it refactoring databases it's pretty good martin fowler imprint lots of patterns for um restructuring databases with forward and backward compatibility mostly involving lots of triggers so i was just trying to work out why my build cache wasn't used there and that's because as i was talking i was modifying that hold directly and that would have been cash are my chests failed i'm going to remove the test because i just want to get this deployed so yeah we'll come back to that hey my make fail isn't it there's an absolute calamity of it's good to show uh you know humanity oh yeah that's exactly what it is boom and i'm gonna assume that image is deployable anyway because that did build and that last one the tests were a different step right no i haven't purchased a production image because it was targeting tests when it failed not uh all right we just need to give it another 30 seconds and then we'll get the deployment done and we'll both be over here yeah uh i think we're definitely going to have to have another session on this yeah so much to cover so much to cover we'll build on this one right we'll make it better oh yeah we'll stick with this okay so now we've got our image built um so i can just do a super ply um kubernetes let's see blind's only got one container duplicating the name is that okay uh i'm not sure i've never seen it i've never tried that there we go out there yeah i name oh because it must merge all that yeah it must flatten it okay so pillow damage and that's because i forgot the policy on our fpm one let's try again let's just run a watcher on it okay both are running oh and theory we're now at the portfolio again so two containers one pod okay so now the logs director needs special permission bro why didn't that happen in previous oh because we uh slumbers log into standard out so the problem here is that we've not configured laravel to log the standard out but why can't it find that why can't it write to that file i think amazon this directory doesn't exist let's confirm is it gonna be file permissions or something like that we'll be exact into this container we need to specify it's the fpm one and we want to run batch what do i call the container uh pencil mmf porridge there's draken trip does the container run as root it does yeah oh no that's a log from that's been mounted then because there's not a docker a docker ignore fail that's given us well as last time i have to go now i've reached the last possible point where i can get to my let's just yeah well let's just call it here let's pick it up again and next week from here and see how far we get because like i said we've got a lot to cover and there's no point in me trying to rush it yeah we're quite close we've got some working images working containers they work fine for dev we just we've only spent an hour trying to deploy it into production and i think that's a reasonable time scale to fail in let's do it again all right yeah i will push this code just now it's available raw code php hyphen exam post kieran and i will be back again next week to continue our efforts to deploy a production marvel application on kubernetes hey kieran thank you for joining me again it was a pleasure and i'll see you again next week you

Channel: Rawkode Academy

Views: 2,897

Keywords: Docker, Kubernetes, PHP, Laravel

Id: CneJf4Amv0U

Length: 163min 20sec (9800 seconds)

Published: Tue Sep 15 2020