Laravel Worldwide Meetup #10: Pest to Perfection and Laravel Sidecar

hello everybody and welcome to another larval meetup we had a little uh break uh during the summer i very much enjoyed that and i hope that you had an excellent holiday as well if you went on holiday in that period i feel refreshed and i'm ready back for another series of awesome meetups and i'm very excited uh by this one we again have two wonderful guests here later on we'll have a talk by luke downing about best and i've already seen a little preview and there are some amazing things to be seen there so if you don't know best uh yeah this will be a very nice though fast introduction and if you already know best yeah then you're going to see some very cool stuff first but first we have another guest i have with me iron francis and he's going to talk a little bit about a package that he made called sidecar so let's have him on hey iron how are you doing man hey good thanks for having me yeah i'm glad that you could come on i've heard lots of things about the things that uh that you are creating like uh torchlight i saw a series of tweets by you but with some amazing robots i think which some handwriting robots yeah man that totally blew my mind you know um and of course what we're going to talk about uh today uh and that is and that is sidecar now before you head into head into the talk i'd like to ask you one question um yeah how did you um got started with the idea for for a sidecar yeah so i was um actually developing torchlight which is a syntax highlighting tool and i needed access to node and i was running torchlight on vapor and so i didn't have access to node i could have you know bailed out and done the container thing but i really didn't want to do that and so i was trying to figure out how can i run how can i run node without having to set up infrastructure and that's kind of where we ended up with sidecar cool so it's really born out of a need of of your own um personally i think that the best side projects are just born that way because you want to scratch your own itch um okay man is it okay if i show the people your screen yep let me switch over to slides okay yep and if you need anything just chime in i won't be watching chat so just talk to me if you need anything all right all right okay aaron the stage is yours take it away man have fun okay thanks um well thanks for putting this together frank this is um my first live talk live coding session so i'm super stoked to be here um so like he said we're going to talk about a little bit of lambda um we're going to go over um a package called sidecar that i wrote but before we do that just a little background on who i am um you can find me on twitter at aaron d francis i'm a software developer in dallas texas here in the united states torchlight is a software as a service that i've been developing that does syntax highlighting and then the package that we're going to talk about today is called sidecar and you can find that at hammerstone.dev sidecar more important than any of that is i'm a new dad to twins so i've got two four and a half month old babies so that's probably the most exciting thing for me but not what y'all are here to talk about and so congratulations yeah thanks it's been a wild ride um okay so let's talk about like what the actual problem that we're trying to solve here is and what we're trying to solve for is we have all of this great hosting optimized for php and we have a couple that are like specifically tuned for laravel which is fantastic and doesn't seem like a problem um so we have you know forage and vapor forge plus envoyer and vapor and those are hosting providers that basically give us like more or less one-click deploy and your laravel app is live to the world so this is you know fantastic news um makes my life easy i don't want to be the server guy i'll let you know fideliber be the server guy i just want to be able to deploy my app and see it live and we've got a couple others that aren't first party um that are probably very good but i only use the first party ones because they're they're just so perfect so with those first party um hosting it's great when all you need is to live inside a larabel application and it does so much for you but what about when you want to do server-side rendering like with inertia's new ssr coming out soon you're going to need node to be able to render to do the actual server side rendering or if you wanted to orchestrate headless chrome or if you wanted to do some kind of you know 3d rendering massive parallel someone is using sidecar to render like 700 3d renders all at once um and just kind of blows my mind but when you get into these situations um you either have to take on the burden of installing and maintaining these other languages or you have to um you have to figure out a way to work around it right so and then with vapor like i was talking about you don't really have access to the underlying system so with forge you can get in there and install node or python or ruby but then you're maintaining node python or ruby but with vapor you can't even like you have to you have to bail out and do the container you can't use the native stuff anymore so that leads us to a place where we may have perfect laravel hosting but we need to do one other like we need to do one other thing like just use a single node library for a very particular setup so the solution would be you could use a whole lot of lambda right so you could use lambda as it was intended to be used and um you know for a little bit of background lambda is um amazon's like serverless functions you know functions as a service kind of thing so you just deploy just specifically the code that you're wanting to run and all of its dependencies and then you let aws handle the rest you don't care like you don't have access to the server you don't care about the server a request comes in they'll make sure that your code gets run and they'll send the response back so there are you know there there are good ways to use lambda but they're all pretty heavy the serverless framework is good but then you're in this situation where you've got your laravel app um in one repo and then you've got your serverless app and another repo the deployments are run separately um they can get out of sync both in the deployments and the development so you're trying to keep these two these two things in sync like there it's all one application but now it lives in two places it's deployed in two places they're talking to each other usually over the public internet so you've got you gotta set up api gateway you gotta set up you have to secure your lambda so that nobody else can call it only you can call it and there is versel which takes away some of that um but still you have a lot of the same like separate separate repos separate deployment processes public internet all that kind of stuff so what we're going to talk about today is using just a little bit of lambda through sidecar to solve this problem of i need a piece of node and so with sidecar the lambdas live inside your laravel application they're deployed alongside your laravel application and they're executed only through your layer bell application so instead of having two repos you've got one instead of having two deploy processes you've got one and then you don't have to secure like an api endpoint because there's there is no api endpoint there is only executing it through laravel and so it takes like it takes the complexity of like vanilla lambda and kind of turns it into the ease of use of vapor um but for everything that's not php right so vapor handles php and then sidecar will handle everything else and sidecar can run anywhere it doesn't have to be vapor and sidecar doesn't specifically doesn't manage api gateway databases resources anything the only thing it concerns itself with is the packaging deploying and executing of lambdas so as taylor would say as dhh would say show me the code so let's take a look at some of the actual code of how this stuff how this stuff works so we can see how easy it is so i have i have here a vanilla just plain laravel application i have installed so i have composer required hammerstone sidecar so that is um the package that we're going to be talking about and that's the part that manages all of this behind the scenes for us so first thing we're going to do is artisan sidecar install and that's just going to publish a configuration file for us there's a lot of stuff here and we're not going to cover we're not going to cover hardly all of it the documentation is pretty thorough i tried to make it as thorough as possible so where i skip over stuff it should be covered in the docks but let's look at the sidecar um configuration file so all of this stuff is um just setting up some normal defaults like the amount of memory um how long there should be allowed to run that sort of thing what we care about right now is this function registry this is where we're going to put all of the functions that we create so that sidecar knows okay i got to deploy you know the render og image and i got to reply deploy process thumbnail so those don't actually exist so we're going to get rid of those okay what we're going to do is we're going to deploy a basic hello world um lambda and so every lambda requires two things we're going to call this basic and we're going to call this index.js every lambda requires two things one is the package um of files that you want to you want to upload to lambda and the other is kind of like a configuration a php configuration class so for this one i have a little bit of code here all right this one is just going to say hello from lambda this probably looks pretty familiar to people who have done some javascript what we're doing is we have a hand or an export named handler and it just returns the string hello from lambda that's all that this does so we're going to create so that's the actual stuff that runs on lambda and now we're going to create the laravel side of that so let's create a new class called what if we call this basic basic okay new class basic extends lambda function so sidecar ships with an abstract class called lambda function and it has all kinds of stuff under the hood but importantly it enforces that two methods must be there so we're going to have phpstorm implement those or step those out and these are the two things that i talked about the package and the handler so for the package check the comment and the parent class it says all the directories and files needed to run your function okay that makes enough sense we're gonna say um since we are we are in sidecar basic index let's just deploy um sidecar basic we're gonna deploy that whole directory and then our handler is it has to follow this specific format that aws requires so it's the path to the file and then the named export so our file is called index.js and our export is called handler so we're going to say index.js you leave off the extension and then the named export handler so that's it that's all that's required so now sidecar knows what to ship to aws and aws knows what to call when it's executed so we need to add this to our registry we're going to say class okay so now sidecar knows those are all the functions i have let's jump back over here and pray to the aws gods red card deploy dash dash activate hurdle number one okay what just happened so sidecar said i'm gonna deploy that function that you registered app sidecar basic i'm going to deploy it to lambda as sidecar name of my application environment is local and name of the function all of that's configurable that's just the default then it gives you some helpful information about kind of what's going on here the environment is local sidecar will separate it by environment so your each person you know each team member can have their own environment staging can have an environment and production can have an environment we're deploying a zip file you can deploy containers this just happens to be a zip the run time by default is node 14 this lambda function did not exist and so it had to create a new one and then it packaged everything up created a zip file put it on s3 which is you know a requirement and then because we said activate it activated the function so that we could start using it if we didn't call activate it would just be sitting there waiting to be activated and you couldn't use it yet i'll explain that later so creating alias for version one which means okay the version you just uploaded is ready to go let's take a look and see so i'm in my aws console hopefully there's no secret keys here i'm in my aws console you can see what sidecar told us just happened did in fact just happen so last modified one minute ago click in here you can see sidecar basic index so so far sidecar has packaged and deployed an aws lambda so let's see if we can make it execute so that's out there on aws now let's jump over here and instead of saying hello we want to run we want to run the thing we just deployed right so let's just see if with any luck this is hurdle number two hello from lambda live coding is working okay so what just happened is laravel called a lambda the lambda executed returned the response and the response came back to the browser which like seems kind of insane so let's just prove once again that that works so what we're going to do now is we're going to pull um we're just going to pull a random aws environment variable out of their little um environment and plop it into our string just just to like be triple sure that we're not running this on our local machine not um okay so we've updated the handler code which means we need to redeploy both the um function you can see lickety-split it is done this time it said the function already exists potentially updating it created a new zip and then it said activating version two so that's that activating flag we needed to make sure that the new one we deployed is now active so hello from aws lambda node.js 14.x lambda so now we're like 100 sure we're running on lambda we pulled an environment variable out of the aws environment and returned it back so at this point we have the pipeline totally proven we've been able to write a javascript function locally deploy it to lambda and call it through laravel um it gets even wackier um so if we do let's see what do we want i think it's runtime so by default if you look in the parent class i just set nodejs14 it could have been anything node seemed reasonable enough but if you look on aws's documentation you can run you can run all kinds of different stuff right so if you for whatever reason i'm not here to judge if you wanted to run java 11 from your laravel application or net core 3.1 i don't even really really know what that is um if you wanted to do that you could so we're going to run we're going to run python just to prove that so instead of back to our function instead of whatever we were just running which i think was node 14 we're going to run python 2.7 so that does mean index.js is going to be no good because it's going to be looking for index.py so i had to find this on the internet because likewise i do not know python um now our handler is that it is a python function and it's going to say hello from python and then do the same thing it's going to get the execution environment out so let's look to make sure so we're deploying the whole package so our or the whole directory so our index.py will go it's still named index so we don't have to worry about that and it's still called handler so we don't have to worry about that so if we run this again you'll see here the runtime was changed to 2.7 and version 3 is now active and that should there you go aws lambda python 2.7 so now you're running oh i'm running now you're running python without having to have python installed on your machine which is pretty nice um so let's look at something a little bit cooler let's close all of that nonsense let us generate an image um so once you start like once you start to realize like oh i could i could run anything from anywhere then that's when your mind starts to go kind of crazy and be like well what do i like what do i want to do um so in this case what we're going to do is we're going to generate some images on lambda and send them back to laravel so in this case this is um just a very tiny very hideous bit of javascript that i wrote that's going to use node canvas to basically just draw an image and graphic design is my passion but this is this is a hideous image so just be aware um one thing that we do here is we add a font called helvetica we add it to the canvas so that we can later use it but we need to deploy that font alongside it so i have those fonts backed up here it's just all the true type whatever so let's copy that and put it in here so this is showing you that you can like you can deploy anything you want there are certain size limits but you can deploy anything you want and then reference it in your lambda so it doesn't just have to be the handler it could be an entire node modules folder if you wanted to all right so this is our handler what it's going to do is it's going to generate an image with the text hello lww on it okay that's the handler side let's jump back to the php side we'll do new class and we will call it image and we'll say extends what's it called lambda function add the stubs in this case we're going to use a little bit different there's the array format um but you can also do package make set base path to sidecar image i believe is what it is so that's the base path and then we just want to include just everything that's in there so just deploy everything and because now we've kind of changed the base path this parts gets easier so we just do index.handler instead of the whole you know file file file folder thing index dot handler good okay index.handler package is sidecar image sidecar image deployed the whole thing let's jump over to our registry i'm going to turn this off you can deploy as many as you want but just because it's a demo i'm going to turn that one off okay so now we are ready to hopefully deploy our second lambda let's clear that out deploy and activate there goes image local node alias okay so we're all live and ready to go now let's see if we can make this work image execute with any luck this will be a base 64 encoded oh so it says um it's ginormous but it said it can't find module canvas i forgot one very important thing fortunately i know what um so i said up here we're going to require canvas i didn't ship my node modules folder but one thing that you can do is you can reference you can reference layers and what layers are is there little extra bits of code either provided by you or provided by the community that aws will add into your package for you so in this case what we're going to do is somebody out there in the world has um somebody out there in the world has compiled node canvas specifically for lambda and so we're just gonna we're just gonna take advantage of that um so i'm gonna tell sidecar i've got these two extra layers and those layers are compatible with node 12 and i don't want to recompile them so we will deploy activate one more time so now it's going to have to add the layers activating now node canvas should be available great so it kind of worked but that's not an image base64 so which is what we expected because if you look at the handler here it returns a base64 encoded version um so what we're going to do here is we're going to say that the result equals the execution saved [Music] the result equals execution and then we're going to base 64 decode it and then return it with the proper header base 64 decode return with image jpeg we don't have to redeploy because we only changed the php side so like i said graphic design is my passion and here's this beautiful image that i've come up with so what happened is the browser hit laravel laravel hit lambda lambda drew an image gave it back to laravel gave it back to the browser with the correct headers right um we have added a nice little way to kind of like encapsulate this so it would be nicer to just return the execution right so these classes do i forget what it's called um i think it's two response yeah so if you in your in your sidecar class if you're always going to be generating an image you can just always return an image and i think that is the right way to do it to response to response there you go so now we're back to the nice neat i just want to return the execution part so we can get even cooler if that's if that's not cool enough by taking in information from laravel and using it in our lambda so we had this hard-coded hello lww we're just going to pull a value off of this event here and this event is provided to us by aws um but the way that sidecar is set up you can send stuff through that event and it's just ready ready to go so we did change a handler which means we need to redeploy and it will notice that something has changed so it's creating a new zip file it does like a md5 hash of everything to make sure that stuff has changed so image number three yes we should see undefined here classic javascript um so then what we can do is just say text hello from laravel let's be excited about it okay okay so now laravel has sent along some extra information lambda has picked it up used it and sent the response back which of course means you can do anything you control both of them you control laravel you control the lambda so you can set it up to do whatever you want which i think is a good yeah let's look at the next one because the next one gets even a little more sophisticated so if we do we're going to create one um we're going to create a lambda that is going to drive headless chrome through puppeteer take some screenshots and send it back so let's do the same thing extends lambda function add our method stubs um this one because it's driving chrome it it's going to be a little heavier so we're going to up the memory on this one to 4098 um and so with with aws lambda the more memory you give it the more cpu you get it's a wacky calculation but i promise that it's true um so let's see we have another layer here someone out there has built chrome specifically for lambda um so you can look at it here google chrome for aws lambda as a layer it i mean doesn't get more perfect than that so we're gonna use that um so that we don't have to compile chrome for aws lambda so we're going to use that our package is going to be let's just copy this we're going to use the same package make set it to sidecar screenshot and include the whole directory and there's currently nothing in that directory so let's do index.js that's where we're going to put our handler and because we set the base path then we can just use the file the handlers index.handler sends the whole screenshot directory it's more it's heavier so we need to use more memory and use the chrome layer okay i think that's good so far and then the actual um the actual handler is another similarly complex um so we'll just walk through real quick what's going on so we're gonna pull chromium out and that comes from the layer so i'm not having to ship node modules or anything which you can do um but that comes from the layer so the layer will provide chrome aws lambda i'm just going to require it then we're going to get a browser instance by launching it setting up some stuff and then in our handler function we're going to set up the viewport we're going to go to whatever url we pass in so that's going to come from the laravel side and then we're going to take a screenshot and send it back so here's the screenshot image is await page.screenshot and we're going to send it back along with some extra you know meta information so we can use that back in laravel if we need to okay so i think we're good so we've got the whole handler it's named handler it returns the stuff memory layers package handler and let's just turn that one off screenshot class so now we're registering it to let sidecar know we need to deploy this one okay screenshot creating new functions alias version one okay good so far so now we can do um let's say let's just go ahead and do this let's just return screenshot execute url let's see if google allows such a thing um coming from lambda you may not be able to access everything but um if they allow that then we're good we do need to implement the two response method because if you look at what we're actually returning from the handler we're returning a base64 encoded image but it's also nested in this object and so we're going to return we need to basically do the same thing we did last time which is turn it into a real image so let's do two response same deal as before um the body is the result which is passed into you from sidecar then you base 64 decode our special key that we set it as we set it as image um then we're going to return the image with a header of whatever type of image so in this case you know by default i think it's jpeg okay so now let's see if we can execute a screenshot headless chrome i'm nervous headless chrome on lambda so it is a cold boot so it's going to take a second and then it's going to have to launch chrome for the first time but ta-da it's a screenshot um so laravel called lambda lambda launched chrome chrome went out to google.com then puppeteer took a screenshot of that base64 encoded it and sent it back to laravel so i mean we can it's controllable right so now that it's no longer cold this should be faster so it's not cold booting um the the container and it's also got an instance of the browser cached and so it doesn't have to do that again so you can see that the time gets a lot faster and depending on how you set up your handler you can do all kinds of crazy things like you can make your handler as dynamic as you want so our handler in this case we did here we go let viewport equals 1200 by 630 with a doubling for retina but also destructure event event.viewport so if you send in viewport it's going to overwrite these two things so then on your laravel side you can just say ah you know what i want it 400 by 400 and it should come back as 800 by 800 there you go because it's doubled it's pixel doubled um so i hope that gives you some kind of like pathways to be thinking about how dynamic like how dynamic can i make my handler such that i don't have to keep redeploying it all the time i can just deploy it for big changes and i can control everything from the laravel side so last one i want to show this one is going to be super quick um i want to show one super nice thing that sidecar can handle for you all that we've been doing here is kind of like request response like do a thing give me the response do a thing give me the response some people i asked on the sidecar repo i asked some people like hey what are you using sidecar for because my um i'm just like generate an image and some people were like i'm spinning up screencasts i'm rendering i'm using python to render 3d models um [Music] so one really nice thing that you can do with sidecar is well let's just show you let me just show you so this handler all it's going to do is return a uid so it's just going to return a super long random string um but before it does that it's gonna sleep for a full second so it's just gonna make a promise that resolves after a thousand milliseconds so this you would expect will take about a second right so we can copy the basic one and call this random random random random and we'll let that default to 14. okay so we have a random function in the random folder with index.handler um handler and it's just going to return a string random sidecar random sidecar random let's add it to our registry here random and let's deploy this random creating new that was fast um okay so return random execute we would expect this to take about a second the first one is going to take longer um because it's cold booting so 1.2 1.2 1.2 yeah that's about right now if you wanted um to get five or i don't want to wait that long if you wanted to get three of these things you would do it like this but this is i mean this is gonna take gonna take three seconds or more so you've got three functions you can see three uids all coming back it takes about three seconds which if you're doing like one and one one and one it's fine just just wait for it but if you're trying to you know if you're trying to spin off like 80 lambdas you don't you don't want to wait for it so we have a way let's do um let's do execute many and instead of executing three we're going to execute let's say 10. so let's get rid of this we're going to say the results equals 10 executions and then i think we do return collect results higher order map pull out the body so what this is going to do is it's going to execute 10 lambdas map through the responses or the results and pluck out the body which should be the uuid um let's watch the time so we just got 10 of them you can see them all there in the same time more or less that it took to get one right so we're doing 10 and 1.3 seconds versus 1 and 1.2 seconds and this is when i said there's some guy running you know 700 concurrent this is what i'm talking about this is what he's doing he's running for whatever reason he's running 700 async executions right so that's really useful um in cases where maybe you're doing a bunch of different things and you can pass through i mean you can pass through different payloads this function doesn't accept the payload so it doesn't really matter but you can pass through different payloads and they will all get they'll all get executed in this case it doesn't do anything so you can pass through as many as you want fully loaded payloads and everything and what sidecar is going to do under the hood is it's going to dispatch all of them and then it's going to wait once they're all dispatched then it's going gonna wait for the responses to come back and collect them all together if you wanted to you could also um let's say you wanted to execute 15 async um in in this case it's not going to wait at all so it totally breaks out of the request response cycle they're out there running on lambda but it's not going to wait for them so you can see we're getting it back in default layer of all times so you can do execute many async you can do execute async and pass through a payload so there are tons and tons of different different things that you can do with sidecar even even just in terms of execution not to mention in terms of deploying a container deploying any sort of runtime that you can find um anything like that you you can do so hopefully this gives you some kind of insight into like the next time you're trying to reach for just just one node or one ruby or one python script you're not thinking well now i gotta go set up vercell or i gotta go figure out the serverless framework hopefully you can look at this and see okay i can keep it within my laravel stack like i'm just doing one thing i can keep it within there and you don't have to you know jump to microservices because you're forced to set up node there's a bunch more the deploy activate thing is so that you can deploy um from ci and then activate from production so that you know they stay they stay like exactly in sync so there's all kinds of other great stuff we can do pre-warming so you can pre-warm 100 containers like if you're doing server-side rendering and you deploy a change but you don't want your page speed to drop you can pre-warm 100 containers and then activate it so you don't have the lambda cold boot thing so take a look at the docs there's a ton more that is basically all of that i have um i have a slide that says that's all that i have so that's all that i have so frank back to you yeah this was uh really mind-blowing aaron really uh let me show you some some stuff from from the comments here and it's got all over here man that is so cool whoa thanks amazing stuff and here another one and i totally agree with this this is some lara con level stuff so i think uh i think you nailed it uh in this stock oh thanks it was my first talk my first live coding experience i'm just so happy that aws didn't crap out on me yeah i was saying that in the comments here as well that it's really courageous to do that because yeah if at this moment aws would crash and people would have seen this this awesomeness yeah personally i i really like yeah how polished that this package feels um you know you showed that to response methods and i thought yeah that's something that you added for yourself and you saw that yeah this is very nice for uh for others too well what i'm also thinking is yeah you showed us uh a couple of cool uh use cases already uh like yeah visiting url getting a screenshot generating an image i'm thinking and maybe it already is is out there wouldn't it be very nice if those would be also packages as well for if people just don't want to create an image there's already a sidecar package for that with a decent api on the on the lambda function yes if you create if anyone creates a sidecar packaged up function let me know i will put it anywhere i can as publicly as possible because i think having those off the shelf ready to use is going to be extremely nice for everyone yeah i think so too i'm also feeling that this this stock it would also benefit from like a part two with like how does it work behind the scenes yeah a deep dive it's kind of fun yeah i can i can imagine i can imagine like zipping the stuff and setting it to able uh aws so yeah there's some there's some cool stuff in there like the zip file never hits your disk it uses streams so it does a zip stream and then sends it off and yeah it's fun maybe we need to talk about the part two [Applause] but first you should rest a little bit and just feel happy with yourself because you did really an awesome job here i think thanks i appreciate it thanks again for having me is there something else that you want to pluck uh no just aaron aaron d francis on twitter um let me know how you use sidecar okay cool thanks again aaron and until the next time bye okay people um that was the first talk i hope that you liked it as much as i did we are going to have a little break now and we'll be back with luke stock in just a few minutes enjoy the break [Music] so [Music] [Music] so [Music] so [Music] so welcome back to the second part of uh the meetup uh yeah i'm still um amassing my brain back because my brain exploded uh in the last stock but the next stock is also going to be very good um the next stock is by uh luke downing uh somebody who's uh helping out with best uh a lot uh personally i've been using best and discovering past and discovering discovering best while i was recording a course on testing and it's clear that luke has had many useful contributions there now when i invited luke to the meetup it was supposed to be a a live dog but luke had to travel to a place today where there isn't a stable wi-fi connection so we decided it would be safer to just record his talk so we have a recording of uh his stock titled let me let me see best to perfection if everything is correct then luke is in the car in the in the chat so if you have any questions for luke uh during the talk just chat uh with him okay with that being said let's take a look at luke stock here we go hey everybody i am super excited to be here thank you so much for coming along and listening to my talk i'm thrilled to be talking about pest again it's amazing to be able to even speak at conferences and i just want to say like i know six months ago i would never dream that i would be doing this today so it is insane that i am able to thank you so much to everyone who has a part in organizing things like this is amazing for the community it really gets us all excited about uh code and laravel and open source and all the rest of it so thank you so much for coming along and supporting my name is luke downing you can find me on twitter at lukedowning19 i'm actually currently on holiday so this talk is pre-recorded but i'm in the chat so i'll be on the chat talking now if you have any questions ideas suggestions thoughts drop them in the chat i'll respond in real time so in a way it's even better than a live talk failing that this is pester perfection i really hope you enjoy just a little bit about me in case you don't know who i am i'm a developer from the united kingdom i own my own company downing tech but i freelance for lots of companies uh primarily i'll be starting freelancing for work some soon so i'm really excited about that and no doubt i'll be talking about that on twitter in the near future i've been using php professionally for around eight years so i'm quite familiar with php and i picked up laravel around laravel at five so i've been using laravel for a little while too i'm a member of the past php core team so i help daily with maintenance and adding new features so i am a little bit biased with this topic i guess but i want to say that i wasn't uh sorry i i how can i put this i wasn't invested in php or past php from the beginning i was actually added as a core member fairly recently after i picked passed up really enjoyed it and started creating content and adding features to pest so i guess the bottom line is that i am absolutely a lover of open source software i think it is incredibly important i think it's the reason pretty much everyone here has a job and i don't think that programming would be anywhere near as enjoyable without it which is why i think it it's really our responsibility to look after and care for open source it's something that's been placed in our hands as an amazing tool we have to care for it we have to put the time and maintenance and effort in so if you've not jumped into the world of open source yet into contributing and helping maintain things try it just try it it doesn't have to be anything big and i guarantee that you won't feel you've wasted your time i think the most important thing here is to explain what pest php actually is you've probably heard of pest if you haven't then i'm sorry hopefully i can explain exactly what pest is in this talk but the bottom line is that it's really a testing framework that sits on the shoulders of php unit so phpunit is the testing framework that comes out of the box with laravel and it's super powerful like really powerful and it's very stable i have never experienced a bug in php unit it is extremely stable and that means that pest has an amazing head start now what does pest offer the php unit doesn't well pest rather than going for classes and methods instead opts for a just like functional syntax so you don't write methods you write functions and that removes a lot of boilerplate around your test code it was created by nuno maduro so new works for laravel if you didn't know and that means that pest is tailored towards the laravel community now it can be used in bare php it can be used with cake php you can use with symphony it can use any php project but it's tailored for laravel and you'll see that as you start using it it becomes very obvious that nuno was thinking of laravel users currently it has over 600 000 installs and that number is growing rapidly i think we're entering the pest golden age so i just want to say go pick it up you don't have to use it professionally if you don't like it fine but at least try it don't be one of those people who just hate on something for no good reason go try it in a personal project i think you're going to absolutely love using pest this talk is split into three sections the first is writing a basic pest test if you've never used pests before how do you get started what's the first test you write so i'll do that for us and i'll walk you through how to write a pest test we'll then refactor that pest test to use higher order tests and i'll show you how that can really make your testing code more readable and easier to maintain in part two we're going to refactor one of our tests to use data sets instead of repeated code and this is a really powerful concept it's similar to data providers in php unit although i will say there is a lot of boilerplate to php unit providers data sets are super easy to use i use them all the time because of how quick it is to build one out in part three we're going to go a little crazy and we're going to refactor some code to domain specific language now if you're not familiar with that term don't worry it's basically a fancy business term for an api that is surrounded by the business logic so i think um alex from the past core team put this best in this little snippet i have a higher level more expressive api based on your business terminology it's easy to read and understand not just for the developers but also for the rest of the team so i want to show you how pests functional style plays perfectly into the hands of a domain specific language i think you're really going to enjoy that section of the talk but before we get there let's jump into our project and write some initial pest tests in this project we have a web.php file as most laravel projects do and you can see we have three routes a login route a register route and this profile route here as you'd expect the login and register route are behind the guest middleware so basically if you're a guest you can see these pages if you are not a guest if you're authenticated then it redirects our profile route is the opposite it's behind the auth sanctum middleware which means if you are a guest you are forbidden and redirected to login whereas if you are a user you can see this page however we have this custom secure username roots middleware and this is going to ensure that only you can see your profile so each user has a unique username if your username is not this here you won't be able to see this particular profile so george couldn't see luke's profile for example that's a basic overview of this application it doesn't do tons but it's enough to get us started with pest i'm going to head over to the terminal here and i'm going to create our login test file and as long as you have the laravel plugin installed then you can run php artisan pest colon test and we'll call this login test and you'll see that pest goes ahead and creates the login test for us which is super useful let's jump into that login test and we'll see that pest has actually already set up a test for us let's see if this works i'm going to run pest which is an alias i have for vendor bin and you'll see that we actually have a passing test which is amazing we didn't write any code we already have a passing test and this works because basically the login route has no middleware it has no authentication and anyone can access it which is why we get a response of 200. so while this is awesome i actually want to write our own tests so i'll remove this base test and we'll start with the it function which is basically uh one of the test functions you can use in past there's also test but both have the same arguments the first is a description of the test so it can be accessed by a guest the second is a closure and this closure is going to contain our testing code so inside the closure we can make use of anything in the php unit test case so any php unit code more or less is valid inside this closure that means we have access to this because this has been rebound to the closure which means that we can say this get root login and assert that it's okay and if we run this you'll see we have the same passing test awesome that wasn't too difficult was it let's write our second test it cannot be accessed by a user we'll open up our function again and we want to say this and then we need to authenticate as a user and we can use the b function or method in li reveal for that we need to pass in a user so let's say user factory and instead of using create i'm actually going to use make so that we don't persist to the database in this case we don't need to it makes our test faster so yeah this be a user factory make and then we want to say um get the root login and assert a redirect and let's just break this onto a few lines so that it's easier to read we'll run our suite again and now we have two passing tests well that was easy we were able to do that very quickly and we have now tested that this page does what we'd expect but before we copy this over to the register root i think we should spend some time refactoring these to higher order tests now i want to just talk about why we do this some people are of the opinion that you should just write your tests and then move on that you shouldn't spend long writing tests i actually disagree with that for the same reason that i would say it's always worth it to refactor your code you are going to have to maintain your tests for the lifetime of the application and often the tests actually live longer than the underlying code so in my opinion it is paramount that you spend time making sure your tests are easy to understand are easy to maintain and make sense not just to you but to other people so higher order tests in pest php give us that superpower they give us the opportunity to really think about our test code they also remove a lot of boilerplate which makes it easy to read so let's refactor our first test here to higher order in order to do so i just remove this closure i close the it method out and i'm going to remove the boilerplate from the bottom of this here we'll remove a call to this so that we end up with a chained method call it can be accessed by a guest get the root login and assert okay this is a higher order test however as you've probably already guessed when i run this i get a failure the failure is that the target class url does not exist and this is actually happening because it's attempting to execute this root helper which requires the laravel container to be booted but at the time of it being executed laravel doesn't exist because create application in our base test case hasn't yet run so we need a way to defer the execution of this route helper well you probably know that in php we can defer things using closures and that would fix our problem but if we run this it's going to say that argument 1 must be of type string but a closure has been given so how can we fix that well seeing as we have access to the test case why don't we create our very own version of the get method in laravel let's go to that test case and let's say i want to override get and you can see the phpstorm has shown us that this is indeed an overridden method and we will return parent get just like that and i'm going to pass in uri and headers but instead of just passing in the uri which would be a string i'm going to say give me the value of the uri and in case you didn't know value returns the default value of the given value it's a laravel helper function that you should be using all the time because it's so handy you see all it's doing underneath is checking if it's an instance of a closure and if it is it executes the closure otherwise it just returns the value in this case it basically means that when we run it it all works now it can be accessed by a guest it cannot be accessed by a user and if we come back to our login test we have a beautiful higher order test if you compare these two tests i think you can see why taking the time to refactor the higher order is worth it how beautiful is that first test let's convert our second test to higher order so we remove this closure let's get rid of the boilerplate at the bottom we'll remove the call to this and then we'll reformat now we already have handled this get here we can pass a callable to that but if we run our tests again we have another error unknown format first name and the reason is exactly the same as our previous error but just on a different method this time b the factory requires laravel to be booted and at this time it hasn't been booted so we may think well let's override b and let's allow that to also be a closure we can do that but i actually think there's a better way in this case seeing as we're going to be authenticating as users a lot in this application it would make more sense if we created a helper method called as user and let's allow us to pass in an optional username so if i pass in luke it will create a user with the username of luke okay let's make that happen so we'll get rid of this call to be we'll say as user and then we'll go back into our test case and we'll create the as user method as user that accepts an optional string of a username and it's going to say this be user factory and we can use the state helper to make this work if there is a username then pass in the username otherwise pass an empty array and then please make the user okay i hope that makes sense basically we've removed the manual call to creating the user using a user factory and we can simplify it to a call to as user where we can customize the username if we want if we now come back here and run the tests again you'll see that they are passing and that means that we have actually refactored both of these tests to beautiful higher order tests i hope that you follow i hope you can see why refactoring to higher order tests is a good idea and how with a little bit of tweaking we can actually build out insanely useful helper methods that increase our speed when we're writing tests later down the line i think we're ready to move on to our redirect tests so let's say php artisan pest test i i did say redirect my register register test and if you don't like pest test and you rather use make test which is probably what you're used to with php unit don't forget or yeah don't forget that you can customize stubs in laravel so you can customize the test stub and just replace it with a generic pest file okay so if you if you prefer to use php artisan make test that is entirely possible i'm going to copy this code here and we'll open up that newly created register test drop our code in and remember to replace login with register and if we run pest again we now have four tests two test register two test login now it's a little bit annoying that we had to copy and paste our tests it can lead to false positives it can lead to errors and the more you have to copy and paste in tests the more annoying things become so we'll try and tackle that a little bit later but in the meantime i want to move on and take a look at a more complex set of tests and we can do that in a test case i created earlier called the profile root test so we'll boot this up and you can see we have three tests here if i run those tests you can see under profile root tests we have it cannot be accessed by a guest it can be accessed by a user called luke but luke's profile cannot be accessed by anybody but luke so no other user can access my profile can we refactor these to higher order well for the most part we can but i wanted to put this test here in to show you that it's okay to not refactor in certain cases so it is actually possible to refactor this to higher order but it would take a lot of work because we actually have to implement our own version of assert redirect which would mean returning a custom test response from the get method so it would be basically a ton of work for very little benefit and in this case it makes much more sense just to leave it as a standard closure it's readable it's easy to see and the effort doesn't justify the work it would take so i'm not saying that every single test you write has to be done in higher order i'm just saying make sure that you've checked whether or not it can be done easily in higher order first rather than just passing it off and leaving it so with that in mind let's move to our second test which can be refactored quite easily to higher order so we'll close that off early we'll remove the boilerplate as before remove the call to this refactor we can now use our as user helper and we'll just pass our username of luke in there and for the get method we can pass a closure if we run it again we are going to have a passing test and this is now higher order let's move on to our third test which is somewhat more complex basically in this one we have a bunch of users and we loop over each user and we check that everyone is forbidden because they aren't luke okay now this is okay it works you can see it passes but our eyes are drawn to this array and this array isn't important as long as luke isn't in these users it could be anything what's important is this piece of code here so can we refactor this in a way that would draw our eyes to the actual code rather than to the data that we want to provide the code with we can using something called datasets and they're as easy as chaining on a with method so we chain a with method like that pass an array and soon as we have an as user helper now i think it would be best if we just reduce all of these users down to their user names and we'll grab all of them and we'll drop them in this array at the bottom here and as soon as we make use of the with method we're actually able to receive one of these usernames in the function in the closure that we pass for our test so with that in mind we can now remove this for each and rather than saying b we can say as user and pass in the username and if we run this we've actually got an error call to member function get on null so why would that be being caused let's have a look i i never return from as a user that's fine let's run it again and you'll see we don't just get three tests we now get a load of tests one test for each username in this array i think that's really powerful and if we want to add another user for example let's say steve we run this again you'll see that steve gets added and anywhere that we use this or these set of users we're going to be able to run lots of tests very quickly so it's really powerful for writing out large test sets in very little time one thing that's annoying about this is the fact that if i wanted to use these usernames again i'd have to repeat myself i'd have to copy this array paste it copy it paste it i don't want to do that luckily pest has already come up with a solution to this called shared data sets so we can say php artisan pest data set and let's create a data set called usernames you can see that it builds a file inside the datasets directory called usernames.php i'm going to preemptively copy this array and then let's head to the usernames file inside this file you'll notice it's used this dataset function and the dataset function takes two arguments the first is the name of the dataset it's called the usernames based on what we passed here in the command but you can call this whatever you want the second is the data that should be returned so i'm going to replace that function with this array of usernames let's head back to our code and i'm going to say with usernames and then i'm going to run this test again and you'll see we have the exact same result but now we've abstracted the data to a shared location so any tests can make use of that data set very easily and we'll see an example of that a little later on second question can we refactor this to higher order tests well you may think no and you may just give up thinking there's no point but i want to show you that it's actually quite easy and it will simplify this test and make it look absolutely gorgeous let's start by removing the boilerplate as we usually do with higher order tests and reformat obviously if we run this now we're going to get an error undefined variable username in other words it doesn't know what this is supposed to be well just as we did for the get root wouldn't it be nice if we could pass a closure in here and what if that closure received the current data set so if we were testing with the username of nuno then this would be nuno but the next test when it tests say with freak this would be freak let's just fix the closure there while we think about it and then we'll head into our as user and we'll say that this doesn't have to be a string it can be a string or a closure now we're going to make use of the value function again that laravel provides so we'll say give me the value of username but i want to pass it an argument the argument i want to pass it is whatever data you gave me for this test so yeah execute the closure if it's just a string i'm just going to pass the string straight in but if it's a closure i'm going to pass in any data that i was given so if we come back and now we run this test again you'll see that we receive all of the data in a higher order test so you can actually make use of data sets inside higher order tests using the get provided data method and it's really powerful there are so many cool things you can do with that so i encourage you to go check out data sets higher order tests play around and see what you can come up with all right i think this is looking really neat so it would be a good time to move on to part three of our talk let's head back to our login test remember earlier we copied and pasted this into register and i was saying it's annoying that we have to copy and paste you may now after part two of this talk think well data sets are the answer why don't we create a data set that includes all of the routes that we want to test behind the guest middleware and then we'll just perform these tests once but we'll pass the data set in so we can say like with um guest roots for example now this works i've used this paradigm before but it falls short in a couple of places first of all what if there are extra tests that you want to perform on that particular route that means that your tests are in two separate locations and it becomes difficult to maintain and if you have a large team people will forget that there is a data set for a particular idea and so what they'll end doing is just implementing the test manually and so you'll have a mixture of manual tests and data set tests you do not want that i think there is actually a much better way i've been playing with this idea for a few weeks with my friend sam rowden and we really like it we think it's super powerful i can't wait to see what the laravel community can come up with because i think that they could take it to the next level we are going to write a domain specific language for testing and we're going to use pest to do it imagine if we could inspect a route so what if we could say i'm going to come into this test case and i want to inspect the route login and i want to say look it can be accessed by a guest but i want to check that anyone who is in that username array or usernames array dataset is forbidden so inspect login check that it can be accessed by a guest but check that it forbids any of those users let's go ahead and see if we can build this domain specific language we need to obviously add our inspect function somewhere and the best place to do that kind of stuff is in the pest.php file so we'll add a function called inspect it receives the route that we're interested in inspecting and why don't we return a new class that we can build our fluent interface with so return new tests dsl for domain specific language this is obviously not documented anywhere so we can do whatever we want we can call it anything we'd like but yeah i'll go test dsl route inspector and we'll pass in the route we were given okay let's go to tests and create a new directory called dsl and in dsl let's create a new class called root inspector awesome so now we have our root inspector we can try running this again but it's going to say there is no method called guest let's add our method public function guest it's going to be fluent so we'll return this and inside here we need our test to ensure that you can log in as a guest so we'll come back to the login test here and let's copy and paste this test we know that this works also in the constructor of this we want to say make the route that was passed in a member of this class and that allows us to use it here so we can say this root just like that okay if i come back to my login test now and i comment this out uh comment this out sorry and let's close guest off early and run pest you can see that login test actually did run our dsl because it can be accessed by a guest was executed so the first part of our dsl is up and running how cool is that i think that is so clean okay let's continue let's try and add this forbids method so we'll come back to our root inspector we'll add a new function public function for bids and we'll say look you can pass in a data set it's going to return itself so return this and let's come back to our login test we'll copy this test because this is checking that they can't access it and we can replace this with this root like that if we run this it's actually going to work it executes but it's not making use of our data set however we've already done all the work in the previous tests to add this functionality very quickly so we can say as user pass a closure receive the username and return that username and we can say with passing the data set and if we run this again you're going to see that our login test just receive a bunch of extra tests for absolutely nothing for free if we go to the usernames data set and we add another name in here let's say tom and we run this again you'll see that tom is added here and here so we're getting extra tests for absolutely no work at all and our login test looks as simple as this we built all of these tests with a line of code how cool is that i really hope you can see how impressive dsl thanks to the functional style of pest could be there could be packages for laravel around authentication routing mail storage everything and it would help people to be able to very quickly come on board with testing and build out amazing test suites in literally minutes okay but you may be thinking it's a little bit set in stone there will be roots that don't want to check for example that a guest sees 200 they want to check that you see particular text well two things there's nothing for one to stop us writing an actual pest test here so it shows the correct info okay so we could write our own path test here and this will actually fire alongside our dsl tests but maybe there's a better way what if we could define what success is on a dsl test case basis so in this case we receive the response and we'll say response assert c please log in and if you're wondering where that comes from you'll see that our login route just returns please login so what if we define our own success handler well let's make that a thing we'll go back to our route inspector and let's say private success handler and by default the success handler is going to just say here's the response response assert okay but if you use the success method which is fluent so we'll make it return this then we're going to say here's the success callback say this success handler equals success callback okay that's all of the boilerplate we need to write to make this work we now just need to make sure that guest makes use of it and to do that we're going to have to go back to using a function and the reason for that is we need to be able to pass the result of get into our callback so we can add a closure here and we can say this success handler for this get this route just like that now when i run this it's actually going to fail and it's going to fail because it's going to say call to undefined method success handler the reason this fails is because this closure has been rebound so this no longer refers to the root inspector however we can fix that by simply saying root equals this root success handler equals this success handler and then we can refer to them using those variables rather than using this so success handler root run tests are now passing and to show you that that did actually work if i head back to my login test and change this to please can you login and run again we're going to get an error it's going to say failed asset in the please login contains please can you login how awesome is that we can now define custom success handlers on a test by test basis let's copy this let's go into our register test and we can remove all of these functions now and replace it with some very simple dsl if we don't need to specify custom success logic we simply leave it out and you'll see that our register test passes just fine but if we want to say something like please register run again it's going to say failed setting that please login contains please register so let's take a look at why that would happen uh oh i tell you why it would happen because i never changed login this is actually inspecting the register route run again everything works i'm not going to go any further because we could be here all day but i hope that you can see the power that is made available to you with pest there are some insane things and territories that people haven't even thought about yet that are made possible thanks to pest and i think we're going to see some amazing things from the community coming very soon as pest continues to increase in popularity okay we are done with the code i just want to finish by talking a little bit more about why you should use pest first of all it's simple and it's easy to read there is no boilerplate there is nothing to get in the way the only thing that will be in that test case are your tests you'll see a nice description written in plain english or even in a foreign language if you aren't native english you can write those descriptions in whatever language you want you can use symbols you can use spaces you can use special characters you can use anything to describe your tests and especially with higher order tests everything is just so readable it's a pleasure to write it's easily understandable when you come back to your code six months down the line you probably didn't realize this but there are no namespaces in pests now you can add namespaces for example if you have duplicate function names but by default there is no need for namespaces and you may think well why is that a plus basically it means that refactoring your test suite is as simple as dragging and dropping the file you don't have to go in and change the namespace you don't have to rely on an ide to do the work for you and probably get it wrong you literally take the file in your finder you drag it you drop it into a different directory it couldn't be any simpler than that i already discussed this but the test names are so easy to understand and in the terminal they just read like documentation in fact you could take the output of your terminal and present that to your boss to the client to whoever and they have a full list of exactly what your application does how awesome is that higher order testing is a real boon it helps you to slow down and just think about how to make your tests better in my opinion that will save you time in the long run and it will make your testing experience more enjoyable true story here we have a a project that it's not legacy but it's big right 4 000 plus tests and the tests were kind of an afterthought a little bit we were just shoving things in there to make sure that everything worked properly i actually hate going into that project now not just the tests but the project itself and the reason is the tests i know i want to carry on writing tests but everything is just so difficult but because when writing higher order tests you slow down a little bit you think about what you're writing then the test suites remain as beautiful as the code and adding new tests becomes enjoyable now we talked about this at the beginning of the talk that past php is tailored for laravel but that is actually a huge bonus that deserves consideration we class ourselves as artisans we use laravel because it's gorgeous because it reads well because it allows you to write simple and yet powerful code pest is written in the same style it was designed to give you ultimate power it was designed to allow you to use all of the sharp knives but in a way that is beautiful easy to understand and so going from a laravel application from your source code into your test suite feels just as beautiful there's no gap there's no distinction it's not like oh i have all of this boilerplate here but my laravel application is so beautiful it feels correct it just feels right with laravel i can't stress that enough i've used pest in um a good number of professional projects now and i can say that i prefer it 100 over any other testing framework for the fact that it just fits with laravel so beautifully and i hope you've seen that it's extremely powerful and flexible from data sets to higher order tests to higher order expectations which we didn't talk about today but which you should certainly go and check out uh to full domain specific language as and when you require it pest is unparalleled when it comes to its power and flexibility and i just hope you had a sniff a snippet a morsel of that power and flexibility today finally i want to say a huge thank you again for coming and listening to my talk i really hope you enjoyed it i hope it gave you some ideas i hope you can get out there and start contributing to pests to the open source community to laravel and just enjoy your time we are amazing we have an amazing privilege to be able to write code that can change lives not just your life but the lives of other developers like the code we write in open source helps other developers do their job better and that is an amazing feeling so go out there and just enjoy working with pest with laravel creating your projects and building awesome things you can find me on twitter at lukedowning19 come say hi you can find me on github and see the work that i do at luke raymond downing and thank you again for listening really appreciate your time guys i'm gonna hand back now and i'll say see you later wasn't that a amazing talk very well done luke i really like the enthusiasm and all the neat things that you put in best it's just amazing that dsl idea it's uh it's it's a great one and i agree that we're going to see some uh awesome things with this in the future so thanks again luke for putting this video together and all the work that you put into best so we're at the end of the meetup i hope that you liked it as much as as i did next month we're going to have a new new one uh probably at the end end of october i still have to confirm my guests but i'll announce the guests in a couple of days so take a look at my twitter account for that so we're at the end i hope you enjoyed it and until the next time bye [Music] hmm [Music] so [Music] you

Channel: Freek Van der Herten

Views: 2,381

Keywords:

Id: 2UyDBArGLDY

Length: 98min 17sec (5897 seconds)

Published: Tue Sep 28 2021

Reddit Comments

[u/mnapoli]

Really interesting talk and approach, I love it!

I have one minor correction on this part: you can deploy Laravel on Lambda with the Serverless Framework, and deploy other functions (e.g. Node or Python) in the same application:

service: laravel
provider:
    name: aws

functions:
    laravel:
        handler: public/index.php
        runtime: provided.al2
        layers:
            - ${bref:layer.php-80-fpm}
        events:
            -   httpApi: '*'
    node:
        handler: my-js-lambda.handler
        runtime: nodejs12.x

plugins:
    - ./vendor/bref/bref
...

Both Laravel and the other functions would be deployed together (unlike what is said in the video). They share the same codebase (which is optional btw).