API Platform Conference 2021 - Nicolas Grekas - Symfony Runtime: wrapping API Platform in a lambda

[Music] hi everybody last conference for for today i am proud to welcome nicola gregas on scene and he gonna talk to us about symphony runtime nicola okay hello everyone i'm happy to be on stage in real life i think that's the case for every speaker today um so i'm going to tell you about symphony run time for like 40 minutes um and which is basically a way to put a php application in a box and the php application can be of course api platform and that links up very well with kevin's talk about edge computing and things like that because this is where you might want to use that um so symphony runtime is a covet logged on artifact i was bored almost one year ago a sunday it was sunny and we weren't allowed to go outside and so i started this day looking at that so this is the hello world in php it's a more advanced hello world because it's reading some global variable to display the request uri right so if you look at this it's basic php is very low level we just bootstrap the php engine there is nothing there is no loader there is no classes there is absolutely nothing and this has a drawback this drawback is that it relies on super globals like we read some magic variable that is in the air and it's not reusable it's very difficult to run that in a loop in a different style than executing it from the php runtime directly the typical way to run php scripts right and we still run symfony applications this way this is how the front controller works it starts from nothing and it builds up a symphony execution environment right and we usually forget about that but that's how the php world starts so i started this day saying okay there are drawbacks to that i'd like to be able to do that actually i'd like to be able to return the closure and say now instead of relying on the super global this closure is going to take an argument which can come from anywhere not a magic variable and then i'm relying there on some outside execution runtime that should know how to what to do with the closure right so that's how i started that and there are some challenges there but the benefits are this is now decoupled from the global global state right you saw that there is no magic anymore you can decorate that so now there's something that is going to run the closure and this something can do something before after like do some instrumentation what you want basically so we can decorate and change the behavior of the runtime environment and then we can also run that many times because we have a closure and a closure is a function so you can call it once twice as many times as you want it's built uh in a box and this box is the reusability unit like a functional um unit so i went one step uh one more step and then i said okay what about that can we make that work like this should be our phone controller we start we open a php file it's empty and we write that and can we make this work so there you can see there is still the closure but now it takes an argument which is a symphony request and it returns a response right so we should find a way i wanted to find a way to make this work there are challenges on the path and it would be nice to be able to not have to care about creating a symphony request on our own because this is a process that is coupled to something like the infrastructure that we don't want to care about we want to decouple our applications from the infrastructure so there are challenges the one is how do we get the written value from a script if you think about that it's not built in the php engine when we you return something from the front controller the return value is just ignored it disappears in the air then there are arguments so the symphony request how do you create that how do you resolve the arguments so another challenge so it was a funny sunday um how do you deal with the return value now you run the closure you get something back oh it's a response what should i do with that right how do we make that extensible that's also the next step how do we build a software design that solves all those challenges in some extensible way so that this is reusable and not hard coded logic so i published not truck bootstrapper that was my prototype i published the we published actually the runtime component this winter and new applications since this winter used the symphony runtime to bootstrap so if you create a new symphony project you will have the runtime component and it will be responsible for starting from the very beginning and wrapping the symphony application in the box so the goal kind of the catchphrase of this component is its goal is to decouple the application from the global state which is kind of abstract sentence right so what's the global state um this is typically uh yeah okay so a code like a php script it needs some environment so this is the house and the house your application is run inside this house and there are some things variables that are floating in the air and you can read them you can mutate them even so this is our runtime execution environment and if you look there this is the front controller the typical one there are many places in the code that read from the global state like the dot env line we read from the file system the configuration for debugging the configuration of the kernel and the request is then also created from global this is even worse because this is hidden in some static methods so wow where does this come from it's magic right that's how php works on this on this topic so now what if we put our application in a box so that there is some kind of well-defined communication protocol like the closure that wraps our application so that we can then maybe change the runtime right because this is the previous script is tightly coupled to php and the way it actually exactly runs and populates super globals and etc so this is what the runtime application the runtime component is about it's about wrapping our applications in this box to decouple them from the outside so this is a short example i have a few in the next slide so that you can figure out maybe get some understanding of what this does so this is a script if you look there um it's a closure there's the autoloader at the beginning then we have this closure that does some uh thing with the request and we run the application at the end so this script has the same drawbacks as the first one the hello world because it is tightly coupling the functional unit like the closure and its execution you cannot get only the functional using unit without executing it and that's tight coupling we might want to decorate or to do something fancy with app before running it or running it many times so to fix that and to make it run using the symphony runtime component we change the autoloader so the autoloader is now vendor autoload runtime something that is going to be generated by the component we wrap this closure into a closure and we return the closure right so we have two closures one is the functional unit to ship that and to return something useful to the outside and then the outside will get this closure so that this runtime thing that is the context outside the runtime execution engine is going to be able to call the closure somewhere else and we don't care there that's the point we do not care about the infrastructure about how this works we just declared that there is something missing of course it's the context variable there we need to reference the local variable instead of the global one so i hope that was okay that you understand the step we made together let's do that with the symphony index.php file the one i showed before so there it is so i told you there is one thing to do we replace autoload by autoload runtime um this is the thing that is going to be able to catch the written value right there is something i'm going to explain you how it's going to work just after so what is global there the env file this is global state we remove that configuring the debug mode this is global state we'll remove that creating the request after the kernel this is global state we don't want that and then also the end the end is not something that is specific to our application the end is the only way to deal with a symphony kernel right so if we have a kernel something this outside like the symphony runtime component should be able to run the kernel so we remove that also the terminate2 and we are left with the kernel which is the thing that defines our application so then we wrap that into a closure we change the super global and we make it a local variable we indent and that's it so that's the new front controller of symphony 5.3 applications and the next versions too and that's the only thing you need to do to bootstrap a symphony application because the runtime component is then going to be able to make that run and basically run all the code that has been removed now it's moved outside and that was the motivation for creating the component so i'm telling you about benefits like theoretical or design and things like that my benefit the benefit i had and the motivation was let me go back this is a script that was shipped by a symphony recipe so that something that is is committed into your symphony application but we want to be able to upgrade that this is not the same script five years ago this was a bit different and it's difficult to update you have to figure out the changes because this is not in the vendor directory so we don't have control over that now if we move those logic those lines in the vendor directory in the runtime component then it's much easier for us to maintain that for you so that was my motivation so why do we do that so this is your application like the one before in php fpm or some apache model but um you might want also to run the symfony application in a breath context so breath is this wrapper to make a php code run on the lambda right so your application needs to be rewritten at least the bootstrap process needs to be rewritten right now to run there or maybe you want to run your application in a small server context so this is again very different because now this is a c extension where you have a long running process so you need also to hook into the way swool provides you uh gives you the responses and how you can communicate with the outside world so there is again some code some glue to write between your symphony application and the school execution engine so now if you put your application in this box this closure we can decouple completely your application and make the infrastructure swappable that's also the benefit now the same application the one we had before basically any symphony application starting from finder three is very easy to run in school or php fpm or etc um there are many runtimes as i mentioned brave i mentioned a swool there are others and tobias niholm i'm sure you've seen his name because he has he maintained so many packages in the php ecosystem when he saw the component he said oh i need to create an organization for that and to put or to publish many runtimes so if you go there you'll see he wrote many runtimes and that's how you can make use of that very easily so he wrote that if you go into some empty directory you composer require runtime school so let's do that so see uh seeing is believing right so we are doing that and this is something you can do just after the talk you will need the the full extension but that's something you'll find here easily and then the same public index php file i showed you before can be run this way so this is configuring through an environment variable the actual runtime we want for running this script right and that's it because we have the school the runtime spool package this variable will be read by symphony and it will be able to select the correct piece of logic to run this application on spool that's for production maybe because you want super high performance but on the local development machine you might want to use php-s like the local web development server because this is more convenient because there is this auto refresh mechanism so that you change a class and you don't have to stop the server and start it again right so same code different runtimes different experience each um tailored to the need or to the actual use case um there is a benefit also which is leveraging swool and swool is super fast so this is tobias telling us that if you run symfony in a loop like in swool you'll get a 50 performance boost this is because the container and the kernel won't have to be restarted again and again for every single request so we you will save that he also runs that on laravel and he has like a 100 times boost this is because laravel is slower because it's not compiled so if you save a slower thing you have a higher ratio so this is basically saying that laravel is faster than uh symphony is faster that laravel for the boot time and then we don't care because usually your application runs right so how does it work under the hood the challenges are mentioned so let's see let's see that so let's create a psr example this is also to tell you that symphony runtime is about running php applications in the box it's not about running symphony application right so this works this is a psr example i don't like psrs but i like showing you that this still works um psr 7 at least so this is another runtime the psr neon runtime and this one is able to create psr 7 objects from the pss psr 7 new home library so we do that we open an index.php file there where we can put that so here if you read that it's one file but you could split that in two files but right one to declare the class so we have this very simple class and this class implements a request handler interface that's the important thing we implement the handle method from the interface and there we do something like returning a response which is the contract of the request uh handler interface and then what we do and the box we give to the runtime component is the application so we return the application from the firmware closure and the runtime psr neon component is going to make that to run so this works we didn't have to care about creating the request itself that's the point so let's see what a runtime is so this is the code an excerpt of the code to declare and to create a runtime so there you see we have a psr 15 runtime this is the thing that is going to know how to deal with the request handler interface and you see there is we extend generic runtime and we have this get burner method that is going to return a runner and there you see oh if it's a request handler interface like our application is then this runtime knows what to do with that so it's going to return a new ps15 a runner that is wrapping the application and if not we're going to fall back to the parent class and the parent class is going to know how to run a symphony kernel for example so then you can have one runtime that runs either pss-7 application or symphony application or many you don't need to care because the runtime will select automatically the most appropriate logic to make that run so this is the runner the runner does one thing which is calling the handle method you see at the bottom and from the response there is going to run the response which in this context means echoing the body of course this works on as an api model you would have a very different logic to run so the runner would be different to run on swool but your application wouldn't care that's the point so this is the http kernel runner so we are back in the symphony world so there you see we have the kernel we call the handle method then we send the response that we uh we've got back and if the kernel is terminable then we terminate it so basically that's the lines of god the last lines of coding we have in the index of php that's where they are they are there in the vendor directory make sense so next step because just now i told you about the runners which is the thing that deals with the return value from the closure but now we need the argument resolver so that we can provide the correct request object to the closure so this is for example uh accepting a request from psl7 and returning this response so it's an even even smaller application like a one request response one control application this is a psl7 runner and there so we have the run method that does return or execute the body echo the body at the bottom so this is an even simpler handler or runner for psl7 so now how do we deal with that we deal with that by also implementing another method which is get argument so at the bottom there get argument is going to be provided with some [Music] reflection information from the closure so we do some reflection on the closure and we give to this function a i want to run a closure and the closure has one argument which is a request a server request interface oh i know how to deal with that let's return the correct object for that so there you can see we have the same stacking mechanism where one runtime can provide and can deal with many different kind of objects like this is uh returning returning a psl7 server request from globals and for symphony we have also somewhere some code that does request create from globals if it's if the argument is a request from the symphony http foundation so there are many different kind of return value and arguments that are accepted so that's in the documentation so this is a console application so this is your bin console script the bin console script is like that you we created the kernel because this is where the configuration starts and then we've wrapped the kernel in the console application and then the runtime component is going to know how to run this console application and that's it we can do that even more simple so this is a one common file so this is basic if we if we want to do a bit more advanced things with the command we can declare a single command application so there you see the closure it accepts a command which means the runtime knows how to create an empty command then we configure the command in the closure and then we return another closure which is the actual implementation so execution logic of the command right so this is a single command application i like that i don't know about you but this is clean this is clear we split the declaration step and the execution implementation in two different pieces since i like it i don't know about you but then there are um there is an extension extensible mechanism to add more types if you want to so more types means how do i handle another kind of request object or how do i deal with my kind of kernel or basically type of application so you do that so there is a trick of course you can write your own runtime because there is an abstraction and you can then do everything by your by hands or you can leverage the generic runtime which is going to look at some special name spaces so there you see this is psr http message table request interface and there is the convention is to add a namespace before and a suffix so symphony runtime the thing we want to handle runtime and by naming a class like that the generic runtime is going to auto discover it and that's how you can very easily add more types to your application or create packages that just add that and that's it so the implementation is very simple you just create an empty runtime like the first line that extends the actual implementation and same for so another example there to deal with responses from psr namespace so we extend general ground time and there i think we already saw that so let me move on what is interesting there is the bottom you see there is some auto registration mechanism so the runtime component will auto discover this class because of the namespace convention it is going then to call the register method and the register method will actually register this type and maybe some sibling types so this is what we do here when we discover this type we also say that oh by the way they implement the runtime implementation for the server request interface and the response interface is this current object right so that's an extensibility mechanism that is built in to make this very easy then you might wonder how many runtimes might need to be written not many actually because there are two things that drive the creation of a new runtime one is the infrastructure side so traditional web server needs a special way to be uh told how to receive or send responses then same for edge site complete computing that kevin mentioned aws google cloud or runtime with react php roadrunner school then on the other side we have applications like basic php application or symfony http foundation-based applications or psr something applications console application and then you create a mix of that and you have 30 runtimes and tobias wrote them all already so you don't have anything to do so thank you tobias for that um short technical focus how do we did we solve this challenge getting the return value so this is um yes this is on the top your script right so we require the autoload runtime script and we return the closure whatever the closure so this is at the bottom um so yeah then let's execute that first we execute the first line line so require once we enter the autoload runtime script this one is going to require the autoload.php file from autoloader from composer obvious then we are going to require script file name which is public index.php so we are now going to require the thing that required us and of course we do that because we can then read the written value from that so we are back to the next line and we have app that is populated and we can continue so we expect that it's an object if it's not we continue and then let's continue we have this runtime because we need to create the execution unit so this is an object so first we have this class which is configurable as you can see the runtime is this environment variable i showed you on the previous slide and so we create this runtime object we configure it a bit so then we call getresolver which is the thing that knows how to compute the arguments of the closure then we run the closure with its arguments and then we exit the script by calling the runner and that's it right easy so you can configure also the runtimes because runtimes might have some options that you might want to set to tweak them so to configure a runtime you would use the composer.json file so in composite.json extra runtime there you can put like the class and this is going to replace the environment variable i showed you before you can also put many other values it depends then on your runtime and what you want it to do so the generic runtime already has some options you might want to have a look there because we changed the class the autoload runtime the php script is a bit different now as you can see we have the two options that are have been dumped into it so this is because [Music] the runtime component is hooking as a composer plugin so when you run composer there is some logic that is generating this file this is not mandatory if you don't like composer plugins you can just commit this file and tweak it to your needs but this is just convenient right and that's it so the runtime interface looks like that that's the core of the components the most complex things we have to discuss for months like two methods two abstract methods um so get resolver that returns a resolver interface and get burner that returns a runner interface and there is the generic runtime implementation so how this works in a more abstract way there are five critical steps and the nice thing of those steps is that you can hook uh in between or between and before and after each step so you can decorate any of those steps to make something fancy that you want to make at the runtime level so that's why this is flexible that's because you can control how each step behaves so first your first your front controller returns a closure then we resolve the arguments of the closure so if you decorate that you can maybe decorate the arguments themselves if you want to etc then recall the closure but maybe you decorated the closure itself you can do that of course then the return value of the closure like the symphony kernel is passed to some runner so you can also hook there so that you can maybe add the http cache on top of the symphony kernel if you want to so then you have a runtime that has some caching built in and then there is this um exit code that stops the php engine usually or would tell some school one time how the request went if you want to deal with the exit status in a different way you are not going to exit from that okay so that's super flexible you saw small side notes about laravel you might have heard about laravel octane larval octane is about running laravel on swool or on a roadrunner this is very different from symphony from symphony runtime it's different because it's tightly coupled to laravel if you look there you might have a hard time adding more runtimes and you might having a hard time using that for something else than novel so it's tightly coupled and the comparison the comparison stops here basically it's completely different even if you can achieve something uh similar with symphony runtime you can extend that in symfony so as a conclusion this is from tobias so i'm joining tobias this is a wish and the wish is that in 2022 we might all abstract the file system because we want we run things you know in containers and we want to be able to write on s3 instead of writing on the file system or actually want to be to be able to swap this kind of storage with however we want so we would use fly system i suppose for everything related to storing uploads and things like that then we would use redis memcache for session caching things like that again to have a scalable design architecture infrastructure very fast and not needing any writable storage right and shared automatically clustering etc we would not use we should not use super globals because that's the beginning of you know nightmares and maybe you do that so that's cool but maybe you still have some progress and the symphony runtime component might help on this path telling you what we could do or what you could do as an experimentation is right at runtime that's an idea i didn't try it but right at runtime that kills the super globals at the bootstrap of the application you might want to create a runtime that empties all the super globals so that then any code that tries to read that fails if you want to do that then you are sure that you'll achieve this decoupling from super globals and the wish is that also every single php application would use the runtime component to get all the benefits of that i mentioned before so um symphony runtime is used by default since symphony 5.3 it makes you ready for swapping the infrastructure that's the point that's one point swapping the infrastructure computing on the edge using swool using a development using php fpm etc it opens up innovation at the infrastructure level so now people that like hacking school runtime react we can do work there i'm sure there's some innovation that can happen there like instrumentation like i don't know like emptying the super globals we can try that without titling that these experimentations to the applications themselves and this shouldn't be your daily working layer that's i'm not telling you something that you are going to use tomorrow because you are still going to work inside api platform inside the symphony and then usually the application is already bootstrapped but you should know that there are still people that are going to push that like kevin maybe so that then it runs seamlessly and you can swap the infrastructure for seamlessly so of course we'd be happy to i'd be happy to listen to your ideas of what things we could do at the infrastructure runtime level but usually you won't hack there that's not the place to hack too much thank you very much i hope this was kind of inspiring and if you have any question let me know thank you very much nicola are there any questions online or on site okay we have two hands there so okay express with you are there any questions online too for the next no no question online okay oh thanks for your talk i really like the idea of swappable runtime i think it's very promising as you said my question is every runtime is providing global state through the context variable how can you extend if you want to add more metadata or variable to expose more things to the cons to the context variable how do you do so you would create a runtime because it's very easy to create a runtime runtimes with this interface there are two methods you can decorate the generic runtime from symfony and that's it so you would create a class in your application that does this extra logic and then you would reference that in either the composer.json file or some environment variables depending on how you want to make that work and that's it so you don't have to use the context variable because the context can be also provided through an object like a symphony request object for example or yeah let's start or your own domain object if you want to bootstrap with some context object maybe thank you i think we had an extra question there i really like it especially the hack on the hey i require myself back so just to be sure so that works only because it's required once and we do the diet then right that's the first thing okay and um my question was more in the on the runtime implementations uh actually forgot the the terms you were using but very you kind of like uh suggest to just do a your implementation extends the generic yes runtime why is that actually rather than more like adapter kind of like you support this um this input and then you provide something why this extension you need to extend the class why do we need to extend the class yeah exactly because that's the way things are registered as a single unit so you can create several in the in inheritance three you can create many different runtimes but at the end they are all going to be wrapped into this generic runtime that is going to dispatch uh to dispatch uh to the correct runtime so right it feels like there is a decorated runtime from the outside but actually there is one generic runtime object that internally deals with many dedicated runtimes and it is dispatching that depending on the type of the argument or the return value does that make sense not yet thank you we might talk about that after any more questions so do we have we have a question online first sorry a question from jeremy how much performance benefit can we get jumping from a container as a service approach to a function as a service like lomda for symphony i don't know zero yeah zero that's the same there is the same logic that needs to be run every time so zero you would get a benefit from swool or react php like long running processes but otherwise you have to boot the kernel you have to boot create with the container so zero i think sorry okay we had one last question there thanks for this talk i've seen this component and now it's like taking real shape now with this conference so thank you for presenting it i had a question regarding functional testing and i was wondering if it was possible to use such features such as like creating the kernel uh one single time and reuse it on every uh test while running php unit for instance in order to somehow save performances maybe because it seems like the kernel uh is capable of resetting itself um but uh i was wondering whether we could like keep the cache in the memory so it doesn't have to reload the cache on every test yeah that's a good question um so right now we create a new kernel every time with some magic inside symphony so i think that would be very interesting to to explore being able to use the front controller inside our test which would return this kernel object and then we would be able to run it into tests i didn't think about that but that's why i mentioned it it opens innovation at the infrastructure level i think this is one very really nice use case of being able to get back this kernel this is something you cannot do right now you cannot get your kernel the real kernel for testing you we need to have some very strange way we would need to create it statically on every test case creation for example will this set up before class or something like that or even have it globally with the bootstrap you would require the public index.php file yeah with the runtime yes but actually with the current setup in the php web test case you would need some kind of yeah there is some more i don't like the code if you when i open this class in the test kernel testing framework of symfony it's a bit scary it works but so i'm waiting for a pull request thank you very much thank you [Applause] you

Channel: Les-Tilleuls.coop

Views: 321

Keywords: API Platform Conference, API Platform, Symfony

Id: vXk2Z1dy_gA

Length: 40min 51sec (2451 seconds)

Published: Tue Nov 09 2021