PHP Generators async with amphp

a couple days ago I was on the git er chat room for the amp PHP library of a synchronous library and there was a question from somebody and I wasn't able to answer it right away a couple days later I figured out his problem is pretty complicated and so I thought we would sort of revisit generators and what are they and maybe how to avoid this problem in the future the guy was kind enough to put his code into a github gist so here it is and basically he's setting up this router this is the Aris HTTP library and so he's connecting his router post to this URL will some sort of rabbit business object host right it doesn't really matter what the implementation is we come over here this is his rabbit Bo and what he's trying to do is get the entire request body JSON decode it do some other stuff do some other stuff and the response right and this didn't work for him and one of the key reasons why is that he's saying this returns a promise an ant promise and perhaps this res end and res set status will return a promise but there's one thing up here that is stopping the post object from returning a a promise and to figure out why we gotta go back to basically what our generators we need to relearn what's a generator in PHP what's a co-routine all that stuff okay so what is a generator generator is an interrupt Abal function where if you combine it with a asynchronous library sort of lets you write asynchronous cooperative multitasking code which means portion of your code can run posits execution another portion can run it's not pre-emptive there's not a kernel or a supervisor that can stop and say you only get ten milliseconds to run now I'm going to pause you right you have to write your methods to be cooperative with other methods and you have to give up your execution or yield it from time to time and the basics of generators are when it's sort of when you read the documentation all you really find out how to use them as iterators and how to count and how to count with less memory you can count to a billion without a whole lot of memory right because you're not returning an array that has a billion objects you're just you're you're just counting and then you're interrupting the function maybe at this yield and that yield is kind of like a return and it pauses the execution and gives that one integer back up the chain and then and then it keeps going and so when you're trying to learn about generators you don't find a whole lot of good useful cool multitasking stuff and one of the reasons for that is that generators can't really do it generators are like a small sub task of co-routines and if we go to Wikipedia it's gonna say here's it's like some pseudocode right I have a queue while the queue is not full do some stuff and what it says is that well okay like generators versus co-routines a KO routine our generator can yield okay I don't know me let me R over teens have the ability to control where execution continues after they yielded generators cannot generators transfer control back to the caller that seems obvious right it's like a function when you're done in you yield when you return you go back up to the caller when you yield you go back up to the caller but this is primarily why generators are used for writing iterators the yield statement does not specify a co-routine to jump to however it is possible to implement co-routines on top of the generator functionality with the aid of a dispatcher right but basically what they're saying is if you have another library you can you can achieve this effect of multitasking right with something that should only be really used as iterators you can actually achieve cooperative multitasking now if we go to this article this is a first article I read on generators back in December 2012 it was written and he starts off you know with the simple x-range sort of example of you know here's what a yield function doesn't here's how it can use less memory and then a little bit of the interrupter stuff a little bit of the send whatever it's a very dense article because he's trying to explain how to be useful with them but he just jumps right in and basically we're going over the building of this entire library that will help us achieve co-routine functionality with just generators nowadays we have amp and react which are two libraries that help us do asynchronous programming on top of PHP generators right and this is a good article to read but it's heavy and dense because you're going over the creation of an entire library an entire co-routine socket and you can't ease yourself into it get your feet wet and try to discover a library so I recommend starting with react or amp and but this is still a good read although it may be dense right so let's look at a little bit of example code this is a basic PHP function when you call example it's it's code gets put on the execution stack it gets a clean slate of a scope and when it hits the return value it sort of passes this value back up to the controlling script which is this one main or global if you will and so we have a equals three now in the generator world it doesn't look doesn't quite work like that in the in the now in the Oh routine world which PHP doesn't have it sort of works like this where you have a main process that has a queue and we're gonna Pat we're gonna start to produce and produce is going to say while the queue is not empty of produced while the queue is not empty creates some items then yield yield to the consume process and that's gonna get bumped off and that's gonna get held to the side here right in the in the in this one one example is finished it gets thrown away right it goes way over here to the side it's done it gets garbage collected right it's out of scope but in the protein world and a generator world it's not these execution cycles are saved now they don't they don't stack on top of each other either when I when I yield to produce and then yield back to consume I don't get infinite recursion like that they don't stack on top of each other they actually bump each other out we'll produce some items then it will call consume consume will remove items then it will call produce so this ability to jump to different co-routines is not available in PHP which is why we need the library but heed one of the libraries either amp or react sit right here and help us with sort of the handoff and some other magic abilities where we call produce and when this one is done it'll do this and then when this one is done and this is what's going to give us our asynchronous ability out of our generators or in our our iterators so this this library along with react can help us turn our our iterator generators into full-blown cooperative async multitasking all right so let's see how PHP hand the basic generator I have this function and it has yield one yield to return three they don't make that let's start this over all right let's look at a basic PHP example of generators and how they work I'm going to run this through my debugger we have this basic function called example says echo I'm starting yield one yield two I'm finished and then return three we step over this so when I call example what should I get should I get one two or three well you get neither of those what happens is you get an object of type generator and I can call things on this object rewind current next things that I haven't defined and this is the crux of the problem the original problem was that the app just didn't work and it's because yield will any function that it's inside of PHP at the sort of the Zend engine level will change this into a generator object and it won't be a function it won't have a normal return value now this may seem obvious but when you're inside the libraries and you're refactoring what you're copying and pasting is pieces of code into different functions this is going to bite you and you're gonna have a yield statement somewhere where it's not supposed to be this is key right yield fundamentally changes a function into an object now with this object when I call current it's going to go inside the execution stack and run up to the first yield and give me back that first yield so now my B is one and then I could say while it's valid while it hat you know while it still has more yields give me the current which I already have is one then do next and then I'm going to get two and then I'm going to go next and it's gonna see that it's going to run all the way to the end and it's not going to be valid because there's no more return so we're gonna fall out of this loop right here and I'm able to get the return value doesn't give it to me as part of the loop right and just get the return value and our is now three and see that and this might be a little bit confusing but what happens is you can use clothes this you can use the generators as iterators right so who's this for each and I can get rid of all this nonsense about valid current key next right as a for each because it implements the iterator class now the part thing is remembering all this is going on while you're programming and while you're trying to connect to my sequel and open your sockets and all that stuff right so 40 so basically the same thing right here if I step over I get my a is still a generator object I can call rewind which will error out if you've encountered any yields if you haven't then it does nothing not true rewind will advance you to the first key I believe the first yield see do I get one say get one so yeah rewind oh no it's because I did current rewind never mind forget rewind it's pretty complicated read the PHP Doc's if you want I can get one to my val is now - and the next time my val will be and pray so you can pass more than just simple objects back right only val is array one it has a key of nine and a value of foo so you don't have to yield simple numbers you can yield anything you want and we still have this great for each and we still have the return value so Jen yeah generators are not function so generators also have this ability to take in a value so this yield is a two-way street one it pauses execution and acts like a return statement giving control back up to the collar but the collar can when it resumes right instead of calling next I can actually inject a value back into it which is pretty weird but I can say send and I can say send hello from me outside right and if I just run this I'm starting on finished message hello from the outside and then the return value three interesting right technically I'm sending every single time I'm just ignoring it the first to yield so let's do a breakpoint here and I can run my Lube will I ever yeah okay so now we're inside the send messages uninitialized one more step [Music] what jumped out jumped out cuz what there was no more anything to do there all right let's sit another let's set another breakpoint right here times sorry my buggered I'd say so step step just run right okay message is uninitialized if I go all right so I'm not actually able to break inside here I don't really fault the debugger for not letting me step now maybe I should fault that I don't know maybe I'm not using it right that way so it's weird I have it now okay I just said I faulted it working I'm in this point yield one yield two messages yield 3 and I have message equaling the value hello from the outside and we have here send so everything is working and I'm able to pass in so man generators they act as iterators you could for each over them you could pause their execution and come back to them later and you can send values in at the pause time you so now let's go back to our original problem where the person in the chat room was trying to set this up as a route and set this post up and immediately we see the yield keyword and anytime PHP this is at the engine level you can't stop a table anytime you call this post you will get back a generator and it actually won't execute any of the lines right it won't execute any of these lines until you treat it until you either call current.next Zend or you treat it as a for each we can see this here I come back here and run hook that's not what I want to do I want to run this sample died run this I'm here just passed this stuff right here I called my function it hasn't said I'm starting there's no output here it's blank because this is a generator and I need to treat it as such right so the problem is that when he calls post it's actually just gonna return a generator object and so what he needs to do is somehow turn this generator into promise because he knows that he needs to return a promise to get it to work here but the tricky part of coding all this so I can imagine what happened is he coded all this at the top level and then went to went to sort of refactor it and it stopped working so here is I'm going to show you this this is from their examples page right it's basically the same thing so we have a statement which is yield prepare we have a result which is yield statement and then we have yield result advance to loop through the records I'm going to simplify it a little bit so we make a pool we yield a query to get a result set and then the result set fetch on we yield that to get rose I'm just gonna run this I sequel the PHP got six rows okay what am i doing selecting star from user not a big deal now if I say farm 12 Bardock LSAT see what we got here so we get this class which is amp my sequel result set now if I don't yield what do we get we get class anonymous which is a shame that it's not a real class in PHP I think they're kind of oddly focused on premature optimization this should be a promise we can see that it has resolved false and then it has unresolved which is a pointer to a function and these are trademarks of the promise interface my script errors out because it doesn't have fetch all because it's not a result set right so I need yield yield is magically changing promises into their resolved values does that make sense so when I put yield in front of something it changes the promise into its resolved value and that seems like a cool thing to do in a asynchronous library a problem comes when you start refactoring they get result hello this only works when you're one step away from the amp library like I was showing you before with the stacks right so this amp loop makes this function and runs it and it's able to do some magic stuff with the yield if I simply make a function called get result count results that I paste everything in here and I say return bro count so I pasted everything exactly the same I think it's gonna work no it's not gonna work undefined variable result set whoops that was my dad I need to do this outside they did too many things but still copying and pasting so catchable fatal error object of class gender cannot be converted to a string Oh what does that mean it means that this function returns a generator right it's a PHP function it has the keyword yield in at oh shoot you know am I not getting the magic when he just in a desperate move try to get the magic of amp back and now amp says uncaught invalid yield error it was expecting a promise or a promise interface to yield so this yield is only magic when the right hand side is a promise and you're you know and what you're yielding to the next level up is part of the amp framework because it has special magic functions to say ah when you yield it to me is it a promise if so I will sort of wait until it's resolved and I'll keep track of it and then when it's resolved I will inject the value back into here as with the send right so it's doing all this cool magic the problem is it doesn't work when you start refactoring unless you're one step away from the library so how do we get around that there's got to be a way around that right please please let me go around it there's one way around it and it's the amp call and what this does is you give it any sort of kala Bowl right now it's the name of this function cuz it's just a function I'm not fancy olp and then he passes variables you still have to yield it right but what this does is this will wrap a generator into a promise if we come back here aha got six rows that's what we started with right so everything is working now so to go back to the original problem is you you know if if we don't understand the yield function from from the PHP level up and we're sort of only dealing with the amp examples and does so much work to help yield be so cool in a synchronous environment but the core PHP is still going to make you have a generator make you have an iterable help you do array counting and so we have to be careful when we're refactoring and moving stuff away from the amp library into other functions other objects and we really have to understand is this object going to be asynchronous is it going to deal with ace the asynchronous stuff inside of it that needs to be yielded right because I can't if I'm just in this function I can't really deal with a result set I can't wait for rows as a promise I can't say like rows arrow like these finish now you can't do that there is amp wait and you can pass it a promise but for some reason I don't know if they're trying to be mean to you or what this will end your loop as soon as this promise is resolved it exits the entire program it's just like loop done I don't know why it's mainly for if you have a synchronous set of code and you're trying to run a little bit of asynchronous library and you don't have a way to wait you can use this but really if if you are doing asynchronous the whole thing needs to be asynchronous up and down the chain and you need to make sure that whatever whenever you have a promise they can yield all the way back up to the loop run so the loop run can resolve it you need to have this like chain of yields all the way down this has given you some good insight into PHP generators and co-routines and sort of the amp library maybe a little bit of react library and how you can use them how not to get caught up in them and let me know if you've made anything in one of the in one of these libraries have you made a a synchronous program of any sort let me know in the comments and see you next time

Channel: hackingwithmark

Views: 1,934

Keywords: php, generators, async, amphp, reactphp, multitasking, coroutines

Id: O7jcuPSnMME

Length: 25min 52sec (1552 seconds)

Published: Fri Jan 12 2018