Efficient Concurrency in Go: Managing GoRoutines and Load Shedding

[Music] so what you're seeing now is the creation of a go routine and that go routine is just blocked on that listen and serve call which is accepting traffic on Port 4000 and then executing any of those end point uh end points um if they exist I want to talk about again this go routine that you're seeing right here on line 101 we're constructing and it's being blocked on listen and serve for those debug end points so essentially this is what we've got going on right now we've got our main go routine main go routine and it's moving down this path of execution at some point it it it's stops waits for a signal to shut down and what we've just done is now started a second path of execution that now blocks on that listen and serve call right it's essentially now A M waiting for traffic from the outside world and then it launches more go routines um when it we hit any of those end points it's essentially where we're at right now got the main path of execution we've got our debug which blocks here on the mck now I want to share with you a general rule that we do want to follow and remember all general rules do have an exception so the general rule is that when it comes to concurrency and creating go tees I I want you to think about this as a parent child relationship it really helps to think about go routines in a in a parent child relationship in this case this go routine is the is a parent this is the child because this gtin is the one that created this one so always think in terms of parent child relationships now as a general rule we don't want child go routines terminating before their parents as a general rule we don't want to do that we if we we want to maintain order with all that concurrency that's going on the parent go routine is responsible for the child go routine it is responsible to know when it's running and when it's not and as a general rule we' prefer that the parent go routines do not terminate until it knows the child go routines have all right as a general rule we want this now what is the problem when a child when the parent go routine terminates first what is the problem when the parent go routine terminates first well what the problem is is you end up with an orphan you you end up with an orphan go routine okay suddenly some some go routine here launches another go routine here and then suddenly the parent go routine disappears we have what I call an orphan nobody even knows that this go routine exists anymore and then when that happens if you're trying to shut down the system you can't necessarily wait for this go routine to finish and maybe it's doing super important work and now you have corruption and this is really hard to to debug and so as a general rule we do not as a well not a general rule we never want orphans H let's say a general rule we don't want orphans as a general rule we don't want or so what are you supposed to do in this scenario in this scenario if the parent go routine is going to be terminated it has the responsibility of handing off those children to another go routine normally or usually in these cases the Maino routine ends up adopting it now how do you do adoption and go what I will tend to do is I will have a package that has an API that's managing all of these go routines right and I'll have some sort of shutdown or stop method and the idea now is that it's the job of the main goroutine to call shut down or stop in other words the the management of all those children are embedded in the instance of that value for that package that we're we're talking about and the responsibility for for wait shut down or stop moves away from this parent go routine and moves to the main go routine so you got to have the facility to allow another go routine to sort of take ownership or adoption when I don't see that when I'm looking at concurrent code I get really really scared all right so as the general rule we want child go routines to terminate before their parents their parents should not terminate until they know they have in the cases where you can't that that parent go routine has to terminate then there has to be a handoff primarily to the main go routine and primarily through some sort of API this absolutely happens this exception will happen when you have a web request and you want to do some concurrent work on it you can't hold the web go routine the web request go routine hostage because it has a certain amount of time to what respond back and so you've got to release the parent and don't use that parents context either for the children because they're all cancelled right and so the idea now is is we got to maintain some parent child relationship so no orphans however look at this line of code on line 101 technically I am creating an orphan go routine here technically once we get to line 111 there is no more tracking of this go routine I am creating an orphan in other words when this server when this service sh shut down this go routine will be shut down with extreme prejudice now why am I doing this after I had an entire conversation with you to not create orphans because in this case there's nothing that's hurting the state of the service with this orphan this orphan is just reading profiling and metrics data there is no reason to add extra complexity to make sure this shuts down before the server shuts down in fact I would argue that if you're waiting for this mck to shut down before you shut down the service you're doing an injustice to the service why should we hold the service hostage for 30 seconds God forbid we're running a CPU profile when that CPU profile is not important what's important there is the shutdown so I want you to find these situations in code I want you to recognize oh who whoa that's an orphan there's no tracking of this anymore there's no weight groups there's there's no channels there's no orchestration is this okay is it okay to have this orphan in this particular case the answer is yes this is a perfect exception to create an orphan but a lot of times the answer is no the developers is missing the idea that someone has to take uh ownership of that go routine if it's no longer the parents responsibility and it just really works well when you're thinking about child parent relationships here so I want you to recognize that that's an orphan that being said we now need to do the following right we now need to create another M that is going to be managed by another go routine to handle our application Level traffic now this these endpoints are hitting databases they're doing reads but they're also doing rights here's a situation where we cannot terminate until we know all of these child go routines have terminated we cannot terminate here we have to do what's called load shedding and again all of these go routines are going to end up having to be the responsibility of the main go routine being the parent in this sense so what we're going to have to do on shutdown is ask this go routine here hey I need you to shut down and this go routine is going to say okay wait because I need my child go routines to shut down it's going to wait for its children this has to wait for its child all right and then we can shut down cleanly so we need load shedding here and what's beautiful is the HTTP package gives us this this was introduced a long time ago uh to the HTTP package so we have the support we just have to implement it because if we allow these go routines to shut down with extreme prejudice they could be in the middle of a transaction a database the worst thing would be a databased transaction that never gets rolled back or committed now you're in a lot of trouble so it's really important that this stuff shuts down cleanly so let's start to look at some of that code that we need to be able to do a proper load shedding shutdown um and then we'll introduce the MX part of of what we're doing all right let's go back to our main code here there it is right here okay so I'm going to steal a little bit of code now again for what we need we're going to start that API service going to have that logging we're going to create our shutdown um Channel again for the shutdown signaling like we had before but in this case we're not going to use the function listen and serve what we're going to do is construct an HTTP server value which has the method listen and serve and provides the facilities for uh a load shedded um shutdown all right you can see here that we're also applying a standard Library logger for any extra logging that might occur underneath the covers this is how the HTTP package is provided that support the eror Handler here is appointed to a log. logger slog converts uh its logger to that API it all works okay I've left the Handler nil because there's bigger conversations to have with that just yet that will be the mck and then what we're going to do is I'm going to steal this code and we're going to clean it up okay it's going to launch our go routine which is blocking on listen and serve again however this listen and serve can return an error on shutdown or while it's running and on shutdown but we can receive any errors from the listen and serve call through a second channel that we're creating on line 126 so at this point we have two channels one to Signal a shutdown sa from kubernetes and one to signal that we have some sort of error on our accept logic networking side and then we block on the two channels using a select case ideally we should never receive a signal on this case hopefully never ever we never have that sort of networking ever um we do want to receive signal on shutdown and when we receive a signal on the shutdown other than some logging here we construct a new context with the shutdown timeout and then we call the shutdown API from our server value anytime an API takes that context like this it's asking for a certain amount of time we're willing to wait so whatever that timeout is we will wait for a clean and orderly shutdown however we can't wait forever so at some point we may have to kill some of these goines with extreme prejudice because at the end of the day we do have to shut down we can't be a zombie process and in that case we call close and just to really make sure everything is is shut down as properly as possible and this is what's holding our server now from shutting down and it's this right here that's allowing Maine gortin to ask essentially this goroutine to start shutting things down so that parent child relationship and there's go routines in here that are managing the child go routines that they're creating so it's all pretty cool here all right and we get all this for free from the standard Library

Channel: Ardan Labs

Views: 1,275

Keywords:

Id: mCUEj5SvRUA

Length: 13min 47sec (827 seconds)

Published: Mon Jun 10 2024