Controlling Concurrency: Advanced Go Programming

today i want to talk about a more advanced go pattern for managing concurrency levels in go it's very well known of course that go is brilliant for writing concurrent programs its primitives you know go routines channels make concurrency easy and predictable and safe um but it could almost be uh accused of making concurrency too easy um and it's always important when you know ex executing concurrent code you think about the consequences for or side effects for anything which you're operating on so a good example of this is imagine you had um you need to make a number of requests say it was to uh i don't know an api and you need to make requests for a number of for a number of ids so we'll write this out as you know imagine they're just integers and i'm gonna i'm gonna make these um uh contiguous numbers here for the demo which we can do shortly but imagine yeah you need to go through and make a request for each one of these records so the standard way of doing this taking advantage of go's concurrency model would be simply to iterate through each um iterate sorry through the slice of request ids and for each just make a request right assume func make request is uh just some you know it in reality it'll be making a http request for example but here we'll just mock its asynchronous nature um so of course also by the way you know you would need your weight group so each one would add to the weight group and then we'd wait for it and then at the end we'd know we were done but there's an obvious problem with this with this approach which is that we have no control over the rate that requests are being made so say that we own this um the server that we're making requests to we might not look favorably on getting all of these requests concurrently and we can we can you know witness the effect of that when we run it here um we've got a panic but i'm not too worried about that you see how the requests were all made at once this could be potentially quite dangerous for a uh for a server if you had one if you were receiving 15 requests all at once or you know potentially hundreds or thousands of requests with no concurrency limit uh this is not a situation you want to be in so what you want to do ah we obviously didn't call uh done sorry just um yeah we want to actually limit the throughput of requests here so for example if we if we only wanted to at any time have three open connections to our server we can set concurrency limit three how would we implement that and go while still making use of go routines and other go primitives well first of all we can we'll keep this but everything else can we can throw away here if we were um going through that we can we can use a channel instead and remember that we can use channels almost as buffers so we could create a channel with a ball type for example and set that buffer size to our concurrency limit so this allows us already to control in some way throughput because we know at any one time there can only be three boolean values inside this cube so we're already going in the right direction and as before we can uh we'll probably iterate through uh we'll range through our slice our request ids um but this time before we uh make our um start our go routine well no let's actually implement that first so um as before we'll make our request inside of a go routine um pass an id always remember as a side note if you're um working with go to routines like this the val this value will have changed by the time that this go routine executes because this will happen asynchronously so this will iterate through c uh sorry synchronously so by passing it into the function here we um cement cement each iterate uh sorry each of these values so we can then pass that in there just as before we're calling this inside a go routine but what we now need to do remember we're using this channel uh like a like a q like a a limiter of throughput so for each id we need to get a place in the queue and we know that that channel can only allow three or however many our concurrency level is into the into the queue or through the actual channel at any one time and if that queue is full then we have to wait uh and we'll be blocked here so we can use that that we know only three at any one time will be passing through here so then of course we can defer that we empty our spot in the queue when we're done and this gives us almost everything that we need to implement concurrency uh currency because um so we're queuing only three at any one time can can get past this line 18 to make the request when the request is complete we give up our spot so the next person in the queue can then make their request now we are missing something let's see if we if we clear this and run it i'm not sure if this will be very obvious from the demo um sorry i've got to clear up this and um ah yeah i can't actually just i can't call this a constant slice um let's see if from this demo we can see we we can witness ah yeah we can so you probably saw from that demo if i run it again and make this sleep a little larger you can now see from the demo that only three are being run at any one time and as soon as one gives up their spot the next one comes through now they're all finishing at the same time because the weight is exactly the same of course if this request was a real request then one might take slightly longer than the other and it wouldn't come in batches of three exactly but it gives us a good indication at least that that concurrency level is working but if you take a look at this request these request ids here you might notice a problem which is that we're missing uh 13 onwards and the reason for it is this that we only block until um each uh id has got on a spot in the channel so when the last three values get their spot in the channel they pass this line we're no longer blocking even though these requests are being made inside the go routines we're no longer blocking the main routine so therefore it exits because there's nothing stopping it and these go routines are orphaned that's why we get into that position where these final three the the code is executing somewhere here but the outer routine is finished and those requests are often so what we need to do finally is make sure that we flush that cue that we do we push out all requests to the other end and the easiest way of doing that is just and that is another iterator which will execute as soon as those final three get their spot in the queue um and for every spot in the queue we just need to fill it with a final value so what this is saying is we won't exit we will block until we've managed to push three in this case extra values into the queue that is fill the entire queue with these useless values and what that tells us is if if we're able to block until we can we can fill this channel with these sort of dummy values it means that we know that all the request values have completed and um and they have given their spot back given their spot because they are um they've completed their requests in this case so hopefully now we should be able to see that in action let me make this delay a little bit shorter so that we can just see we can see this in action but as before you know it's coming in chunks of three and then we get the fine the final three and they actually execute in this case again because we made sure that that q was flushed so that is how we limit concurrency and go and i think this is a very important anti-passing sorry very important pattern to know in go it's very nice because it uses two of the most important go primitives the go routine and the channel and this is exactly how go is meant to be used using these communication and concur under parallel premises or concurrency primitives one other thing to note and uh i'll run this so it's clear these only come in batches of three because like i said this time is exactly the same but um in fact each new request we we don't wait for all three to complete before sending the next three as soon as one is completed the next one fills its place but it's important to know that the throughput is three that doesn't mean we batch in threes but anyway that's um how we limit concurrency levels and go i think it's a very important um pattern to know and um i think we'll leave it there for today uh if you if you like these sort of short explanations of the more advanced um say features of go or another programming language or other concepts and please click the like button so that um so i know that leave a comment with any feedback you have or any questions you have and then subscribe because these videos are coming out every weekday um and if you like this one i hope you'll like the rest too in the meantime thank you much very much for watching and i'll see you soon [Music] you

Channel: Sam x Smith

Views: 2,307

Keywords: distributed systems, software engineering, go, golang, programming

Id: M-ofC23gSZ4

Length: 13min 34sec (814 seconds)

Published: Tue Nov 17 2020