Golang Worker Pool

hello internet welcome to my golem tutorials right now we're going to implement a worker food using golem so let's start by the way this tutorial assumes that you already have background knowledge and how to install golem and the basic syntax and colon and some knowledge on channels and goroutines colon concurrency so if you still don't know about that there are lots of tutorial out there I'll be making one also soon so right now we'll go directly on implementing the co-worker fool using golden concurrency and some channels and go routines so let's start with a your an empty file let's name it naming that go and start with package name you'll have our function so what we want to implement is we want to have at least well at least three or four workers maybe yeah maybe three three that's two three and we'll define a worker as a function worker and you'll have an ID worker ID and it will receive a job jobs okay and this will be a channel and paint all by the way it's going to only to receive okay so if you have this symbol right before the channel means it only receives from that channel and if you have it from a sorry only send from that channel but if you have it on the other side it only accept value okay so we have this and then after a job you will send out the result also going to be a channel but this only receives okay see the difference between the two okay so that's our worker and our our main function we're going to define some work okay so let's let's have a for loop here or for work you see cost starts with work ID 1 1 and then why any job probably let's give them 6 X is less than or equal to 6 then we have incremented okay and we will call the worker function the number one worker x equals two jobs and there is so okay but since we are going to create a worker and I think the only you only have to make probably three workers on week sorry so since we're going to create workers and they're going to be on performing the job independently or phone currently so we'll call them ass ability and we're going to declare your channel jobs as a channel an integer so this is what we call a buffered channel okay there's a difference between a channel and a buffered channel okay the difference are buffered channel they keep on accepting values in this case ten values and then they will not block until the buffer is full but if you have non buffered channel they only receive one value and to only receive another value and when that value is has already been passed to to another variable something so okay if you need more information there are you'll talk about this on another tutorial basalt is also a channel channel integer okay so here we have our workers three workers so after calling all three workers we will now give them the jobs okay so let's have six jobs for them one is j j is less than or 12 to 6 in jobs faster to the jobs channel okay so after giving them jobs we're going to say close jobs so this is signaling our goal teams that this channel is closed and no more values are going into this channel anyone okay so after giving them jobs or going to okay then we're going to get the results since sort six okay now we've got a going to print let's print the result let's say result received worker see from results channel okay okay so I see not this year it automatically importantly the library that we needed this is just plugin from Adam you can you can download it I believe there are also plugins for that on other editors like sublime text yeah so okay let's go now to our worker function let's see what what is going to be doing in in this function so we're going to receive a job okay so it seems it's a buffered channel so it's going to be like an RA so let's say job seekers to range jobs that's been cleaned out with her worker ID is working my job job okay then so while the worker is working let's let's put in some time raishin to simulate that it's working on something okay so let's let's randomize the time hey divide that time and milliseconds okay we have that friend after that it's assumed that the child is completed so let's say here worker / I ve completed work on child within I think you need to put this up so we can print out the result or the duration okay okay so after that we're going to return the ID of the worker so signaling that the job is done okay let's see is there errors so respected oh yeah okay okay I think that works so it's cool to our command line it's fine okay so what do we have here so in the first three line we our program created three workers a worker bee worker one referred to and each are working on chopped 1 2 and 3 then working 3 completed work on chopped 1 with in 81 milliseconds then after which were quickly completed it started working on job number 4 and then we receive we result you see from worker 3 I think this is the completion of job number one from red to blue and then we also completed chat number 4 then we receive another assault from job number 3 then also pleated work on chap and verify so were quickly as a lot of work and we're good to complete that is chapter 2 and so all these are working concurrently as you notice we we have oops sorry okay okay so as you know this we have actually six jobs for three workers but they were all done with in just milliseconds okay so I think that's all that's how we implement workers we : okay can we first we declared two channels jobs and results and then we created three workers from our core function we call them as goroutines then we give them six jobs and close the channel and as soon as we have given them all the jobs then here we we wait for the results okay so we uploading this on on github and you can see the code and the link below okay we'll next time in the next tutorial bye

Channel: Keep It Simple

Views: 5,597

Keywords: golang, go, concurrency, worker pool

Id: 7XS7fVX8SF0

Length: 15min 48sec (948 seconds)

Published: Sun Jul 08 2018