gRPC + ProtoBufs in Golang | Protocol Buffers | gRPC | Golang

this video will be looking completely into grpc with coolant and how does it actually work and also be looking into protocol buffers and how it interconnects with grpc and go so we'll also be will be for building a project on this um it's a very small project but i think it would be giving a very uh good insight into what's uh and i mean how it happens with grpc so let's get into it before even going into jrpc we need to learn the its competitor which is rest so rest is usually what we do like we have roots in any of the language and we use we call it as rest api and usually how it goes is that it has a browser and it has a server and request and response comes and goes so we have a single request and a single response comes back and if there are like multiple requests and responses that goes into different kind of topic where it's um kind of like websockets kind of thing so let's see what grpc is so before that i think i know this is like a complex diagram if you have seen in on this in the on the internet but uh before even going into this diagram what i want to say is that jrpc stands for google remote procedural call the main advantage of grpc is that it is used it can be used with many languages and it can be used with i mean um if you integrate jrps with protocol buffers then you can like write code or you could structure your code very well and you could integrate it with any language so that's the plus point of using erpc in this diagram you can see we have go service and this go service has a grpc server and there are some clients each server has client so client is what contacts the server and we have a c plus plus kind and we have a java client and we have something called grpc stub so we'll be talking about this a bit later but what happens is that we send a request and we receive a response this is not this traditional kind of or requested response but this is like more like a function call where one where the client calls a function which is present in in this grpc server so that's how it happens so in this diagram different uh language clients are contacting our grpc server and they're also receiving response to that so let's get started with grpc so in grpc even for function calls we have four different types so different types of what different types of services so grp offers four kinds of services which are i mean we mostly use unary which is in this diagram i think you might have guessed what what is going on in this diagram we have a request and then we have a response it's just a single function call and we receive a single response from the server we don't receive any more responses or we don't receive any more more than one request we don't so that's what unary unity stands for single so we have a single client and single server and responsive request and response so that's simple and if you go a bit down there's something called client streaming so in grpc we have a concept called streaming so streaming in a sense we can send multiple number so more than one is called a streaming um in rest api you can take web sockets as an example because you'll be sending multiple uh streams of events so that's an example for from the rest api but in grpc we have we have streaming so we have client streaming and we have server streaming and if the client streams then it's called as flame streaming client streams in the sense client might send multiple requests which is more than one request but after completing all this request or you can say after receiving all this request we receive the client receives a single response which indicates the end of this request and response cycle so that's what is known as client streaming so you can say we have multiple blocks here each each corresponding to a single request so we have multiple requests we don't know how many it might be one it might be more than one but it's multiple third service is server streaming and server streaming is uh quite opposite to what we have seen before for client streaming so in this case the client will be sending one request but we'll be receiving multiple responses from the server um in this case we are using the stream of responses from the server so requestably single but we but we can receive more than or one we would say our responses so that's what server streaming is and finally we have bi-directional streaming so in this case we have streams of requests and responses coming from both the side so this kind of makes like a web socket so it's not exactly websocket but you could say this is an um mimic version version of websocket so we have this four entities um let me zoom in we have these four entities unary client streaming server streaming and bi-directional streaming and we'll be seeing how this can be used in our application so when we will be seeing this when we build one uh now that we have we have known uh about glpc let's dive into protocol buffers i just wanted to give a quick introduction of protocol buffers um what is protocol buffers so you would say protocol buffers is kind of a structured way to make the interaction between client and server so this in this case the client is sending a request and a response right so this risk request and response is very specific very specific instance it is um very strictly defined in the protocol file and this request and response should obey what are all uh present in this proto buffer so that's why this protocol buffer exists to control what request and response is being sent between the us client and the server so here the use of our protofile is that it is what we are seeing in this this diagram so we have different clients you can see c plus plus client and java client so how do we actually build clients do we build for different languages when we are building for different languages do we need to write code separately for them um that's where pro proto protobuffers come into play it's actually not needed to uh there's no extra code needed because we'll be serving this compiling this single protofile into multiple uh language specific files so when you compile f protofile it can be used with that specific language so in this case one there's a dot product file which will be we will be writing or defining the request and responses and it will become it would be compiled using this uh plugins we'll be using in this case we are using a go plugin because going to be writing code in go and uh you could either use c plus plus or js or many more languages so you could use any of them so let's take an example let's say there is a book and we are writing a definition for book in a protocol so when we write a proto our book.profile and we write all the definitions in it and we compile that into with our go compiler and we then it will be creating two different files so one it would be book.pv.co and book underscore grpc dot pv dot go this will be automatically generated using our after company so you could we'll be seeing uh how this is useful let's check how protocol buffers actually works uh let's say we have a book and we want we are defining that book in a protocol okay and let's say that this um the name we are giving for this message we say message for a structure in protocol buffer so let's say it's a book request so this is a request will be the the client would be sending to the server um this this request should contain all this information so you could um how you define um let's say a field in a protocol buffer is that first you give the type of that variable and then the variable name and then the you'll be giving an integer number which is unique to this field and it should be in sequential order eventually protocol buffers are serializable which means they can be converted into binary and then into the original uh code bank so and vice versa so we need then we we can define this number so that they contain this would say they may they store the order of how the things are going to be in the request so we say string our name because it's of type string and isbn author name and price so price is of type in 32 so we have um we don't have in but we have in 32 and protobuffers so we give one two three four so we can also write the same thing in a different manner so we could do one thing we could uh this book request will contain a variable called book of type book so what is this type book we'll be defining that again um which we have defined before but the we have just separated it just for organizability and reading purposes but this looks uh much much better and here we have we start with one but we here for every structure it starts with one and goes on so so we could do this um this is a better way of doing this next i want to talk about what will be building in this video i'll be building a very small api using grpc and we'll be building a heartbeat api a heartbeat api in the sense will be having a client and a server and it's just a requested response from regarding heartbeat so it sends an heartbeat request then we receive an hard bit response so this is a unary similarly we have uh for client streaming service and we have a bidirectional streaming service which you have seen before and we have server streaming service so we will be building this i will we'll be coming back to this when we're building the application so i think we can get started with the control proto producty or you could say protocompiler you have to you can just google it protoc and you can just download the first you could just click the first link where you'll have all the documentation for computing installation for uh as i'm using mac i could easily do proven style protobuf and then use proto c version um there is actually um many many installation options for this if you're using windows or linux any of this you could just install this you could actually find your go paths by saying echo go path so that gives you the go path where it exists and inside that go path you could just do src and it would go into your src directory where all of your project can be made so i'm inside my src folder if you could see that yeah and what i want to do is i want to create a folder named heartbeat and go into cd into that right now there are no folders i want to create uh let me create the first folder which is heartbeat underscore pb so pv stands for protobuffer so let's go into that directory and let's edit um let's say heart beat dot proto so we have this proto file which we have seen previously we are going to be defining our services here so first thing we need to be saying is we are using the which version of rotor we are using so you have to say syntax is equals to proto3 and we have to make sure that this is ending with a semicolon because it's very strictly typed so next we want to define what the package is so the package i'm going to name this um i'm going to name the package as heartbeat underscore pb and next we can set an option which says our go package is equals to where this exists so it's the slash heartbeat pb so this is where our this protofile exits save this and let's quit and let's go back we have one directory now i want to create another directory let's say heartbeat underscore server so this is going to be the server part which is going to be handling the server stuff and let's cd into heartbeat server and uh let's create a file call server.go first thing is that we need to listen to port by default our grpc listens at port number five 50051 so let's try to connect that we could do that by saying uh let's say list comma error which is you could suggest net dot listen uh first thing is we need to specify the type of connection and the address of the connection which is localhost 5 50 051 and i actually want to define a function so usually what we'll be doing is that we need to in go we need to use every variable so we have to listen for errors if error is not equals to null then we can say log dot fatal of error so we could just use that let me save this so that it automatically imports the packages for me but i want to write a function here book so that i don't want to handle error every time i just want to call this function so it is of type error and what i want to do is i just want to move this here and now i can just say handle this error and send the error into this for this project we'll be using mongodb so we don't we also need to do a we also need to install mongodb so if you you can just go to this you can just do go get this so you can just do you can just go to mongodb driver um and you can just do go get this url in order to install mongodb or mongodriver for your project to setup a mongodb what i can do is um i can just say client comma error which goes to dot new client and i want to create this client basing options dot client all right which is going to be a mongodb db url mongodb local host and the usual the default server port of mongodb this is 27017 and then close it and if i save this it will just import the options for me and i can handle the error here by saying handle error i want to create a global variable called collection which is of type starmongo dot collection so that i can actually use this to create collections or to make queries because you'll be using functions i want to keep this as a global variable if there are no errors what we can do is we can just say println mongodb connected right and we can say error is equals to client dot connect and we have to send a context here we don't know what the context is so we can say just say to do if you don't know what the context is you can again handle the error by saying handle error of error so after connecting this we need to connect to a database because we haven't defined our database yet um we can just do that by saying collection we can just assign the collection variable here uh by saying collection is equal to client dot database of which database you want to connect i want to say i want to connect to heartbeat database and dot collection which collection inside the database so that's also going to be heartbeat let's go back to that file let's say pv slash heartbeat.proto let's define our let's define our first request and response so here we have client and server and there's a heartbeat request and response so let's go and define that so first i want to create a type of heartbeat so what this heartbeat consists is that we'll be using this heartbeat for every request and response or maybe or not but most probably if you want to use if you want to know the heartbeat then we have to use this structure so the heartbeat contains bpm which is of type let's say hint 32 bpm is equals to one so this is how you define variables in proto you say the type and then you say the name of the variable and then the number which is which should be unique and in sequential order the username to which i mean this ppm should belong to some user so that's here's the username well we will not be doing any authentication stuff but we need something to uniquely identify to which this to whom this bpm connects let's say the first heartbeat request so you can say heartbeat request and we are going to have is in this request we are going to have one variable of type heartbeat and the name is going to be heartbeat we have to give one here so is equals to one and similarly we'll have heartbeat response so here it's just going to be a string called result is equal to one so what we're going to do is just when we're going to send the bpm and the username of of the user and we're going to send it as a request and for response we'll be getting just a string because we'll be storing this in mongodb the request data and then we'll be sending uh the request store or we could say the hard the heartbeat of bpm so also has been stored so that's a response going to be so it's just going to be a string so we have defined our request and responses but what we need to do is we need to define our service service in the sense we need to define our functions so you could do that by saying service and let's say heartbeat service so the way to define service or a function is that you need to have an rpc keyboard before the function name and after rpc we have the function name in this case it's user heartbeat and this is going to take one argument which is our heartbeat request and it returns that's another keyword we need to return something and you could say this so we're going to need to return a response right so we could just return the heartbeat response so that's it this is going to be our first service which we haven't uh yet run this but you could say we will be building this so let's quit and let's see what's actually inside let's come to the root directory and let's see what are all inside this heartbeat underscore pv this is one proto file uh right now we this is just one protocol so what we need to do is we need to compile this protofile so that we can generate what we say as stubs so you could do that by saying protoc where the file exists so it said heartbeat slash heartbeat underscore pv that's the directory where this heartbeat dot proto exists and uh we need to right now we are using go so we need to put out those plugins here so you can see dash dash go underscore grpc dash out so dash dash grpc it's under underscore out is equals to colon dot so that's that's how you compile this so that's for grpc output so we need to add another flag which is dash dash go underscore out is equals to column dot so this is our two flags which is for two generate two files so it says um let's change this to heartbeat pb so it's saying uh it's giving us an error let's see what we have forgot we forgot and a semicolon so let's go into the proto file and uh here we forgot it semicolon let's save that click the screen and again run should actually be dash not underscore and now it executes successfully and let's see what is inside heartbeat underscore tv so before before you were compiling we had one single file but we have now two more files which we have seen dot pv dot go and underscore grpc dot pb dot co let's see actually what that contains so let's go into dot pv dot go so there is so much of gibberish here but one thing you can see is that we are seeing something some uh entities which which are familiar to us there's a heartbeat structure which have which has some auto generated fields and also which there are some fields which are given by us so there's a bpm and username field of in 32 string and we have some functions built-in functions we can use that or g set string or some others so get username get bpm these are some functions defined in this file so we could use that in our server file so we have a heartbeat response structure we need to have results you can also see that every first letter is capitalized so that needs to be taken care of and there's a get result and there's a large byte file so this there's nothing to know about this but this is just all auto generated so let's also take a look at heartbeat underscore grpc so here there is there there are some interfaces and each interface needs to be implemented by us so right now you can see that we need to create a server here heartbeat service server in our server.go file that should contain two functions or you could say you have two functions one is the user heartbeat and second is this must embed unimplemented heartbeat service server or it's quite a long name but we need to include these two or else it would giving it would be giving us an error so this is what we need to build first so this takes context and a heartbeat request and returns a response and an error so let's go build this so let's go back to our server file and even before starting to build this we need to um we need to import our grpc library and as i said you could go to that url grpc go and you can just go get this url in order to install grpc we can initialize our options here options is equals to grpc dot um now you can say you could create an array of grpc dot [Music] ops it's an array of grpc dot server options or its server option and you can send this into a variable called jss so we could say grpc dot new server and options so we are sending this and we are creating a new server variable so let's save this an import so that it imports it we can use this as variable to connect to jrpc and before even if you have taken a look if you take a look into heartbeat underscore grpc there is a function saying register heartbeat service server so we need to even before um even before using this uh in interfaces or something everything inside this even before using the entities inside this we need to register the heartbeat service server so we have to do that in our server dot go so we could go here and just say uh heartbeat underscore pb dot register heartbeat service server and this takes s and a server variable so we can say ampersand server we haven't created this server so as i said uh it ha this service server has an interface which needs to complete those two functions right so we need to create that so we let's create that struct uh it's actually type server which is of type struct it needs to have heartbeat pb dot unimplemented heartbeat service server which we have seen and also it needs to we can just that's on this is enough for now and let's save this and we have registered to that server here error is equals to s dot so of the listener and we can just handle error by sending error so we can put this into a if statement if error is not equal to null handle the error let's save this and let's open a vertical window and try to run this let's say go run heartbeat server slash server dot co so we can actually put this inside go function and just put it inside like that so we have defined our collection database and what i want to do is i want to create a channel here so we can make a channel um that's a chan os dot signal and this channel is of size one so that whenever we close our server we also close our mongodb server so we can just say signal dot notify ch comma os dot interrupt so why we're doing this is that it notifies the channel whenever there's an interrupt occurring so let's say that and um let's just say the ch what this does is that this blocks until we receive an interrupt from from our keyboard which is control c or control d in our case so after it receives an interrupt the whatever below gets executed so what i want to do is i want to print a new statement saying closing connection because we will be closing the mongol connection and let's say if r is equals to client dot disconnect and it needs a context so let's again send context.to do and the error is not equals to nil then handle the error send the error here so after that we can just say s dot stop which stops the server so let's try to run this then we can see that it's uh mongodb is connected and it's the server is running so whenever we say control c it we it gets an interrupt and it closes the connection so that's what is happening here so we have we have our basic server running here so now we can implement that user heartbeat function we can create a function so let's say a heartbeat and let's see what it has here heart beat so in this let me bring that down here so in this interface we are having user heartbeat and it must implement it it takes context and heartbeat request as parameters so let's say ctx is of type context dot context and request we have request and it is of type heart beat underscore pb dot heartbeat request and this also has a response coming in so we can say heartbeat it's of type pointer so heartbeat pb dot heartbeat response and an error so you can see that we this is inside this server right this is inside the server there are there are these two functions so they we have only implemented one function we have defined the function but we haven't tied this with this server right so to do that we can just use the receiver functions in go so you can just say server here so that it ties up with this structure let me close this and now you can define whatever is inside this server so for for now let me just return nil and nil we it actually throws the error but it's okay and i want to just print ln the request which is coming in and i quit this and let's right now we have two directories server and proto but we we need to create another directory named heartbeat underscore client and let's say heartbeat underscore client slash client dot co we need to connect to grpc first let's say connection comma error sgrpc dot dial so dial in the sense whenever we are making a request make the request to this um url and usually grpc needs to be in ssl mode but we are not going with us because we in localhost we don't have ssl so we can just pass on flag saying grpc dot with insecure so it makes sure that even http is accepted so we again we need to define let's define the handle error function it takes an error which is of type error and then log dot if error is not equals to null then we can just say log dot battle of error and here we can just say handle error of error let's save this so that it imports everything for us you can just say connection dot close so that it's uh it's actually capital c so that it closes whenever the function whenever this main function ends so we need to clear create a client so um we need to create a client for this server to make a make communication or make calls for the server so to do that we can just create a variable c and we can just say heartbeat pb dot new heartbeat service client which takes connection as a parameter let's save so that it imports um the heartbeat underscore pb has a function called new heartbeat service client what this does is that it creates a new client and it returns it so we can store that at c and we can make calls using this c variable if there are no address we can just say client starter we need to do that for server so that we can know that server is running so the first let's create the first function which is user heartbeat so we need to use the c function c variable in this function so let's take that here c and this is of type heartbeat pv dot heartbeat service client this so in order to make a request let's save this uh let's go into proto file which i have created so the request should be of this type right so it should be of type heartbeat named i mean we have we need to have a variable named heartbeat which is of type again it's of type heartbeat create that request let's say heartbeat request let's close this is equals to heartbeat pb dot heartbeat request so we have this built-in uh you know we have seen that structs right uh for heartbeat request and heartbeat response we nee we have the trucks so we can just create a new struct which is of type heartbeat request and this heartbeat request in turn has another um struct which is heartbeat right so we have another heartbeat which is of type heartbeat pb dot heartbeat so now we can define our variables we have bpm and our username so let's say bpm is um 75 and the username is my name which is mono srira it takes a reference so we need to send an address not the struct itself so again we as i said uh if we whatever i mean in the protocol if you have seen we have given all small letters but whenever it compiles it the first first first letter is uh converted into uppercase one so we need to make sure it's uppercase here so let's save that and we have this request with us but we need to send this request so to send this we can just make use of this c variable here we can just say c dot user heartbeat so this is a function we which we have defined right so it takes a context which is context dot background in this case and the heartbeat request so the c dot user heartbeat we are we are calling this function because you know we have tied this um by using the receiver function if you remember that so we just call this so let's actually create open another tab let me open another tab and let me split this into two pins and on the on the left side there is the server i will be running the server so heartbeat underscore server slash server dot go and on the right side i'll be running heartbeat client slash client dot go um we have an error we haven't used the c variable here it's because we haven't called the function itself so you can just call this function use a heartbeat and send c in here so let's call the function again now you can see that on our server we received the response which is heartbeat with bpm and username but also we received an server or error it is because as i said we are sending the response as nil and the error resonant that can be nilbert um i think this both should not be nil because nil we it i mean this response is a pointer so it needs an address so we should not send nil we need to send at least an empty response but not nil so now what i want to do is i want to store this request here this bbm and username in on in into mongodb so let me write a function for that so that i can just use this function to push any data into so i can say push user 2 db and this takes the context because you need mean con we need context to as a parameter for queries so we need that and the second is the item so here what needs to be defined is we need to have a if you have if you have worked with collections then we need to define a model for so we haven't defined that yet so let me comment this and just see let's say heart item which is of type struct so this is going to be model for our collection so in our collection we are going to have an id which is a primitive dot object id what it means is that it is just a type of type object id which mongodb offers for us automatically so this we are going to have decent and underscore id which is automatically generated by mongodb and we can put a comma and say omit empty and just close it so that's an id for us and we can next have bpm which is of type in 32 and this is going to be bsn bpm close it and we also have username which is of type string and we can say bpm sorry basin and username so this is what we're going to have so these three things this a bit just to keep things simple so let me uncomment this and item is going to be of type heart item now so what will be returning is just a primitive dot object id because i will be retaining the id of newly created object so now what i can do is i can just say rest comma error is equals to collection dot insert one and i can send the context and or the item so then just handle the error here so that's what push to db is and we can just return res dot id inserted id but this is of type this this is not in a match with what we are sending this is not a primitive dot object editor so we can just convert it like this and that's it and here we are having a request here we need to take bpm and username out of out of this our request so we can say bpm is equal to request to dot get heartbeat it's a function and dot get bpm it's a function so the pro one of the pros of working with protocol buffers is that it we get this inbuilt functions for our protocol variables protocol buffer variables so that we can just take them out so we have a username here similar to this request.getheartbeat dot get username so we have these two variables and i want to store i want to create a new struct here which which is of type heart item so let me say new heart item is equals to heart item which is going to have bpm which is of bpm and username is going to be our username i need to put a comma here and what i want to do is i want to push this into db so i can just do doc id is equals to push user to db and i want a context i can send this context which you are receiving here and i want after context i want to send this item so instead of this item i'll just send the new heart item which you have just created so now that we have inserted our document we need to send some response back so what i will be sending is i'll be sending the document id with some other string so we can just say fmt dot s printer just to keep things simple i mean i'm using s printer because it makes formatting easier instead of adding this plus and things use a heartbeat is some value and newly created dock id is another sum value so the heartbeat is bpm and newly created dock id is dock id so i'll just return this result but you know you can't just return the result because this is of type string the result is of type string but we need to return heartbeat response here so what we'll be creating is that um let's say heart beat response is equals to heartbeat pb dot heartbeat response and inside this we have a result that result is going to be this result and a comma here and it will be sending heartbeat response and make sure you are sending the address or the reference of this response not the response itself so the reason we are adding the result like this is that if you go to the proto file here we have the response inside this response i mean here we are returning the response we are not returning the string result itself so we have to create a response and insert that response we have to put the result so that's why we are doing so much of this restart the server the server and while it starts let me go into my cla okay and i want to use the database database heartbeat i'm not entering it i'm just keeping it so my server has started and i'm running my client so the client has started and it exited but there's no nothing from heartbeat server because we haven't put any indication for the heartbeat server to acknowledge it so we can just say here println heartbeat or you could say user heartbeat called okay and let's and let's restart the server and when it restarts we can just send another request and now we can see that there is an user heartbeat called call so now let's go into mongodb and let's use the heartbeat new collection heartbeat so we can say heartbeat dot find so now you can see that we are having two different i mean we have we have two entries in our database so it works so we have finished building this first unary service and now we can move on um in this case i mean to just to remind you we are just sending a single hard bit request and response there is no streaming here now let's go to stream uh from the client so for this you can see uh this heart is beating it's a gif um yeah so what and this hard static which means that this is not going to stream this is going to stream on the client so let's send a stream of heartbeats request from the client and then we can just respond a single response after all finishes from the server we are finished with our unity service which is heartbeat request now we are going to build our second service which is our live heartbeat so let's define our service here which is live heartbeat and this takes live heartbeat request as a request and returns returns live heartbeat response so this is the service you want defined but we need to define these two request and response variables so let's go to let's go here and just define what similar to what we had done before so let's say message live heartbeat request so this is actually similar to what the heartbeat request is we'll get a heartbeat request of type heartbeat which is equal to one but now the response for this live heart beat response so the response is also going to be a string but this is a different different type of string you may ask like everything is similar here then how we are going to stream i mean isn't there any difference for when we are creating the protofile so there is a difference because we need to change this service which we have defined here so we are saying live heartbeat request and returns a response uh but when we where this request is not going to be a single request right we are going to get multiple requests from the client so we need to add a keyword called stream before this request so that there are there can be multiple streams coming uh as a request so let's first go go to a client yeah and this router client and let's define a function new function which is live heartbeat this is also going to take heartbeat sorry heart beat bb dot heartbeat service client so what we're going to do is we're going to send multiple um requests from our site from the client side and the server will just create a string saying the user's heartbeat is so and so and the doc id is so and so and it keeps on appending the new string and finally it just returns as a single string so if you don't get what i'm saying right now it's okay because we'll be we'll be just seeing what to do so what i'm going to do is i'm going to run a loop let's say for t starts from 0 to 10 and for every iteration i want to send a request or you can say send a yeah send a request to the client server so for that we need i mean to send a request we need to create a new request create a new request heart pb heartbeat underscore pv it's underscored pp dot live heartbeat request uh i you know we are not getting any autocomplete because we haven't compiled that because we have made it we have made some changes to the proto file we have to again compile that so you can just go to protoc and just run the command again live heartbeat request and this is going to consist of a heartbeat so let's say heartbeat and that is in turn it's going to be a heartbeat pb dot heartbeat so in this heartbeat we need to send a bpm and a username so username is going to be same on our stream because i don't want to change it but the bpm needs to change i mean we can instead of just sending a static number we can just generate random numbers so let me create a generate bpm function and what this does is that i create a bpm variable here and i can do rand dot um int n and i can take up to 100 here and i can just send the bpm back this takes in 32 as a response ah response back but we need to convert this into in 32 to send this and now at this ppm i can just say generate bpm now another comma here and comma here so we are sending this ah we are creating this request but we are not sending it yet before even sending the request we need to create a stream from the request so stream comma error is equal to c dot live heartbeat and context dot background which is by default and we can just handle error here and stream now we can just do stream dot send this new request and after everything is done after there are 10 requests sent we can just close the stream so we can have a response and error and we can just say stream dot close and receive so what this means is that it's close the stream it indicates that the stream is closed to the server and then it waits for the response to arrive so let's handle the error here and it waits for the response to arrive and we can just print that response here so it it's giving me an error saying live heartbeat is not implemented yet but we'll be implementing this let's go to heartbeat server and let's create the second function and let's try that to server the function name is going to be live heartbeat and this is going to take different parameters than the previous one if you go into the grpc form file so you can just search for live heartbeat this is the service this is the client one we need the server or you can just serve so yeah we have this live heartbeat and this time all the previously we had context and heartbeat request going in but this time we have just one thing saying a heartbeat service underscore live heartbeat server so what this means is that we are there they're just giving a stream of request so you can just say stream and heartbeat pb dot heartbeat service and underscore live heartbeat server so this might be a bit long name naming conventions but it is automatically generated so we can't do anything about that so what i'm going to do is i'll be continuously receiving uh if you remember what we did in the client file here client.go we said this we call this function close and receive so until this function is called we'll be running a loop and we'll be receiving messages uh on and on so let me run an infinite loop here for this and i'll be receiving the message we're using this stream variable here so i'll be saying message comma error is stream dot receive so it's like uh if you have used c to read a file then we reach the end of file right so similar to that we reach the end of stream here whenever we call that stream close and receive from the client side so we can just check if error is of type e io dot eof then we can just break it or to handle error here what i want to do is from this request i want to get the bpm and the username out and i want to create a new mongodb our document so we can just do that by saying doc id is push to db push user to db and they'll be saying context.to do because we have no context from our client coming right now so we will say contextual to do and we'll be creating a new heart item there is no reference so we'll be creating a new heart item that contains bpm which is yeah message dot get heartbeat dot get bpm and the username of your username is going to be message.get heartbeat dot get user name and there's a comma here and what i want to do is i want to have a string here result is an empty string and for every iteration what i want to do is i just want to say result i will be at appending a new string so let's say as printer user heartbeat is equals to this doc id is equal to this and i can just say the user heartbeat is audits um it's bpm we have not defined that let's just copy and create a new variable out of it and the doc id is dog id but we need to yeah i think it's you know let's do this for differentiating all this the because we are happening many strings i just want to make sure we can separate them strength so another client we are actually not returning anything from this uh function so that's error so we'll be receiving returning and error whenever whenever i reach the end of the file or you could send off the stream i want to return this result back so i could just say return heart heartbeat underscore pb dot live heartbeat response and i can just say result is going to be the result this the string which you have used till now so instead of return we can actually say stream dot send and close there's a function called send and close so what it does is it sends and it closes the connection and then just put an ampersand here and you can compile this again so it's running and we can send actually we need to call the function again it's in the client so if i open the client dot co there's a user heartbeat what i want is live heartbeat of c so uh when i run this you can see that i'm getting this um account i mean i think i'm getting like 10 responses back you can see that i have differentiated it by these dashes i just want to make a statement here i want to create a new lan and uh i want to say request sent and i want this new request so you don't want to print the response let's run this again okay so we haven't yet we can just put an underscore so that we don't need to use that or you could say the stream let's run this again so now you can see that we are generating um this random numbers for bpm and we are also sending the heartbeat so let's go into mongodb and just find this again so we can see that everything is inserted here so this is what client streaming is i mean we are not seeing anything in server because not we are not printing anything so i'll just i'll just print the stream here inside or i'll just print the message which you're getting from the stream and run the server file and i can run the client now so you can see that we are getting this and finally when we get nil that's when other stream is closed and the client knows that so that's how live heartbeat works we sent multiple streams of live heartbeat request and we got a single response from this so the single response we got from the client is just a single string of them all um so that's what we got in return so we have done um urinary and client streaming so now i think it's time to move to server stream so this is really really similar to what we have done before it's just uh you have to stream it from server instead of uh instead of client this server streaming for it's known as heartbeat history we have here what we're going to do is that we're going to send one single request with either user's username in it the server is going to return all the documents uh who who's uh i mean whose document relates to this username or you could say whose bpm belongs to this username so let's go to heartbeat bb slash heartbeat.proto oops heartbeat.proto and we need to define a new service now so you could say rpc similar to that so we're going to say heart beat history and this time we are not going to receive a stream of requests we are going to only give you one request so its heartbeat history request and which returns a stream of responses so we could say heartbeat history response that's it so we now we need to define this response and request so you could say message heartbeat history request so the request is quite simple it's a string with the username is equals to one you only need the username and the response is going to be let's say heartbeat history response the response is going to be of type i mean we're going to return a type heartbeat with a name heartbeat it's heartbeat which is one so that's it it's actually reverse now this in this case because we are using response we are going to send a response of type heartbeat so the response must contain the bpm and username in it so that's what we're going to do so now we have done this we need to compile we should not forget to compile this so let's compile this and then we can start from the client side let's uh just send one single let's create a new function first and let's send the username in it that's it and let's say heart beat history and this is going to be the same heartbeat pb dot heartbeat what was that service client so what we're going to do is that i'm going to create a new request new request or you could say new history request um this is going to be of type heart pv dot heartbeat history request so this needs to take a use this takes a username the username is going to be monastery ram and we're going to send this to the service so you could just say c dot heartbeat history and the context is context dot to do and new history request and make sure you send it with it as a reference so put the ampersand symbol before the request so this actually contrasted background so we can just send this and after sending what we need to do is we need to wait for the responses or we need to receive the responses uh until or how many we don't know how many responses the server is going to send so we need to receive all the responses so for that uh what you can do is you can this heartbeat history returns a stream so you can say rest underscore stream which is equal to this so now we are going to do a similar infinite loop and for this uh similar to the client streaming we are going to say reststream.receive which will receive the let's save this first which will receive um the stream details so we can handle the error here and if error is equals to io dot eof then i want to break just want to print the message too so it's quite simple for the client side because we are just sending a request that's it so now let's go to the server and let's uh let me go up let's create a new function and tie that to servers and it's heartbeat history and let's see what we have they have defined in the proto grpc protofile and let's do server search for server and we have heartbeat history function here the first argument is the heartbeat request and the second argument is our stream so there are two arguments coming and we can just return the error so the first one is going to be the request so it's request of type pointer type heartbeat underscore pv dot heartbeat history request and the second one is going to be stream of type heartbeat pb dot heartbeat history not request but uh underscore heartbeat history client sorry so first i want to acknowledge this so i want to say heartbeat history call so that we know this is being called here and let's see if this request actually contains the username which we sent from the client so we can just grab that so you can say request.getusername and now what you're going to do is we're going to querymongo for to get all the documents which has the user imminent so what i'm going to do is i'm going to um you know this when we have created this hard item which has the structure or the model for database and this we have many documents or you could say we have multiple documents with this username right so we need to first store these results somewhere so we need to create an array for that or you could say a variable for that so we could create an array of heart items so that whatever the results are we could just store this so store it in this result so what has is that it has casa com error let me show you what it has we could do collection dot find and what we need to find uh it has a context first let's say to do context because we don't know what the context is and because also we're not getting in context from the client side so that's a context and we need to send a filter here now oops so we need to send a filter filter here so we don't know what the filter is but we can define the filter here so this filter is going to be bsun dot m so m in this m stands for map so we create a map so we want to have a username which is of value username so this username stands for the key and this is for value so the value is what i'm getting from the client right now so i'll send that filter and i'll handle the error now we have something called cursor dot dot all and what this takes is that it takes context.background and we can just send the result or you can say the difference of the result data which is or you could say result we can just rename this to result data and we can send its reference so that um whatever whatever we get the results we get are put into this array and now we can use this array so what i want to do is i want to loop through this array so i can say for i don't want the index i just want the value in the range of this result data what i want to do is that i want to stream stream this result data or the value to back to the client so i can for that i need to create a response so let's say history response is equals to heart beat underscore pv dot heartbeat history response so if you remember we have the response inside that we have another heartbeat and this has heartbeat pb dot heartbeat and now we have bpm which uh is v dot bpm and we have the username which is v dot username so of course we don't need to use them because this is all we are getting back so we can put the commas now what i can do is for every iteration i can just say stream dot send this history response and i'm sending the reference again i mean reference instance i'm sending the response with the numbers we need to say there is no error so we can just say return then actually we need to put that here we need to define it expects a return type error so i think we can test this now let's go back here let's restart the server and now we can run the client but i think we haven't called the function now client so live heartbeat is complete now i think it's heartbeat history of c so we can go and just call run the client file we need to handle the error so we can run this again so you could see that we are getting our heartbeat history call and we also getting this response here and you can see we are getting much much uh many many documents from for the user in a monastery ram so yeah that works so this is being streamed actually from the server side so if you have uh if you want to see how it's if it's really being streamed or not you can just put this time dot sleep here and you can say one into time dot second so every one second it sleeps and then prints the message so that would be then you can really see it's being streamed i mean it's just a graphical thing you could see that it's being i mean this is what we have actually received but we are sleeping and for one second so this is right now what we are receiving the finished heartbeat history and next is the final one which is our bi-directional streaming and we are calling it normal abnormal heartbeat so in this case we will be sending a stream of request and then server would be responding to all those requests uh it will be i mean both both the sites will be streaming back to back so in our application what we are going to do is that um the client would be sending a request with a bpm and the server would be sending back if the bpm is normal or abnormal so let's say if the human's normal heartbeat bpm is 60 to 100 then if the client sends 110 then the server must respond it's up now so let's update the profile for that we are going to have the final rpc for which we are going to direction by directional stream here so we can say normal abnormal heartbeat returns something so if you have seen for client streaming we are putting the stream variable before the request and for server streaming we are putting the stream uh variable before the response so for bi-directional streaming it would be a combination of both uh we will be putting the variable stream for both uh requested response before because both service so server and client are using this stream variable so you could just say normal abnormal heartbeat request the naming is getting quite long but i think it's okay and returns a stream of normal abnormal heartbeat response let's save that we need to define that here so let's say normal abnormal heartbeat request so the request is going to be single thing which is the bpm which is going to be of type int 32 i think it's in 32 bpm is equals to 1 and the response that's a normal abnormal heartbeat response is going to be a string because will be let me tell you because we will be sending a string saying uh given bpm is normal or if it's abnormal we'll be saying given bpm is abnormal so that's the only thing we'll be doing so let's we have defined this let's go back and compile this that's an important step and we have compiled it so we can go into the client first to write the function so it's normal abnormal heartbeat and it takes heart pb dot heartbeat service client what we did for client streaming is that we send we run a loop and stream from inside for every iteration we stream it so similar to that we can do something like this so for every iteration i want to create a new let's say normal abnormal request so which is going to be heart beat underscore pb dot normal abnormal heartbeat request so there will be a bpm for this and this is going to be generate bpm i want to send this i want to send this in a stream so i can say stream dot send and new and a request and then just send that so we need to have a stream variable for this so we need to call that here so what we similar to what we have been done before it's stream comma error which is c dot uh normal abnormal heartbeat and it takes a context which is context dot background and we need to handle the error so um we have this so we can just print a line here send i can just do printf sent a value and that is going to be new any request oh there should be a comma we need to close the stream so we need to say close stream dot close or close send it's new and a request we are sending requests or we are streaming from the client side but we also need to receive from the client side at the same time so let's go down and let's uh listen for server streaming what we did is that we have received when we put an infinite loop and we listened for until the server sends us and close flag so until that we were streaming i mean we were receiving for the whatever server was doing so similarly we can do this here you can say stream dot it's received and if error is equals to io.uf then you can say break and just break out or else you can handle error first and then you can say print ln i could say printf received the value which is message so let's see what happens if we send this but we need to define the server file also we need to define the function so let's go let's create a function here which is of type star let's attach it to server and let's say normal abnormal heartbeat so for this because this is streaming by directional streaming it is also server streaming because it's a bad action for me so we'll be receiving a stream of type heartbeat underscore pb dot heartbeat service underscore heartbeat is um is that normal abnormal heartbeat server so this is quite a long name actually so uh now what we do is for server streaming previously what we have done similarly for we can just listen for client i mean for client streaming what we're done we just listen for whatever the client is sending so we do the same here for request and error i'll just say stream dot receive and uh if error is equals to io.uf then you can just return nil same speaking of which we need to return an error here we are getting the bpm from the request right so we need to get that out let's say request dot get bpm or you could say yeah you could say just get bpm so um the usually let's so let's create a variable result which is of type string and we need to check if this bpm is less than 60 and bpm is greater than 100 if it is so it is abnormal so we could say result is equals to i would say fmt dot s printf user heartbeat off i can just put this like this off so once a value is abnormal and the value is bpm or else i will just copy this is normal normal and i can just stream this back to the client and the stream dot send result but it would not go because you need to send it in a form of request so we can just say response or you could say normal abnormal response is equals to heartbeat pv dot normal abnormal response and it contains a result which is this result and here we send the any responses so this we comment this out and we say normal abnormal heartbeat of c you go to the server and run the server so what needs to happen is that the client needs to send a bpm and the server needs to respond if it's normal or abnormal so that's the result we are expecting let's run the client client file it's because how we are getting i mean this is this should be an address here itself instead of here because when we are printing this it is printing this request itself so we need the to print the difference so moving this here or here i mean removing this and putting it here i think it's the same but just for printing we're just putting it here let's just print it up here let's say fmd dot println normal abnormal heartbeat called i think this condition is wrong it should be r so we can just run this again after restarting the server so we're getting the our results back so we send this bpm and we are receiving as a result back so what what we can do is we can create a go routine and put it um we can create a routine and put back put the put this block of code into that core routine so instead of putting a separate function we can just call the function with a go keyword here so create a function and call it right there but uh if you if you know uh the problem with go routines is that it spawns this in a new thread and it goes down so this finishes this function and then this thread is also gone proof like that so we need to make sure that this function waits for this function to end so we need what we need to do is we need to use weight groups or channels to make this happen so if you see my if you have seen my previous previous video you can get a difference for how to use weight groups so let's create a weight group variable so let's say where our wg is of type sync dot weight group so we can say before this go function we can just say wg dot add of one because there's just one go routine there's just one go routine uh i'm going to spawn so whenever i'm going i mean after this i'm going to do wg.wait to wait for the go routine so whenever i hit the last page or last end of file instead of i mean i break from this function but also from this loop but also i will say wg dot done so so as to indicate not to wait let's put this here so sending back response which is the result so let's restart the server and we can see that um this results which we have received and i mean which we have sent and which have received are in you could say in sync so i mean the the part which is to be focused here is the bi-directional part where we are streaming back-to-back so that's the one we need to focus on and uh i think uh that is what i wanted to say in this bad action video i know this is a quite a long video but i just wanted to make sure that this concept is uh implemented in the right way so there are much more uh depth depth concepts in this grpc i mean it's a very wide topic but um you know there are there are deadlines and there are ssl uh there's the ssl there is deadlines and uh there are much more i think handling errors in grpc that's very important but i'll be making a separate video on that because this video has been so long so i just want to keep it short i mean it's already long but i just want to stop it here so i think um that's it for this video and uh thank you for watching make sure you like share and subscribe this video thank you

Channel: Mano Sriram

Views: 546

Keywords: grpc, protocol buffers, manosriram, mano grpc, grpc golang, grpc tutorial, gRPC, proto, bufs

Id: jboQkdNJiRQ

Length: 83min 7sec (4987 seconds)

Published: Sun Aug 22 2021