Building an Basic API with gRPC and Protobuf

hey guys my name is tensor welcome to another go tutorial video today we're going to look at gr PC inside of go in the last tutorial we took a look at our PC and we took a look at a fairly basic implementation the next logical step is to look at the G RPC protocol now G RPC was created by Google and the basic idea is that it makes use of the HTTP 2 protocol and that gives us a lot of benefits and it uses this along with another technology called protocol buffers to make extremely quick and extremely performant our pcs my favorite feature of G RPC is that it's completely language agnostic this means that you can build a server in one language and a client in another so for instance I could build a server and go and a client in dart or I could build a server in rust and a client in say C++ or Java or Swift and there are a lot of languages that are supported also the proto buffers specification itself is completely language neutral and that's sort of what makes it so performant and powerful now to get started with G RPC you're going to need to install a few things you're going to need to specifically install the protocol buffers compiler you can get it from this developers.google.com backslash protocol buffers website if you go to this download link it will then link you over to the github page and then you go to releases and what you want to do is grab the release version that corresponds with your operating system so my operating system is currently win 64 so I would click this one and then I would unzip it somewhere on my path once you have it installed and on your path you can go ahead and go into your command line and run proto C dash dash version and this will give you back the current version of the compiler now specifically for the go programming language we're going to need a few packages first we're going to need a package called Google golang.org backslash G RPC and this is just the main G RPC package then to properly use the protocol buffers you're going to need this package which is github.com backslash golang backslash protocol backslash prod oxy Jen - go and once you've got those two libraries you can go ahead and start coding your G RPC server and client in this tutorial we'll be making a fairly simplistic application we'll make a server that will allow us to essentially add and multiply numbers together and then we'll have a client which will serve that server to the API so we'll have API endpoints that we can call from the client which will then call to the server and then execute the multiplication or the addition and then that will send back the numbers to the client which will then serve them back to us as JSON for this to work however we need to create a proto file and this is essentially a stub that tells go how it should encode and decode the various pieces of data at the top of the file you need to define these syntax that you want to use and the latest version of the protocol buffer syntax is called proto 3 this will be the syntax version that we will be using then you want to define the package for this particular file because what we're going to do is actually generate a go module based off of this file that we're writing so the package for the go module will be the package that we define in here in my case I'm just defining it as package proto because the folder that this is in is called proto as well.these protocol buffers are interesting because they're fairly flexible and they're also language neutral you can kind of think of them like an XML file or a JSON file but much much smaller and much more efficient so here we're defining what's called a message and this is essentially just the data structure that we want for our request so we want to add two numbers together so we're going to send in to numbers number a and the number B both will be integers 64 type so we'll create a message called your request and then we'll put those two inside of it this message will be our request serialization message which means that when we send the request from the client this is the format that we want to serialize now notice that I've added these numbers at the end of each of the lines these are not values for the integers these are just numbers that denote what size these data types should be when they get serialized so each of the values inside of a message needs to have a unique number attached to it and the numbers between 1 and 15 mean that the value inside of the request will only take up one byte when it gets serialized if you have more complicated data structures then you can denote that you want them to take up more than one byte and if you want them to take up say 2 bytes then you would use a number between 16 and 2047 now we can go ahead and create the response message this will be similar to our request except of course it will be what our server will respond to as a result of getting this request we wanted to respond with one single number which will be the addition of these two numbers and we'll call that number of the result and again we only want it to be one byte long so we'll denote it with the number 1 now that we have our data types we need to create what's called a service just for reference for you guys there are various different scalar types that we can use for each of the values inside of our messages we could use doubles we could use floats we could use integers 32 and 64 we can use boolean x' we can use strings and we can use bytes now that we have our data types let's go ahead and define a service or call our service add service and like with the messages we want to use the keyword service followed by the name of this servus and then inside of this service we want to define the various different remote procedure call functions that we want to be able to call on our data so in our case we're just defining a function called ad which takes in the request and then returns a response and notice the format we start with this RPC keyword followed by the name of the function followed by what the function takes in and it must take in some data type even if it's an empty data type then we need to use a keyword called returns now keep in mind this has an S on the end it's not just a return and then we need another pair of parentheses with the actual return result which in this case will be response of course our service can also have multiple functions inside of it so if we wanted to add another function called multiply which also takes a request and then returns a response we can and actually I'm going to go ahead and leave this in here so we'll implement both an ad and a multiply function for our server and our client before we go ahead and compile this proto file that we just created into a go file what I'm going to do is grab all of the include files that came with the product compiler and put them into a folder called third party so in my case I put my protobuf compiler in a folder on the C directory called proto see inside of protobuf and then inside of here we've got the binary called proto C and then we've got the include and if I just copy this Google folder I can then paste it into this third-party folder and the reason I'm doing this is because it actually makes things a lot easier on us when we go to run the commands to generate the actual file now we can go ahead and run the proto C compiler to build out our generated go file we run proto C and then we add a flag which will point towards the folder where our profile is inside of and then another flag which points towards all of those include files that we just added to this third-party folder then we need to specify that we want to output a gr PC file inside of our proto folder and then we'll just specify the name of the file that we're going to use to generate this gr PC file with now that I've run this file you can see here we now have a service dot P video file inside of the proto folder and if I open it up you can see here it says code was generated by proto see Django and we've got a ton of generated code inside of here we've got our request structure which is now a structure with a and B inside of it along with a few new field and request has a bunch of methods attached to it the most important of which are these getter methods which allow us to get the a and the B out of our request we've got our response struct which has the result field inside of it again this response struct has some methods and the most important is this getter then down here we have a interface called add service client which implements and add an a multiply method we also have a function called new add service client which creates a struct called add service client and this will become important when we actually build the client itself we've got an add function as well as a multiply function inside of here which are attached to this add service client type and we've also got an add service server interface which has add and multiply inside of it along with this register add service server function which again will become important when we build out the server now speaking of building out the server let's go ahead and do that I'm going to come into this folder that I built already called server and create a main dot go file and I'll give this package main and the first thing that I want to do inside of this file is create a structure called server and the main reason why I want to do this is so that server can implement the interface that was generated by this service PB echo file that is this add service server interface so our server type needs to implement both of these functions to implement this interface first let's start with the add function and this add function needs to take in the context which we'll name CTX and then a proto request which will name request and then it needs to output a pointer to the proto response and the error and notice up here at the top I'm actually importing from the file here which remember we gave it the package of proto the actual function for add is actually pretty simple we can get a and B by using our getter methods on the request that is coming in here so we can just say get a and then get B and we can then bind them to a and B respectively and then we're just adding these two values together so we just add them together put them into another variable called result and then we want to return a pointer to the proto respond struct where the result is equal to our result type and then the error is nil now we need to implement the multiply method again this will be implemented on our server structure and again it needs to take in the context and a pointer to our proto request and then it will output a tuple with a pointer to our proto response followed by an error the function body will be very similar to the add function except we'll just change the computation here to multiply the two integers together now that we've implemented the two methods to our server struct we can set up the rest of our server inside of our main function let's first start by opening up a TCP listener and listening on the port for T for T and then of course we need to deal with the error that potentially come out of calling this function now to actually get a server that we can use to wire up all of this stuff we need to use the Google golang.org backslash G RPC library and we can use a function called new server from that library to create a server object we can then take this server object and use a function from our service dot P video file called register add service server passing that server and then pass in a reference to our server structure type so this function here creates a new G RPC server and then we register our service on that server using the type that we created here and this function then because we're going to be serializing and deserializing data we want to grab the reflection library from the g RPC package and call the register function again on our server now finally we can just use the server that we just created here call serve on it pass in our listener so that it will listen on TCP port 4040 and then of course check to see if the error exists or not and then if it does the world panic and that actually sets up our G RPC server now let's go ahead and create the client so I'm going to go ahead and create a main that go file inside of our client folder and of course like with our server this file will be packaged main and it will have our main function inside of it so that we can actually execute code inside of it now we need to connect to our server and our server remember is on localhost 4040 we can do this using the G RPC library so we just get the connection and an error by calling G RPC dial and we dial into localhost for T for T and the connection itself is insecure because it's not using HTTPS so it's just G our PC with insecure on the second argument of the function we then need to of course handle the error we can then call to the generated proto library and use the new ad service client function that was generated passing the connection and this will give us a client now because I want to create a set of API endpoints to call to our G RPC functions I'm going to use a library called gin you can get this library at github.com backslash n dash donek backslash gym and we can create a gin server by simply calling gin default and gin is a pretty cool library because it allows us to very easily add endpoints to the server with our server here we want to add to get endpoint one for ad and one for multiply and both of these will take in two arguments so the routes will look like ad backslash a backslash B and multiply backslash a backslash B and then both of these getters take in an anonymous function which takes in the gin context pointer we want to be able to parse the two arguments that we're getting from A and B so we'll use string converge dot parse you int and grab the argument from our context / am function or we just passing a key so for a we pass an A and then for B we pass in be inside of this parse you int function we then want to pass in what base of the number will be in this case it's base 10 and then the type of integer which will be integer 64 then if we do get an error from running this string conversion we want to use the context to create a JSON object where we have HTTP status bad request and then we can use a gin map called age to add a few keys to the JSON so the keys will just say error invalid parameter a and then for B it will be error invalid parameter B and of course we can add the same exact lines to the multiply handler so that it will work exactly the same now that we have both a and B we take the a and B variables we need to cast them as integer 64 and then we can put them inside of a proto request structure and again we can do the same down here for this endpoint too now with our request created we can use the client that we created up above and in the add endpoint we can call client add with the gin context and then the request and then this will add the two variables together for us on the server side and then send back a response we can then take that response and if we don't get an error from executing this we can create a new JSON object with HTTP status okay and again we'll use that gin h map to create some keys or will just say result and then use the fnts print function to make it look like a really nice string with our value inside of it and we're getting the value from the response and it's just the result field now if we do get an error from running this that means that we have a server error so we'll just throw an HTTP status internal server error and then we'll add that error to gin h the only major difference between the add and the multiply handlers is down here in the multiply handler we call a client multiply rather than client dot add finally to finish up this API let's just run the API on a new portion of our server which will be localhost 8080 so we can use the G variable that we created using the gin default function up above and the G variable has both of our routes attached to it and that has a run method attached to it and of course if we get an error then we just say that we failed to run this server now we can go ahead and run our application by going into the server folder and using go run may not go and then we can also open up another command line go into the client folder and run may not go for it and you'll see that Jin has a few more items that appear here and now I can open up Mozilla or Chrome and just type in localhost 8080 backslash ad and then pass in two arguments in this case 152 and then 535 and when I hit enter you can see here we get back a result I can also use the multiply function so I'm going to use the same arguments and then I'm just going to change this to multiply you can see here that we get back our multiplication function and of course this is pretty fast as well and of course we can use some fairly large numbers and it's still fairly quick because everything is being crunched down to very small binary formats with our numbers being passed back and forth very quickly between the client and the server and then back to the client again and then up to our API alright guys well I hope you enjoyed this tutorial if you did feel free to like and subscribe if you have any questions or comments feel free to leave them in the box below and if you dislike this video then by all means download it as much as you like have a good night

Channel: Tensor Programming

Views: 70,347

Keywords: protocol buffers go, golang protobuf, golang grpc, go grpc, server streaming, go server streaming, go client streaming, go API, go gRPC API, go remote procedure call, microservices, go gRPC, go programming language, microservices tutorial, learn to program, microservices go, go tutorial, golang tutorial, golang gRPC, golang RPC, learn to code, programming tutorial, go programming tutorial, go rpc server, go gRPC server, golang gRPC API, microservices go example

Id: Y92WWaZJl24

Length: 20min 55sec (1255 seconds)

Published: Wed Feb 06 2019