When RESTful architecture isn't enough...

for the longest time rest has been my go-to sorry dexra architecture for building apis and I've had little to no complaints however rest hasn't been the only API architectural pattern that I've used with the other major one being RPC RPC comes in a number of different flavors such as Json RPC HTTP RPC and the main one we're talking about in this video grpc before we talk about it however let's look at what RPC actually is RPC which stands for remote procedure cool is a form of interprocess communication it works over HTTP similar to rest but instead of being resource based it works with actions for example let's say you want to create a new Pokémon in a Pokémon based API with rest you'd send an HTTP request with a post method to the/ pookemon endpoint with your Pokémon details in the request body if we want to do the same with Json RPC we'd instead send an HTTP post request to a generic endpoint with the following request body specifying both both the action or method that we're taking in our case create Pokémon and the data in the form of the pams field now I know what you're thinking why would anyone choose this over rest and you're correct creating resources isn't the best example of using RPC a better example would be an API for a calculator service in this case it's actually rather difficult to model a restful API and that's because of the lack of any tangible resources or objects for us to model a restful interface around instead calculators typically perform actions such as addition subtraction multiplication and division in an RPC interface these methods are incredibly easy to model in fact it's why slack decided to model their API using HTTP RPC instead of rest whilst their API does have the concept of objects they mostly have methods and ultimately that's where RPC shines so whilst it's not its most optimal use case you can use RPC to create and manage resources which when compared to rest gives a a lot more flexibility for a Services API whilst this does imply that RPC is more powerful there is one major drawback this drawback is the lack of self-documentation for example if I told you I had a well-designed restful API that managed dragons there's a good chance you'd figure out how to use it even with little documentation the same can't be said for RPC however which is why I've always opted for restful apis in the past that is until grpc came onto the scene and brought with it one fundamental change that change is protocal buffers or protuff for short which is a language neutral extensible mechanism for serializing structured data it's similar to XML or Json but with one key difference it's a binary protocol instead of a text based one which gives it the benefit of being more compact however in order to accurately decode a prodab payload you require the schema that was used to encode it in the first place you might be thinking that this sounds like a drawback but in reality it's actually a strength the requirement of the dot Proto file acts as a sort of contract for an API so what does a protocal buffer schema look like well here is one for a calculator service I've created already this probably doesn't make a lot of sense at this stage so instead let's look at a service in action that uses this contract to do this we're going to need an instance of our calculator service to interact with fortunately I have one which you're able to deploy an instance of yourself for free using the sponsor of this video Acorn to deploy it all you have to do is click the deploy Link in the description down below then sign in with your GitHub account click the deploy button and it will create an ephemeral instance of the service for a couple of hours no cost no code to run which is much more secure than running locally and it has all of the infrastructure you need to test with it's a really cool product which allows me to easily share deployments with you for free we'll talk a little bit more about acon later on but for the moment that's all you need to do after a short while you should get back the URL of your deployed instance make sure to take a note of it or copy it to your clipboard the next thing we're going to need is a grpc client for this video I'm going to use grpc UI one because it's open source and two it seemed to work best for some certain use cases however you may want to use an app like Postman or insomnia yourself either way make sure to First install your chosen client as per your operating system once installed you then want to connect to your deployed service using the URL that was generated earlier if you're using grpc UI like I am you'll want to remove the https from the start of this URL and add the Port of 443 when the UI opens the first thing you'll notice is that all of the RPC methods are populated this is done using reflection which is an opt-in feature of grpc it's not recommended to enable it on external facing services but for our test server it saves us having to mess around with any dot Proto files until we learn a little bit more about them let's go ahead and call the ad method of our service all we need to do is select the method fill in the required parameters which are the two numbers we're going to send and send off the request we'll get the result of our two numbers added together showing that everything is working we can do the same with any of the methods that our service provides this is pretty cool and shows the strength of having a contract associated with our service which is really difficult to achieve for the other types of RPC to do so you'd need to Define your contract using something such as an open API spec which isn't the nicest thing to write then you can load the open API spec into a open API UI and get something similar however there's a catch your open API contract isn't always guaranteed to match your actual service so even then you need to add contract testing into your API to ensure parity with the open API specification all of this is a lot more work and maintenance just to get to the same feature set that grpc gives you for free so now that I've hopefully convinced you of the benefits of grpc let's take a look at actually building our own version of the calculator service we were just using both protuff and grpc themselves are language agnostic but I'm going to use use goang in this video as I feel it does a good job of showing the endtoend process if you want me to do a video on grpc in a specific language then please let me know in the comments down below let's begin by creating a new directory and initializing our project using go mod in it next we'll want to create a folder called Proto with a file inside called calculator. prototo which will then open up in our editor the first line of this file is used to specify which version of protuff we're using in our case Proto 3 the next line is used to name the package of our protuff file I'm going to call mine calculator then we Define our actual service which again is called calculator inside of the service block is where we're going to specify the RPC methods that our service provides we do this by using the RPC keyword followed by the name of the method in this case we're defining the at method we then specify the inputs for this method inside of the parentheses here our input is called calculation request which we're going to Define in a minute next we'll add the returns keyword followed by the type we wish to respond with we've named our type calculation response let's go ahead and create both of these types in grpc types are known as messages and we can Define one with the message keyword followed by the name we wish to give it here we're creating a message called calculation request inside we then need to define the body of this request known as the fields let's add our first field which will be the left hand side of the calculation the first thing we need to Define is the fields type in our case we want to set this to be an int64 after the fields type we can give the field a name we'll call ours A lastly we need to set a field number this number must be unique across the fields inside of a message let's go ahead and set this to the number one we can now do the same for the right hand side of the calculation called B with our request message defined let's go ahead and do the same for the response message inside we'll Define a single field called result which is of type int64 and has a field number of one lastly we just need to add in the following line which is an option when generating Goan code this option will specify which package we want our generated code to reside in once that's done go ahead and save the file with our service defined in protuff we can now use it to generate some grpc code to do that we first need to install a couple of tools the first tool is protoc C which is the protuff compiler this can be found in the protuff package next we need to install our language specific plugins for goang that's the following two packages Proto Cen go and protoc Cen go grpc these can be installed through goang using the go install command or through your operating systems package manager I've also left instructions on the GitHub repo for this video you can find the link to it in the description down below with that done we're now ready to generate some code to do so we just need to run the following command in our terminal which will load all of the dot Proto files in our Proto directory and generate both go models and go grpc code this command is a little bit gross to type however it's usually a good idea to add a make file or something similar so that we don't have to keep typing it out to do so create a new make file in the project and add in the following lines now we can run make generate inside of our project directory and our protuff will be created for us if everything worked correctly you should have a directory or package called PB and inside of it we should see our generated code now we're ready to go ahead and use this to create a service the first thing you want to do is install the grpc package from Google using the goget command once that's completed create a new directory tree called server and inside add a new file called main.go opening this up let's define our package and create an empty main function inside now we can begin creating our server let's first Define a new struct called server which inherits from the unimplemented calculator server found in our PB code g package make sure to import both this and the context packages next we need to Define our ad method we can do so by using the following function definition which accepts a context. context and a pb. calculation request finally we'll return our response type of pb. calculation response and an error if one exists the actual function itself is pretty easy we'll just return a response which contains the sum of our two numbers and a nil for the error pretty simple All That Remains is to load up our server inside of our main function to begin first import the following four packages then scroll down to the main function and create a TCP listener to listen on port 8080 followed by creating a new grpc server type let's also add in the following line to enable reflection which we'll want for testing at this time we then just need to register an instance of our calculator server to the grpc service which will then serve using the TCP listener we just created as a note it's worth mentioning that grpc only works over htttp 2 rather than HTTP 1.1 which is why we're using a TCP listener now if we run this code we can test everything is working by connecting to it with our grpc client using the following command if we go ahead and test our ad method we we can see that everything is working as expected so far so good let's go ahead and get this deployed to the cloud before we add any more functionality to do that we're going to need a Docker file let's create a new one and add in the following lines this will build our app using go 1.20 and we'll use a different Docker stage in order to run the binary with the docker file in place we can now test that everything is working by opening up a new terminal window and running Docker build next up I'm going to show you how to get this running on acor to do so is pretty simple we first just need to download the acon CLI to our machine which you can do pretty easily using Homebrew on Mac OS or following the documentation on Linux with the CLI installed we can then call the acon login command to log to acon Cloud we should get a success message letting us know that we're in now that we're logged into acon we can go ahead and start writing our acon file this file is basically a really simple way to declare complex applications which when pushed to the acor platform will spin up the environment as we've declared it inside of this file we'll declare a container for our calculator which will build the local Docker file we'll then expose Port 880 with an http2 interface which is needed for grpc to then deploy this all we have to do is call acor run and pass in the name we want to give to the app acor will then build our app for us and deploy it providing us with the URL of our deployed application if we then take this URL we can use it with our RPC client when testing this for the first time I received a context canceled when trying to connect most likely due to DNS propagation I haven't seen that error come up since so if it does happen to you just wait a few seconds and try again connecting to our server we can see our OD method is available for us to interact with and works as expected so this is great we've just managed to create our first grpc method the method we created is pretty simple and isn't likely to fail but what about a method that could cause an error how would we handle it to understand error handling in grpc let's look at implementing The Divide method of our calculator service to do that head back on over to the calculator. prototo file and add in a new method called divide this method will accept the same calculation request and return a calculation response let's save this file and use the make generate command we made earlier in order to regenerate our code now let's head back into our main file and implement this method we'll do this naively at first which is just dividing input a by input B pretty simple now let's run this interactively on acon using the acon Run command with the I flag to test our divide method I'm going to use grpc Curl which is a CLI tool for sending grpc requests as you would curl if we cool our method with an input of 10 and two then we get back the result of five which is expected however what happens if we try to divide by zero well it crashes the server not great but this is also expected bad input is a common problem when it comes to apis as you can never really trust the client ever so our service need needs to validate the input and return an error if there's something wrong with the values in a restful API this error would look like a status code of 400 being sent back to the client with perhaps an error message letting them know what went wrong when it comes to grpc however things are a little different as our service is backed by a contract we can't be Loosey Goosey about what we send back to the client so what do we do one approach could be to Define an error message within our protuff schema and we could then add it as an optional field in our calculation response this means we'd also need to Define our result as optional as well given that our response would only ever contain an error or a result this approach is a little clunky however we'd also have to repeat this for every response message we defined luckily for us there is a better way let's jump back on over to our implementation code in the main.go file in order to handle our error let's first import the codes and Status modules of the grpc package next scroll down to the Divide methods implement ation let's first add in a check to see if the diviser the right hand side is zero if so then we want to return an error in the function which we can do using the status. error method this function expects both a status code and an error message string the code most applicable to our situation is the invalid argument error code and we'll set the error message to be cannot divide by zero let's now redeploy this code on our server in order to test it whilst acor is running in interactive mode if we run the touch command against our acon file it will automatically rebuild and deploy our project for us once it's done we can go ahead and rerun the grpc Cur command against our server as you can see the server no longer crashes and the client receives a response telling it what went wrong which provides a much better developer experience when consuming the API so that almost wraps up our grpc server but there's one last thing to look at before we begin implementing our client for this final task we want to add a new method called sum which will accept a list of integers and return the sum of them all to do this let's head back on over to our calculator protuff and we'll add in a new RPC method called sum this method will take a new input message which we're going to call numbers request and it will return the existing calculation response message now we can Define our numbers request in order for this message to contain a list of integers we can use the repeated keyword on a field which tells protuff we expect this type to be a list containing zero or more values with that we're ready to generate our code opening up a new terminal window let's go ahead and run make generate to generate our code and then start implementing this new method this should be reasonably familiar at this point the only difference is the new type that we've defined for this implementation we're just going to iterate over the numbers in the input array and return the response now let's go ahead and redeploy this using the acor Run command with the update flag we can then test this new method using either the grpc UI or grpc curl when using GR RPC Co we're setting the numbers as we would a Json array when we invoke our request we receive the result we're expecting back telling us that everything is working as intended that wraps up our calculator service but we're not finished yet so far we've been using pre-made grpc clients to interface with our server but in the real world we're going to want to use a grpc client within our code fortunately whenever we generate code with the protuff compiler we also generate client code that we can use let's look at how to do this to begin let's first create a new directory called client in our project with a main.go file inside opening up this file let's add in the package name import the packages we're going to use and Define our empty main function which we're going to fill in first we'll specify a CLI flag so we can set the server address when we run our code the next few lines are used to enable TLS which we're going to need when deploying to acor if you're running this locally or deploying without TLS enabled check out the source code on GitHub for how you can disable this next we'll want to create a grp connection using the dial method of the grpc package passing in both our server address and the grpc options let's handle any errors and add a defer to close the connection when our main function is complete this code creates the basic grpc connection for us which we can now use with our generated protuff to do so import the PB package which contains our generated code and then create a new instance of the calculator client passing in the connection with that we can now cool our RPC methods using this connected client here is how we would cool our recently implemented sum method making sure to also use our generated types with our code completed let's go ahead and run this using the G run command making sure to pass in the URL of your deployed service you should get back the following response letting you know that everything worked as expected with that we've managed to create both our calculator service and client all using grpc and protuff I've intentionally left out these subtraction and multiplication methods as an exercise for you to complete as you should have all of the building blocks available to do so before you go off and do that however I have one last thing to show you whilst I've given you the building blocks of grpc you may be interested to know what a grpc crud app looks like so I've created one that you can reference this is yet another acor app that you can deploy yourselves which also integrates with postes for persisting data you can find a button to deploy this yourself in the video GitHub repo which is linked in the description down below this repo also happens to contain the code itself if you want to reference it this crup provides all of the operations for creating reading updating and deleting Pokémon the one feature I really want to show is inside of the create method if you look at the request data you can see we have an ID field a name field and a type field which is an array if I go to add a new item to our type array you'll see that we're presented with a drop- down list this is done using an enum in our do Proto definition which allows us to easily constrain the possible inputs providing us with a greater level of type safety while it is possible to achieve this using a restful API it's much harder for us to communicate that enum representation to our clients at this point you may feel like I'm saying grpc is superior to rest and in some ways it is but even with everything I've said in this video there are some drawbacks to using grpc some of these drawbacks include the lack of a mature ecosystem difficulty when introspecting data and the lack of native web support however more and more solutions are being created to help with these drawbacks one such solution is Envoy which can act as a reverse proxy converting rest into grpc but that's a video for another time whilst I still build a good number of restful apis grpc has inspired me to take a contract first approach changing the way I build services for the better either way I'm interested to hear more from you what do you think of grpc let me know in the comments down below I want to give a big thank you to Acorn for sponsoring this video they have a really cool product which has been awesome in allowing me to share these services with you otherwise a big thank you for watching and I'll see you on the next one

Channel: Dreams of Code

Views: 245,210

Keywords: grpc, gRPCui, grpcurl, api, rest, resftul, architecture, software, development, coding

Id: _4TPM6clQjM

Length: 21min 1sec (1261 seconds)

Published: Tue Nov 14 2023