Building A Complete API with GRPC ft Lucas Santos of Microsoft | HackBuddy

right so uh well i i have my just to to give you my my things if you want to try it out um this is my website it's el santos from lucas santos dot dev it is going to redirect you to this and you this website has all my references all my social networks and if you have any doubts anything please do contact me and we can talk we can chat i'm always here to solve things and if you want to try it out in your home tried out your code and everything uh this is the repo i'm gonna put out i forgot we have a chat over here so uh let me copy these and put on the chat uh where is the chat okay so this is the website uh this one is the demo repo i'm gonna show you here but the idea of this presentation is to reflect spring explain the grpc protocol and the grpc technology uh and then we are going to do like live coding and this is where everything that can go wrong will go wrong and probably something is going to fail but uh no worries but the idea of grpc is much like vj said uh it's a technology that allows you to communicate services more efficiently the idea behind it is not that it actually uses a different protocol it still uses http 2 like you know under the hood but it uses another technology called protocol buffers so protocol buffers are a way to uh you know to create a contract between two parts you actually called what your api is going to send and what your api is going to receive so you have uh like a real contract the size on both parts you don't need like uh open api specifications swagger and whatever things that uh you use to specify apis today uh because you can download these protocol files we call the program files and you can use them directly it's like a if you have like a strawberry specification that is runnable that is executable you can actually run them and you can actually execute this uh explanation you can execute the api straight from the profile okay so you don't need to uh have a very very you know a lot of things to do uh but the most curious thing that the the most curious thing about the djrpc is that the concept of grpc is not new but the technology is so it came from something a long time ago that's called remote procedure call or rpc but java was the one that actually implemented it uh very nicely there is something in java called remote java remote method invocation uh which is oh my god java oh here it is uh java remote method location should be and i think it was like deprecated in 1998 but uh uh it was something that was very widely used uh when you had distributed systems the idea of a rpc or rmi jrmi we have like this is that uh in your application you have this server and you have a client on the on the rest mode you have a server and you have a client the client sends out a request to the server and the server responds to that request but the request itself is stateless it doesn't have any thing to do with the client or the server it just has the required information to be interpreted by the server from the client so we have posted posts we have gets i have patches boots deletes and whatever thing uh the thing with uh rpc let me see if i can find here uh yeah let me see if i can find a very nice image of it i don't know if you're seeing this but let me get this a bit bigger so the idea on the grpc stuff is that you can actually instance uh the objects you have on the server on the client side because you have the contract so on java since it was like very object oriented and it was very typed uh it was uh it was a type language like unlike javascript or you can use jpc with typescript too but javascript itself is not typed it's dynamic so it won't have the same functionality but the idea is the same so on the client you have something that called it was called stub this tub was a representation of a remote object that was called skeleton the skeleton is the actual uh object it was the actual thing let's say uh our example we're gonna do here is a note taking app so we have a note app and we have uh the notes we're gonna insert and list and get from those things so on the server you have a skeleton that has all these methods the list method the boot method the ad method the delete method the gap method every one of those implement it uh you have this remote reference layer which i don't think it exists on the the grpc i haven't looked at it very further because it uses http 2 as a transparent protocol between two parts uh and you have the network layer so when you have the client you get the the proto file uh which which has the the all these uh definitions for your api uh it's going to build up and you can create objects from this profile so you can create a new note inside your client and manipulate this note remotely so if you have like note dot set id or something like this you're going to set the id of a note from the client and you have to save it on the server so you can actually call like uh notes dot add and note and you are going to send a request to the server and this server is going to run the add method and it's going to implement it uh as vj also said uh grpc is widely used by not only docker but uh kubernetes use it like a lot uh and keda uses it like a lot i don't know if you know okay ikeda is a project from kubernetes actually from cncf uh that helps kubernetes to scale based on events so if you have like um let's say i don't know if you're familiar with kubernetes or not i'm not going to be introducing kubernetes here but uh kubernetes is the distributed application infrastructure so if you have like several tons of applications like thousands of applications inside kubernetes and you have to scale those applications based on like how many messages you have on a wrap game queue uh cada can help you with that because it can reads the wrap it reads the wrapping cube kill and then it scales your applications based on this and this communication between cada and the revtmq is doing is done by something we call external scalar and the stainless external scalar is not nothing but a grpc interface implemented because the way jrpc works the way brutal buff works actually brittlebuff is the the star of the show here grpc is just the method but the way protobuf works it makes very very easy for you to implement someone else's interfaces because it is version into a git repository so you can only download this i don't know proto file and you can compile this profile and you're good to go and this is why purplebuff itself i'm gonna show you this also if you have time but uh this is why protobuf is something that is also used alone so let's jump into the code i think is going to be very it's going to be more uh it's going to be better for this uh i created a a new folder here uh with a jrpc demo uh the only thing we're going to have to do is to install grpc as dependency so npm install grpc uh and if you want to check it out with protobuf uh the other the other thing we're gonna need to install is another lib called google protobuf i'm gonna show you here let me increase the size of things and as you can see my cpu is very high because for some weird reason uh mac is not good for screen sharing and coding and all that stuff my mac is actually frying out it's going to fly over here as well uh so let me increase a little bit the size of this ui so you can all see it but as you can see uh i have two dependencies only so it's google protobuf and grpc this is the only dependencies i need for everything i'm gonna do uh and basically what we're gonna have uh is a file let me create this uh this folder over here this directory all right mac let's go you can do it i believe in so as you can see i have like a 100 cpu being used let me close some stuff up so i can free up some cpu right uh so basically here what we have uh is the the package.json file and we're gonna code something we are calling protofiles which is definitely the uh contract between our client and our server so in our server what we have is basically a file called notes.pro we're going to view a note taking out so it's pretty simple i'm not going to lie it's very very simple it's a very simple api that we're going to build but the idea is not to show how uh it's not to show the the the complexity of the project but how we can do it uh so basically what we have here uh is a syntax file from a printable file and these printable files uh are universal they can be used on whatever language you are on whatever thing you are so it has like outputs for different languages so what we're going to do here is i'm going to define the syntax for this they have two syntaxes it's probable uh two and pro three they have substantial differences so uh you can use proto2 but the thing is that it's better to use brittle three because it's newer and there is uh some changes like most of the the fields are required and it's specified as optional and so on uh and we're going to create a new thing here so when we want to define something that we're going to send or not send in this case we create a message so in this case i'm going to create a message called void which is just an empty object and the empty object is just like when you need to send nothing like i need to get the list of uh notes and the list of the notes doesn't have any uh input parameters so it's a void object and now i'm gonna define my api my service so we have a service and i'm gonna call it node service the node service uh is composed of two uh different apis uh which is the first rpc oh someone wants to enter the chat uh which is the first rpc i'm gonna call it list so this is the first method uh it's going to receive absolutely nothing and it's going to return to me a note list okay so every rpc every grpc method can be either a one-way stream or a two-way stream you can send a screen of data you can receive a screen of data and you can send and receive a stream of data as well so uh and you can actually only receive one parameter per uh rpc call so i cannot have like a list or i get with an id and a parameter i need to get have a get is a single parameter and this is why we generally use objects to do it okay the second one is the find and find is going to res to to receive the id so i'm gonna send the id uh and this is going to be note id and this is going to return me a note so as you can see we have different oh i'm missing the columns uh so uh we have different uh types over here so we have to define those types using messages so message we have like a message note note id okay we're going to define the way we're going to receive this so imagine you have a rest call and this rest call receives a single parameter which is like the id of the the like the id parameter from the url uh this is the id brand we're gonna you're gonna send which is a string uh id and it's this one that makes the whole difference because uh grpc is essentially a binary protocol uh so it doesn't care about the uh titles you give for or your messages in all your fields because it's all converted into indexes and these indexes are the way uh it's this indexes are actually the way jrpc takes to make it very very small because it can uh reference fields from their index only you don't need to have like a whole bunch of code to you know do this uh whole pattern matching stuff to match the key name with the value because you can only use you can you can use the simple uh indexes so you have to put this indexes and they have to be sequential so we're gonna define another one here that is actually going to be the core entity so we have message and note and the message note is uh has four three properties the first one is the id of obviously so this is the one the second one is the title of the note which is two and the third one is a description which is three you can define those in any order you'd like like anything like this because it actually doesn't matter it's going to see your uh index and the order of the factors doesn't actually alter anything uh so jarp is actually brutal buff literally doesn't care at all for what you're gonna do with your names and fields and whatever since uh you put all the indexes correctly it's going to work okay uh so oh and there is something very nice too uh this is why indexes are very important this is why indexes are very dumb actually because as you can see all my notes are uh strings and if i send a title first instead of id it's going to set the title on my note because title is also a string so purple buff doesn't have any way to tell which one is a title which one is an id which one is a description so basically protobuf and jrpc does not uh they don't uh validate anything okay it's up to you to validate those data okay uh so the last message is the note list which we're going to return from the yeah the list uh service so we tell it it's a repeated which means it's a it's an array uh note so the type and the name note list so sorry notes so we're saying that we're going to return a notes uh property which is from type note and it's a list so it's a list of notes okay so basically this is everything that we want to do uh oh sorry it's returns here it's missing an s i was wondering why it wasn't uh linking this for me uh okay so this is our contract to understand the importance of this uh you can have like kubernetes has several of those files uh in the github repo if you go over to github.com kubernetes.com you'll find tons of grpc proto files like these and you can download these files and you have the entire uh kubernetes api in your hand and it's actually working and you can do communication using only those files uh the way it's done uh it's called it it uses a compiler which called it's called protocol compiler uh you have to to install this in your machine uh it's a binary i mean i already have this on my machine but you can install it if you want to test it the repository already has the uh npm script for compiling so you don't need to actually understand how compiling works but what you need to do is what you need to understand is that protoc takes a input which is a profile and outputs something in whatever language you want so i think protoc supports javascript go c c plus plus java ruby uh c sharp and there is something else i miss i think it's python uh so you can output uh pre-generated code for all these languages in a single command this is what i'm gonna do here so i'm going to go on quick quick question uh so uh um you're using proto to generate like proxies i guess for these messages and services right and you you mentioned you mentioned c sharp for net but that's net core right you can you do that for net framework let me check let me take a look because i don't even know if i'm actually guessing c sharp i know it is uh so if it's c-sharp it's going to be net and i i know core but i've got a lot of legacy code.net framework and yeah i think it's for framework because protoc is pretty old uh let me get this brutal c because that's that's a challenge i've been running into is trying to figure out how to how to use support grpc in legacy.net framework servers and clients let me take a look here all right so the search is not that good okay let's let's use the plano i'm going to search google in another search but all right this is just to save time uh cruelty and uh they have a uh what is i i didn't want to derail you i i if you had the answer off the top of your head that would be great yeah from the top of my head without looking into the documentation i think uh it's used to used in c-sharp framework the dot-net framework i think it's i think it's only.net core i don't think it supports.net framework um yeah this is my doubt as well because i uh since pretty buff uh product proceeds uh very old it's not that new uh and it's as you can see it's pretty shady where you can find it but uh i i don't know if it's a very very uh documented on this but if it is anywhere i think it is on the proceed here rodol c perdue see [Music] oh here uh so this is the definition so c sharp yeah is it only states c sharp uh but i don't know if it's for core or not so as you can see there's a lot of other languages so php dart go rugby javascript c sharp objective c python java c plus plus and uh simple plus because it's already built on top of sql sql so it's built on net uh 4.5 so it supports uh 4.5 and core so it's both of them so this is this is nice okay so it supports uh both of them i'm gonna send you the link for this repo as so you can take a closer look at it i'm not looking into the chat okay oh yeah can save me a while ago but i wasn't looking at the shop thanks can uh but getting back to the well to the code uh in it it can generate files from all these languages so it's a lot of languages uh and it generates like a huge file that's going to be uh pretty practically uh unreadable basically but uh you don't need to read it you just need to use it so the syntax uh always gets me because it's very complicated to just put on the the command i hope you're seeing this because i can't uh enlarge this text for some reason uh but i am doing like this i'm going to say to protobuf that i need a js output uh with uh some import styles so basically what i want uh is to uh generate a common js style uh output with a binary from proto disk so basically it's going to generate a file on proto slash dist okay from all the proto files inside the pro gear so i don't need this because i already have all the things and it has like allow perl 3 optional and so on so i'm not going to use all these flags i'm going to use the simplified version which is basically js out and i'm going to shamely copy from our compile script over here so common js binary and just here and dot because i want to i want the output file to be on the same folder i am proto slash notes dot product all right so basically this is what i'm going to do i'm going to run oh i don't need proto because i'm on the right directory here so as you can see there is a new file here notes underline pb dot js and as you can see it's pretty unreadable and it has a huge uh disclaimer here so you don't have to edit this file ever okay so every time you do any changes every time you change your proto file here you just need to run this command again and it's going to regenerate the file with the following extensions and everything else so let's use the protobuf extension without grpc so i can show you how this thing actually works i'm going to create a new thing called node.js and as you can see here on the right side of the left side of my screen uh there is an outline and in the outline i can check all the methods that we have so basically if you're going to just uh fold everything and it's going to be like a lot of pretty product stuff and uh when you get to this you will see that we have the void object that we generated on the protofile we also have the note id object we generate on the profile and we also have the note object we generated on the profile over here so if we expand this we see that we have a more java like expression like set id get id get title set title get description and set description getters and setters and we have a serialized binary thing and it may not seem like it but those three uh methods all belong to this profile which to this proto uh property which belongs to the note okay so uh we can deserialize a binary and we can serialize something into binary of format so why is this useful we can use protobuf only decides from jrpc jrpc something that uses protobuf but protobuf can be used along standalone and the biggest use case it's uh to serialize stuff from message queues so uh as you can see and i think you have already passed some these problems as well uh when you have like a webcam key or kafka cluster or whatever uh you are you are actively sending messages that have no schema so you don't have any schema to send the message and you can't actually take the schema off because it's not typed so uh if you are using type languages that's okay but if you're not using typed languages like python or javascript you are going to have a lot of problems when someone sends a message that's out of your desired format so what you can do is that you can use protobuf to standardize the communication between every service you have a singular and a public profile distribution every repository that is going to use this this portal files clone those protofiles they generate the the compiled protofires for their extensions and you can use this extension to generate serialized objects which you're going to put in the queue so basically what you're going to do uh is serialize things into binary format put them on a rapturemq or kafka queue and then you're going to uh deserialize this on the other side let's have a quick example over here so uh basically what i'm going to do i'm going to import this as brutal and it's going to be required from notespb okay so basically this is everything i need to do uh i'm going to create a new note so i'm going to create like const note as well it's uh the same as new proto dot note okay so you can actually pass on the parameters like id title and description over here but i don't like do it because it's not very explicit what we're gonna do so i'm gonna set the id so no dot set i d and we set the id as like i don't know something uh then node.title set it to title and it's going to be simple things uh note dot set description and this is going to be desk okay so basically this is it uh and when i get to note and log it right so let me go to the larger terminal and let's go to proto and let's execute the uh node node.js as you can see this is not a array this is not anything you can actually you can't say anything you can only see the array of parameters i have because it's indexed so i can actually create a serialized note so serialized note and say that this is the serialized serial binary so this is the serialized version of my note which is a typed array filled with beautiful bytes right so this is the same thing as our note so we can actually get get stuff if we do like no dot no dot right my cursor is here so no dot get id if i run this i'm gonna get like something here literally i'm getting something so node node.js as you can see note something okay so i can get it's an object it's something that it's uh you can you can interact with so imagine this is going to be sent out to a queue like repton key or kafka or whatever on the other side this uh the other application that's going to receive this q message is going to unserialize it it's going to deserialize this object and serialize it it's going to be a turtle.note.d siri d serialize binary from the serialized note okay and now we can log it as a simple note and then get id so hopefully if everything works as expected we should have this okay so basically you have the same object into two parts and you don't need to worry about what uh the communication channel is because protobuf is basically binary so it's simple it's easy to use and you can use it in other projects other than grpc itself okay so uh this was not nothing i wasn't expecting to do this uh demo on protobuf itself but something that's very interesting and something that i never had when i was learning grpc or protobuf because no one actually talks about protobuf in standalone mode just mixed with grpc and other technologies because it's the way it's meant to be used but it's interesting for you to understand how protobuf can be used alone without the need of jrpc or anything else you can serialize and deserialize stuff using javascript and protobuf you can communicate of everything even with rest using protobuf okay so uh you can use this technology you can leverage parts of it not only the grpc stuff so let's jump on to the real part so basically i'm gonna write a server in a client the idea is that we're going to have a server and we're going to communicate the client with the server and we're going to create some notes list some notes and get the notes okay so uh the first part we're going to have to do the server is actually a pretty simple thing uh we are going to import the grpc stuff so grp grpc so it's require prpc and you're going to load the definition okay the definition i'm going to load here is uh deprecated deprecated in the current version you can use another version of the same load method but it's more complex to explain but uh in the official documentation you'll find the new version so use this new version okay so i'm gonna call it notes definition which is basically the automatic process of grpc uh uh serializing and combining your file okay so uh what we have like uh it's grp g rpc uh dot load so i'm gonna pass on my uh file name so i'm gonna go on and give the the notes dot proto uh f so i'm gonna require the path uh lib here i'm gonna resolve this to a deer name so we can avoid problems with uh deers and i'm gonna get into proto and no stuff so this is i'm gonna that's what i'm gonna do and basically it's going to compile the protofile and give me a definition a service definition that grpc can read okay it's basically a huge json file with a lot of things so i'm gonna build up my database so we have notes and my database is like id uh one title one description one okay so this uh is the first note and surprisingly this is the second note uh now what we're going to do is to implement the methods that we are going to call remember that we created two methods we created a list and we created find so if we want to add a new method here uh we can add like add method we can create this method okay uh this is uh from person person so this differs from person to person the the way you do this uh because grpc is based on callbacks uh i know it's pretty odd but you can create uh these promise like things but you have to call the the grpc callback okay so the idea here is that i build the functions on their separate scope i don't build them on the server implementation you're gonna see how this looks like so what i want to do i have to create a function which has the exact same name as the uh rpc name so it's list with a capital l so it's function list with the capital l it always receives the first parameter which is the request and the second parameter which is the callback which i'm going to call just cd okay dcb is the most important part of jrpc okay so to list everything i want i just want to return the whole list i just want to return the whole note state so i'm going to return cb the first parameter is always the error so it's no i don't have any error uh and the second parameter is the uh return the message we're gonna return okay so now the other part is function uh find it's going to take a request and inside this request we have a parameter we have all the parameters we send like so if we want to send we have if we want to create like a message like note and we have a rpc ad which receives a note note and returns the same note that was inserted i would have all those three properties inside the request so it would be like request dot id request dot title and so on okay i'm gonna just leave it over here but i'm going to comment it out because i don't want to create this right now so fine is going to receive an object which is the request and this request has another object we need which is the id so i'm just extracting them out i could just leave it like request and use it less as requests dot id okay and the callback okay so i could to so so it can be clear i could do this okay so const uh id equals request dot id and i could just remove this and use request okay i can do the same thing but this is exactly like this all right so uh basically what we're gonna do is that we're gonna return return uh the result of the array find from uh id so basically uh the first callback we're gonna call so it's no i don't have any errors and the object we are going to send back its nodes.find uh and we're going to send the predicate so note and no dot id equals id okay basically uh this is what we're going to send back to the user and now we create the server server equals new grpc grpc dot server there is new here and we well since i am creating the new server i have to add this definition to the server okay i loaded the definition but i need to add it to the server so the server can understand what is going to serve so server dot add service which is here and the service is notes definition dot uh note service dot service okay and this is the implementation this is why i said uh i like to do it this way because i think it's more organized but uh it's basically an object and in every key you have the services so i could do it like this i could do it like this list right so list uh underscore cb and implement the whole thing over here okay i could do it i could do it but uh it would be like a huge uh object and i don't like it so it's pretty much it's really easy to do just like this list and find right so this is what we're gonna do so this is the implementation of that service okay and now we create since uh grpc uses uh http 2 so it needs a ssl certificate we need to create this as insecure because we're going to use it locally so i'm going to create insecure but you have to create it using your own tls secret certificate if you want so i'm going to create server.bind and then bind it to all the interfaces at a random part let's say like 9756 and i'm going to use the following certificate uh server credentials dot create and secure okay so this is what this is where you put your certificate okay and in the end of this server.start right so this is everything we need to do for our uh api to work so if you have more services you can implement them here you can implement them on other files you can if you're using typescript it's very nice to use a barrel so you have like a file with you which is like index.ts and you just import this file using require inside this implementation implementation parameter and it works like the charm uh and we can actually run it so if you do like node search server it should work all right it should work it's working as you can see there's a deprecation warning because i'm loading things without using the proto-loader module which is now the new standard you have to download the proto-loader module but it's very documented it's very well documented on the documentation and now we're jumping on to the client so this is where things get interesting for real uh i think i'm talking a bit too fast is there any doubts any things that you want to ask everything so we can have a conversation without me just mumbling around does anyone have any questions all right then or are either is everyone understanding everything or no one is understanding nothing so basically this is the idea uh thanks robbie he is leaving to make another meeting i hope to see you all again uh so let's continue to the client part so the client part we're gonna do is that we're gonna call them both these both two methods and this is how uh grpc actually differs from rest and this is why a lot of people don't like it and this is why a lot of people actually love it but uh the idea is that we're not going to do a single network home we're not going to code a single network it's all going to be uh automatic it's magical but it's basically a think of it as a framework okay a and you call it and it's going to do all the networking from part for you okay so we're going to import it just like we did on the other part so i'm just going to copy this first two lines because it's the exact same thing so uh we have the notes definition and you have the uh grpc stuff when you are building like uh kubernetes native applications or uh something that uses jrpc like docker uses jrpc or container to use jrpc i have a very good art on my blog that i use container the sjrpc to communicate stuff from go go link and all of them needs to import the purple files these two lines are the most important lines in every jrpc server or client because it needs to be there it cannot be omitted okay so basically the only thing different is that we're going to we're going to instance a new client so the client is basically a new notes definition dot note service so uh here we used notesdefinition.nodeservice.service so this is the service that the server can understand and uh this is actually the note service itself it takes a parameter which is the the server address so we're listening on localhost i forgot the part it's nine seven five six so nine seven five six and again you need cls certificates so grpc.grpc.credentials.create insecure okay so it needs both uh certificates on both sides okay uh now the thing we're gonna do is that we're already we're ready to call it so we're ready to do our uh work and call this uh remotely so we're gonna do just like client dot uh list and this is the first parameter we don't have any parameter and it takes a callback which is error and the note list okay so now we do like if our throw error nothing so far and then console.log notes okay uh we can do the same thing on the client.find so the first thing we have to do is to pass on the note we have to we want to to uh find right so the the note to define uh takes a note id which is basically a string id so we just go on with a id so let's go like what one and it takes the callback error and the note so if error error and if not note dot id because note always exists uh return a new console.log note uh let's put this one no not fun and if we find something we return okay uh this can be promisified this is the uh most annoying thing in grpc all takes callbacks and stuff but this can be promisified you can just promise to find stuff like you do with opera with all callbacks you can uh click create and promisify all those uh methods so it's kind of easy to do it and this is everything we need to do all right so if i run node search slash server and open another one to use the client uh let's see if my computer doesn't crash it's going to work hopefully it's going to work enjoy the time to you know make some questions talk whatever so node serves client dot js and when i run this uh it should take a while because it needs to create certificates and so on so the bootstrap is uh a bit slower than rest because it's not that stateless but uh it worked as predicted so as you can see i called uh find uh lists in last and it's uh showing me first so because fine it's a bit faster than the the list one so as you can see i didn't do a single network call i didn't do any uh rest calls i didn't have to ask the backhand guy or the front-end guy which route i need to take which parameters i need to take i didn't need to go on and search for thousands of years of documentations lost in some company archive or something with well i never worked at a company that had a good api documentation so basically well they so if the if the api documentation existed it would be like a win for us but uh most of the times uh api are not documented and this in this way we can not document something we have to have the proto files so jrpc is self-documented in a lot of ports okay because you need to have the protofiles and with all the protofiles it doesn't work so all apis are self-documented and the documentation generates the apis so this is basically the good stuff uh the bad stuff which is uh it uses callbacks uh but it can be very easily avoided you can do like any promising file function on client.list to take uh a sync function and return promise this is no problem uh the only problem is that you're going to create like a middleware service among uh your grpc and your actual views you have to like to say it uh but it's pretty simple and the other annoying stuff from grpc is that it has a outstanding documentation for gold java and c plus but for all other languages they are very poorly documented javascript is very okay with the documentation and languages and tooling but it's kind of weird because there is a lot of tooling for the same stuff so uh there is actually a very good article i don't know if i can find it again but it's called the weird world of grpc tooling on node.js it's a three-part article from someone that i don't remember the name uh which is on medium and he explains all the weird stuff that goes on grpc tooling for node because uh jpc wasn't actually intended to be working with dynamic languages like this this is despite having the official uh you know libs but uh people started saying like well mandy sucks so i'm gonna build my own and like everything we do on note there's like 5 000 packages to do the same stuff so uh what this is the the weird part but uh stick to the official ones and you're gonna be okay okay so uh the other one is the protobuf js which is a very good lib that is developed by community and supports typescript so you can use typescript with grpc is a bit more complex but it can be used it generates when you build the protoc thing it generates a notes uh underscore pb.js and a dts definition for it so you have type intellisense you have everything on your servers uh to do it so it's pretty nice and it kind of gives the sensation to be programming on a typed language or in doing things like object range and programming and so on so it's pretty nice and this is actually everything i had to show you it's pretty simple i told you about it but uh the idea was to be simple uh the other stuff i'm gonna do is i'm going to implement the add function here so as you can see we're gonna add a new uh note so function and it's going to take the request i'm not going to use all this uh thing because uh it's a lot of properties so cb so i'm going to just get uh the note so const id tile and desk oh description equals quest so i'm going to extract those i'm going to have like notes dot plush um a new note which is id title and description like this and i am going to return i i'm just going to get the cb and i am going to get the i'm just going to return the the object itself okay so uh id file let's see if this works this is completely something that i haven't tested yet uh all right so client dot uh at yeah like that and we are going to send a request to it so we're going to send the id so it's going to be three we're going to send a title i don't know if it works like this but let's say title three and description right function error note same stuff without this let's see if it works right so spinning up server again all right so the server is oh sorry oh this is something uh it expects all the method handlers to be added so i created the add method here but i didn't add to the implementation here so i need to add it it's going to complain if i don't so we just do like ctrl c and spin it up again so all right it's being used and then we do the client stuff and hopefully it should work let's see yeah uh no we didn't because because it didn't pass on the parameters on the right stuff right so here i think this is because i use an object instead of an array so let's take it off and take it off here all right so let's run it over again all right so it doesn't work like this it's clearly as i said uh something's gonna go wrong and it's gonna probably be on uh a live coding session okay so this is something very nice uh a string object know what we expected probably oh one moment help me out over here okay okay no argument mismatch is it defined in your service you know like when you are setting that up at the beginning um you know like say he adds a note takes id and title and description and [Music] it is defined here as well so a detailed description it pushes everything from the request all right it shouldn't be that hard why is not working let me take a look at it this is the the the other weird stuff when using jrpc with node.js without typescript uh you don't have any way to tell how these services are going to be called because you can't uh actually understand how they're going to be called over here so there's no documentation something that you're not going to be using for this so you can have a object here because uh it has an arguments match uh and you cannot have a array because it's actually an argument this match the weird stuff is that it worked on other versions i probably updated to this newest version and it doesn't work anymore so uh i'm gonna be uh correcting this and i'm gonna sure put it on the uh repository so you can have everything set up over there right um yeah no problem um you know and typically like when i look at these kind of like apis and stuff like that um [Music] you know like the way this kind of flows to me at least like when i take a look at it you know it's like you were saying like it's it's simple similar to like stuff that you saw in java a long time ago we have like an interface and then you define the implementation then um you know basically like here i think the weird part to me is that like the parameters that you're throwing in um don't have any kind of like schema shape or anything like that so i mean when you add like like typically in like a programming language like java you actually just add the object and then it maps over to the to the um back end and the back end like and it says okay i got this new file object or i got this new note object and then it saves it to the server whereas here it's it's kind of like you're passing in a string or passing in an array um passing in an object passing in multiple different types of arguments um and it's saying argument mismatch so to me like maybe it's just because of that like the types are not matching is that correct yeah as you can see uh it actually worked so we have the the first time i i did it it actually worked so we have the request so we have id title uh and description for some reason uh it's not adding it up here uh so i don't know why it it's let's let's log it uh it shouldn't be i understand what you say uh it it's going to be pretty weird if i needed just to add a array of objects right so uh this is why i thought it was like very difficult to understand but uh it wasn't like this so this is what i'm thinking uh this is something i think i'm doing wrong probably so let's call it over again and see if it happens because i have the request stuff over here sure so it's undefined because of what i do have a request id okay so a request uh id over here and we have server generic oh okay i did something bad uh the request but uh i forgot about it so this is important uh the request has a request object uh so um you can call it like a lot of people call this call because it's a server you never recall so you have called a request okay request request here uh but uh most of the time what you're gonna do is just use the request object you don't need to use all the other objects so we're just gonna put here like request and i think this is where the issue was lying on because i forgot this uh inner request object so we are going to have a note 3 so now it works yeah okay so uh it's good that these things happen because uh this is how grpc is actually used among like a lot of services a very new technology not new but uh it's not being as much used as rest so it has little to [Music] documentation uh in a lot of languages it has a complete documentation for c plus plus in gold because it was built by google but uh it doesn't have grpc is built on sql plus operator is built on simple support as well but it doesn't have a very good documentation on java and rjs or whatever so sometimes get stuck in some of those things but the cool stuff is that you don't need to have you don't have much to be stuck on because it doesn't do much more than this so you don't have like a lot of things and if you if you have like any doubts on the protobuf syntax and stuff now it's another matter so it's on protobuf and not grpc itself so it's way easier to find the syntax and the the whole problem above documentation and uh where to look up to excellent um anyone have any questions so far and i you know also along with that too um you know i know sometimes like the stuff that we cover you know in this group can be kind of high level or you know very um technically advanced um but we just kind of like you know try to cover different topics you know sometimes um even if it's complicated or quite complex or whatever you know i think it's a chance to kind of preview you know something new um maybe the first time you see it it might be difficult but then later on like as you get used to it it starts making more sense and you're like oh yeah you know what it's not too bad i kind of get the flow um so i mean just like you know for the people watching i mean and you know even if you're watching this later on youtube or something you know feel free to you know just either now like kind of say what you feel or maybe leave comments you can use a chat as well um you know maybe not all of you may be using jrpc like for your job but it's still like a good thing to kind of know about um if you're you know interested in web development and you want to uh experiment around i think part of the thing that we as web developers do is uh you know we build for our job but then we also tinker around a lot too um just to understand like uh new dynamics and things like that so that if we have a chance to kind of work on something new at the job or propose something new we can talk about like you know the new findings that we have um as well as compared to maybe like older stuff that we were doing and maybe there's a better way of doing it or maybe you know like you tinker around with something and you find like maybe it's not for you at all you know so you could come to that conclusion as well so yeah and the i think the good stuff on grpc is something that some people always ask me about it's that is grpc more uh performatic is what about the performance stuff now that we have http when jrpc was invented it uses you use speedy spdy and it wasn't actually the first implementation of http 2 and yeah it was i'm like several times faster than uh http 1.1 uh and now this speed is not the the thing that makes grpc stands up because uh jrpc is based on http 2 so any rest request that you do on http 2 should be almost as fast as your pc jpc is slightly faster uh but it's still way faster than uh http 1.1 with no keeper live connections uh but uh the good part of this is that grpc actually is isometric it kind of works everywhere at the same time with the same stuff so there is a repository called jrpc web which is great because uh you can use the same thing i use here on any web browser and the things you can do honest um it's very important because like container d container d is basically a daemon with a grpc expo an exposed grpc interface so if you access the socket from continuity if you have like a way to access the socket from container d uh from the web like with an envoy proxy or whatever you can control uh containers using only the front hand you don't need to use the back end don't need any back end to use it you can create a full application to manage containers just like kubernetes does or just like docker does uh straight from your site without doing anything else so uh this is the power that grpc actually provides you the the weight so you can uh i'm gonna write about this on my blog but there's another article uh about this uh grpc and continuity stuff that is awesome uh unfortunately my blog is in portuguese so if there's any portuguese speakers over there my blog is blog.centros.com but the idea of grpc is to unify things is to make the development standardized so you have the contracts on both sides and you have the same implementation of servers on both sides so if you do have a protofile you do have the implementation of the server you don't need to ask anyone else so this is how uh grpc and in my opinion is better than rest in most cases but i recognize that uh grpc might not be the best solution for uh whatever you are going through right now in your company or on your enterprise application because uh it cannot be it's not very easy to understand or to learn and doesn't have like a lot of docs uh and you're gonna have like two it's kind of the the no something outside of the comfort zone of everyone that's using rest right now but uh it's worth it i promise it's kind of like it's worth the the weight is worth the job uh and it makes like a very it's makes very easy to develop front-end and back-end application even distributed applications uh if you want to distribute it to like a huge team or a huge company you can just distribute the proto files you don't need to document anything it's pretty pretty nice um i was going to ask you know like what are some companies using grpc including the company you're at like what are some yeah or examples you could show us uh docker users a lot uh google uses it a lot actually google invented it um let me see other one that is i think volunteer is actually using too facebook uses it as well uh mostly for internal communication like internal service to service communication uh inside the kubernetes cluster or something like this netflix definitely uses it um basically every company that deals with containers in uh the flash like deal with creating containers and generating and managing containers themselves uh they use grpc because it's the easiest way to uh integrate different systems so microsoft uses it like a lot i'm working on a cada add-on for cncf right now that we are going to you we are actually using the external scalar interface that cada provides as a grpc interface uh uber uses it a lot too because um if i am not wrong i think uber is trying to solve one of the other problems that jrpc has actually probably has uh the libs so you can import protobufs from proto files from other profiles like we do on node modules so you can import uh like a definition google has a lot of definitions you can download these definitions and import those definitions much like we do on go or java or whatever uh the problem is that they don't have any package manager for this so you have to download all the files manually or using make files or using i don't know bash scripts which is weird and bad because if someone updates the file you don't you won't ever know that the update the file was updated because there is no version and there's the protobuf was not built to be version or to be a module so uh uber is trying to solve this problem uh by creating something like an npm for uh brittlebots but due to the nature of brutal buff it's already spread out in like several tons of repositories so you don't you can't use a package.json file because no one has it so you have to you know rely on commit hashes or uh checksums and so on and this is very complicated so uh this is something that i uh well if you're a different the node.js developer and you're looking for possibilities to expand the uh ecosystem and create something very nice that uh you know there is a hole in the grpc community that lacks uh tooling for node.js and javascript in general so this is a very good area to be tackling right now if you're like a lib developer or an open service developer awesome awesome thank you for that i mean like yeah exactly like what you said right there like if you're looking for work trying to learn something uh this is probably exactly what you should be doing right now so um if you like jrpc and you're like trying to create or you want to have an open source repository that's people like sorry if one i have a public repository that other people uses grpc is something that's needing help so graphql is another one so this is something too for sure uh any other questions like from the uh audience let's see let me check chat too no one kevin what about you you're awfully quiet got a thank you from uh dancing we got a lot of thank yous actually today so that's good i think uh probably also glad you all liked it yeah good presentation definitely i liked uh the simplicity of it like you know explaining about like the beginning part creating the um the service then you know building out like implementation yeah so it was uh the original presentation i did for this uh was at a brazilian event i'm brazil i'm in sao paulo so uh i did this presentation for a brazilian event uh it was like in very the beginnings of jrpc here in brazil not the beginning of grpc itself but uh it was the beginning of people knowing what grpc is in brazil and uh it was designed for portuguese audience so i you know converted it but the idea was to have a quick and simple presentation to be given in like half an hour so you could understand what grpc is and how it works and there is a slide deck attached to it on the github repo if you want to take a look but uh it's mostly everything i showed up over here awesome uh i guess like outside of code you know or inside of code still um what all do you do you know like um like do you go to any other meetups are you part of any other groups like how do you stay like on top of stuff you know outside of microsoft and all that well uh right now it's kind of easier to get on top of stuff because my work actually is to get on top of stuff so i go actively looking for some things but before it uh i also organize a few meet-ups here in brazil one of them is node node vr which is the brazilian node.js community and the javascript sao paulo and typescript brazil it's a lot of communities vue.js sao paulo and i also participate on uh docker sao paulo 2 and the vox brazil so but as more as a speaker or something but those first three are the ones that i uh actively organized before the pandemic we had monthly meet ups but now uh people are getting kind of tired of online events over here so we're waiting until everything goes back to normal so we can do within prayers and events again and continue to grow so i i appreciate the work you do with uh the javascript delay and have buddy i know it's like very hard uh work to do and something that you have to you know give yourself a lot i've been doing this organization like for four years as well and it's something that takes a lot of time and needs to be very very uh passion for what you do so congratulations for the the meet up congratulations for the community it's awesome yeah thank you thank you for that and um i hope uh you know i hope we can kind of visit your group um without too much of a barrier i mean you know maybe we can try to find somebody who knows a little bit of portuguese and can help out we can have a translator we can have like uh what we did uh on the first uh typescript brazil meetup we we brought uh one of that one of the typescript developers was in brazil for a conference and uh he agreed to join us so it was like the best of everything we had and he joined us and uh i was like the translator so he was talking in english i was trying to translate in portuguese in like real time was working but uh it was definitely an experience so uh it it would be very nice to have like a multiple meet up like javascript's apollo and javascript delay would be awesome i would love to do it yeah you can talk about that for sure like outside of this yeah definitely uh and i did an extra question right how do i get up on top of those stuff uh basically i read a lot i uh i am i joined like a lot of newsletters on i don't like javascript weekly uh i have i'm a member of some brazilian newsletters as well there is a podcast which is called a morning loop here in brazil uh and that it it's like a newspaper from tech news i uh have i contribute to them with a small amount and they send me every week and mail with all the news in a week and i try to read every week and something that i do that helps me a lot as well is that every month in my blog i publish a monthly newsletter on a news being everything that happened on my blog and other news so this helps me get on top of stuff and i'm always actually reading about javascript and despite i'm not actively working with javascript anymore and using containers but containers use javascript so i'm kind of working uh by reference and by reflection on this but uh uh i still like to know about javascript a lot so i know a lot of people who works from javascript i know a lot of people who works with javascript and they send me some news and some things but mostly the the biggest uh source of news is like medium uh dev two and sometimes uh the javascript weekly reports from emails and stuff so i have like tens of thousands of reading lists to read every day and this can take some time walk yeah it takes a while yeah i mean even same with me i mean there's a lot of information sometimes to go through and i think like it's it's you know sometimes like i think you kind of have to take a step back um you know with all the new stuff coming out um i feel like sometimes like i see stuff come out and i feel like you know the thing might be new and cool but i feel like sometimes like we're reinventing the wheel a little bit rather than kind of moving forward it's just like hey it's the new way of doing the same thing we've been doing you know like 10 other times and so you get where it's like okay maybe i need to kind of like step back and like just think like okay do i really need to you know do this or can i kind of like put on the back burner and then can i proceed with what i'm still doing because i feel like you know the way i'm going like like the directions that i'm going like for example react right like there's a lot of new stuff coming out and react but you know like the old stuff still kind of works just as good and sometimes like you just want to get like you know a consistent project finished rather than like re-break it over and over and over again um and then just kind of like make everybody else on your team sort of unhappy so you kind of have to find that balance i think yeah i had the same problem with react as well i worked on a company where we built a project with react from the ground up i was 2017 about and now the company still exists it grew like a lot but i'm not in the company anymore and the the other developers had to support the same project the same developers that were with me they are still there so it's not a new team but the problem is that uh react is a lot of breaking changes so every time you update to a new version you have to spend like three days just correcting breaking changes and they simply couldn't you know manage to do it so they took the easy way out so they don't update anymore so basically the idea is that uh react is always fixed on the same version and uh someday they're gonna update everything to a new version using hooks and so on but this is not gonna be today and probably not in the next few years but what you said is correct so we're kind of reinventing we i think we're moving forward faster than we can actually walk so yeah some things are getting out of hand like front end frame i think the today backhand had this a few years ago it exploded everything working with a lot of stuff like node.js getting uh you know being one of the top frameworks of runtimes in the market javascript on the back and then we got go rust we have a lot of things but back end is kind of quiet now there's nothing too fancy or too breaking you know getting being launched but front end i think it's on fire so front end is getting a lot of new things every day and we have like tons of thousands of uh uh front-end frameworks so we have view but view is too slow so now we have react but react is lower than view now we have angular and now we have this valve because this valve is faster than everyone and now we have preact inferno we have a lot of things just because we saved like three microseconds on every request so uh sometimes we're getting very you know theorists on what we think what we do and we are forgetting about the i think the real business of this thing of the creating software for people instead of creating tools for software developers uh so we have like the there is uh the this is uh to to explain this a lot we have the war for the uh package manager war like we have npm which worked just fine but it was too slow so facebook created yarn which was exactly like npm but it was faster so okay it's a good way to use it i use the iron one a lot now er2 completely breaks down everything we know and it's completely unusable but if you're using react you need to use yarn and there's those things because it's facebook and then there is pmpm because who needs uh to have less problems with no modules when you can have more problems with signaling node modules so uh now you have like three things that does exactly the same stuff and are based on the same stuff they create exactly the same stuff and everyone uses uh different ones because they want but uh one is not compatible with another and i don't know i i i feel like npm is the one that's trying to it's they're trying their best to be compatible with uh yarn files and pnp and files but uh it simply doesn't add up like we have yarn lock pmp unlock shrink wrapped i yamo and a lot of things like this so uh i understand i agree with this we are creating a lot of tools instead of softwares we have the gem stack explosion so this is the thing uh nathan i think brought a very interesting topic so instead of using just a streamlined developing uh experience we have like a ton of things to do with yeah like if if you use like a lot of uh cms's like drupal and wordpress and back then back in the day if you if you're going to be a blog you just start wordpress or drupal or whatever now if i ask someone a simple question i'm i asked myself this question when i built my blog what am i going to use to build my blog so there is yes b there is hugo there is axle there is jekyll i can use view press i can use react i can use view i can use angular i can use velt and i can use html i can build my own cms i can use a scrappy i can use uh next i can use for cell i can there's a ton of things yeah and it's like yeah why it's i'm gonna be on my blog okay so i don't i don't i don't want to use wordpress because i had like a lot of problems with wordpress and i had to choose between a ton of things and when i chose hugo because whatever someone i got hates because i chose hugo over jekyll or over things that has said dude why uh it's just a blog so why i think we're getting very very uh technology addicts yeah we are defending technology instead of just using it as a tool exactly um all right so good good talk so far um i just want to like give one more chance for anyone to ask me questions and if not then um i guess we can wrap it up and um you know we can always have you not come back uh lucas i truly enjoyed this talk and i actually want to talk to you more offline um but yeah anybody else with any questions go on you're asking too much like i said they're getting a lot of thank yous i think it's cool that um everybody's like you know on the same page and uh yeah like i said too i think the presentation was great i think it was pretty straightforward um so yeah you know anything that maybe in the future you want to like continue to keep us in the loop on update um you know i'll reach out to you myself uh and then we can talk about like future stuff um but um yeah you know i think everybody else too like if you have questions i see one from nathan um you know like with grpc i definitely recommend you to check it out you know just do like simple search um you get an idea of what this thing is then um yeah is it lucas's boy i hope we can do like a further session someday uh using jrpc with typescript because it's very nice i think it's the right way to use it so i'm gonna put something together and maybe we can we can do it again there is a lot of other things like uh i i think i said uh when we talked about on this about azure monitoring for containers and how we can monitor containers from prometheus and so on if you have container stuff using javascript creating prometheus explore exporters with javascript and no jazz it's very cool yeah that sounds actually really interesting i'd love to see the container slide for that especially uh jrpc onto each stream can we get a question here um let me see i i don't watch a lot of streams uh but there is a i don't know if no i don't think there is anyone actually using grpc itself uh but there is a guy that uh it works with me it's called aaron schlesinger uh i think his uh twitch handle is are less i think it's like this uh he daily he puts like daily lives with uh gold but uh he uses jrpc a lot and he fiddles around with kubernetes and kubernetes natives uh applications a lot so kubernetes and docker containers in general uses jrpc a lot so yeah so basically uh this uh maybe he is the best choice to to go to because he holds goal and in a daily basis like almost every day so it's he's very nice and he's worked with me on this project the things are happening yep that's him talking by the way i just had the channel yeah yeah um yeah really cool and uh actually that makes me think um you know this you're working with some pretty cool people i'm pretty sure so you know like do you um maybe if you could later like send me some links like for the links of people that you follow that'd be really great you know just because like i know the community loves to you know check out like some of these um you know cool bloggers cool twitchers um you know even youtubers yeah sure thing i can i can uh i i'm not uh i'm my obligation to just uh i i don't know many twitch streamers or youtubers because video is not my my thing so i like to read a lot so mostly i just yeah i most like to read i i am probably going to create a new youtube channel sometime this year but uh i need to you know there is a lot of things going on i need to just focus on some things yeah otherwise i'm not going to finish anything of course of course um all right yeah well thank you all for the feedback if you want to talk to me offline i'll just give you my linkedin so you all can follow me there um you know if you also want to take the time while i'm typing my own stuff out to share your linkedin on the chat feel free to do that and we can all connect afterwards there is a discord for this group um if you go to the website and i'll just put it here javascript la dotnet there's a link to our slack into our discord channel and i look at discord pretty much every day so um if you want to message me there you can do that a lot of times i will work with people um you know outside of you know these meet up video calls right like outside of zoom calls i'll work with people on open source stuff and projects and so one that i'm doing right now is uh something called team hack and some of you may have heard of it um but basically it's just a simple website you know that um uh talks about open source projects with the group and so um you know if you are interested in learning how to like code with other people and you know kind of build projects together you can visit this one this link teamhack.org uh we have a github we have uh twitch channel which we are experimenting around with um you know and so if you want to build things with us we're we're doing things like you know looking at how to build like you know our own uh business dashboards uh portfolios um like a chat app one that we were thinking of doing um you know for the next month was something kind of like a stock app where you know it kind of watches stocks and you know um you can create like a custom watch list and you like research like information something kind of cool gets you a chance to like play around with some javascript libraries as well as like do stuff on the back end so you know for some of you i think who are interested in um you know finding work or even like improving like your own position network uh doing open source coding is like a really good way to kind of you know upgrade your skills and then meet cool people and uh work professionally and i think you know if you if you do that for quite a while then it kind of becomes really easy you know to go to the pro level right maybe you could contribute to like a pro open source community like maybe docker or nodejs or something like that you know once you understand like the basics so i encourage everybody to link in with me to chat with me either on discord or slack and then you know um if you want to do some open source coding that's what i'm currently doing that's why i like you know i haven't really been doing the youtube thing just like lucas other than putting videos up recently but more of that should come with time like i want to give my attention first like i'm just getting better at coding um and then from there like start sharing some of that stuff that i do on video with youtube and twitch i hope that makes sense to everybody got one more question here from luca yeah how long have you been working with jpc uh yeah i i didn't mention that but uh i worked it for the past i worked with it uh actively in the past two years uh in the past year three years in the past year uh i worked it sporadically like sometimes and just stopping by uh and now today as of today i not actively working with it but i'm using it a lot on other projects but uh not actively doing grpc servers or jrpc clients and not the javascript as well all right everyone thank you so much for coming tonight uh great attendance and then great uh presentation all around uh lucas thanks for your time i appreciate it and um look forward to seeing you again soon same here thanks a lot for the opportunity love to see you all glad you all liked it if you want anything if you have any doubts please come to me on any of my channels anything i am glad to help everything and whatever if you have like doubts if you want to just talk about code or technology just pick me up i really love to do it i may take a while to answer but i will eventually all right thank you everyone see you all yeah bye bye have a good night bye

Channel: hackbuddy

Views: 528

Keywords: javascriptla, hackbuddy, hackbuddyorg, grpc, nodejs, grpc docker, grpc api, rest vs grpc, what is grpc

Id: 5RVRbDHFhNw

Length: 94min 11sec (5651 seconds)

Published: Tue Mar 02 2021