Connecting Python and Javascript with gRPC and Protocol Buffers

hey everybody today I want to talk about how we can connect Python and JavaScript together with a really cool tool called G RPC before I talk about that I want to give a quick overview on what an RPC is RPC stands for remote procedure call and simply put it's a it's calling a function that is not on your local machine and I'll get more into the details in a moment the G IPC library is a library by Google and it allows us to build our pcs with these it's used many different places in the industry which includes of course Google slack and Microsoft here's a conventional function call this is just some Python code so here we have a function called order icecream it takes two parameters scoops and flavor and it just returns a string and just below that function we can just see that we're calling the order ice cream function sending in two parameters the number two in strawberry now let's imagine the same example of ordering ice cream but in terms of our pcs say we have two computers that are separate from each other will name them the client and one the server the client wants ice cream but it's unable to make it on its own so has to go look somewhere else the server however can make ice cream in order to get this done the client needs to send an RPC to the server to order this ice cream it'll send over an important information like scoops and flavor and the server after receiving this request will respond with the string back to the client using our pcs now one question you might have is how do our PC send information you know between the client the server like we just saw so the answer is protocol buffers protocol buffers are developed by Google and simply put they're just special objects that are designed to be used across languages so Python can interpret them JavaScript can interpret them and we can use them say if these two computers are using different languages all right let's build this ice cream functionality from scratch I'm in my terminal right now and I'm using Linux keep in mind that you can use Windows or Mac OS to get this done so first thing I want to do is I'm gonna make the directory and I'm gonna call it ice cream okay you can call whatever you want I'm just gonna call this so I'm gonna go in there and in order to get this done since we're gonna be connecting Python and JavaScript together I'm gonna install some dependencies that I need in order for JavaScript to work for so this is assuming you already have node and NPM and stuff so we're gonna do npm install' gr pc we need google protobuf and we're also gonna need gr PC Pro to load it cool gonna hit enter and after some time this will finish installing now that that's finished installing I'm going to install one more tool from NPM this tool will allow us to auto generate helpful JavaScript code when we're creating our own protocol buffer file so just like before NPM install I'm gonna add the global flag so that it's installed all over my machine that's gonna be proto c and then jen gr pc now I've noticed that when installing this sometimes there are some weird errors that pop up like this well this was just because I didn't use sudo so let me just you know repeat that sometimes this install does fail and there's a way to get around it if in case that your installed did not work what you can try and do is you can before installing product jenji RPC you can do npm install we're the global flag again and then request so you would run this example before installing the actual tool and then after doing that one you would bring back the old command just like before I used sudo in my case I'm on Linux and then you would add one more flag which would be - - unsafe - burn then you hit enter and then I would that should resolve it if you have any questions just leave a comment below and I can help you out great now that JavaScript's all good to go let's install what we need for python so i'm on linux and i'm gonna do python 3 if you're on Windows or I think even Mac you can also just do Python so I'll just go ahead and ooh Python actually Python - ok install we're gonna do Google pasta so there's three things GIP cio and then we're also going to install GRP cio - tools give that a minute and that was pretty fast because I already had installed and now we can go ahead and start developing our solution so something this up in vs code it's my favorite one you can use whatever you want cool so this should be an empty directory which we opened up just earlier you know we just have node modules and packets lock in here make this a little bigger okay cool so I'm gonna make three directories one is gonna be server that's what I put on my server code like we saw in the diagram I'm gonna make another directory called client for obviously the client and I'm also going to create a proto filter and this is going to store any protocol buffer information so I'm gonna start by just hopping into Poteau I'm gonna make my first file and we'll just call the ice-cream dot proto and what this file is gonna serve as is this is going to be the information that the our pcs client and server are gonna send back to and from each other we're gonna define those special objects that I talked about earlier so the first thing that you doing at cono files you send x equals proto three letting you know the interpreter know that we're going to use this version of you know proto syntax and first we're going to define the object that the client is going to send over to the server right so we're going to define this object that's going to hold important information like scoops and flavor so in order to do that first type message and then we're going to give it a message so I'm going to call it ice cream request open brackets and then inside of this block here this is where we're going to define the different fields that go in this object you can Google protocol buffer types for more information on how to make these but you know first thing I'm going to do is have an int variable which is n 32 in this case I'm gonna call it scoops you know the number of scoops that somebody wants and you assign it a numerical value so first we're going give it one and you'll see why in a minute so next field I'm going to define is flavor and that's a string so straying or just caught a flavor and then we'll give that to now notice I'm as I mentioned before I'm assigning each field a numerical value and this just keeps incrementing for every single field that I have it was one two three four or five it's just so that each field in here has like a unique ID number so that on each end of whatever language is using the RPC you can better interpret these special objects awesome now that's done I'm going to define another protocol buffer the one that involves responding back to the client so we had ice cream request seems only fitting that we call it ice cream response so I just all right and I think we'll just have one field right that string so we'll call it will do string again and I'm just gonna call it message and will equal to one there you go now that we've finished defining our two protocol buffers let's go ahead and define the RPC itself do it up top above these two messages start by doing service and we'll call it ice cream why not open a block and inside we're gonna define an RPC I have this cool shortcut where fight enter it will show this but I'll type it out for you guys so you do RPC and then we'll call it the function name this is the function name goes here I should say I'll call it order ice cream now parentheses this is where we specify what it takes as a parameter so it's gonna take ice cream request and after it takes this as the parameter it's gonna return - Mallory quick so you can see the whole file so this is what it looks like I'm to find an RPC here I have two protos and this RPC called order ice cream takes an ice cream request and it returns an ice cream response all right now that our profile is done let's make use of this file in order to auto generate the Python and JavaScript code to make the our pcs work to get this done I'm gonna just first CD into the proto directory and then I'm gonna do this command so first Python you might need to do Python 3 if your computer doesn't by default do Python 3 when you type Python so I'm gonna do Python - M - your RPC keep in mind this is Python 3 like I just said - Lucy - and what this is doing right now is this is this long complicated command to make use of G RPC in order - like I said before auto generate the helpful Python code first so we have to you know a couple options here we have - I means this current directory Python and G RPC Python they're both equal to dot mean you know output these files to the current directory and then I'm just gonna supply the 1 proto which I hit tab to do answer and here you go right here you can see that the two files are generated so let's just grab them both right and let's drag them to our server cuz you know they're gonna help support our Python server very cool and like I said before these are auto-generated follows why don't we take a look inside of them and we can kind of take a look around this is all auto-generated like I said I didn't write this like a class called ice-cream service sir what else can we see yeah these long complicated come in so you go to the PB - it's also pretty long complicated so I want to go ahead and get started this let's create our ice-cream serve your phone so I'm gonna do ice cream so every can call whatever you want but might as well follow the convention right of what else is getting named here so cool now that done I'm going to import some helpful things that we've already installed earlier so import gr PC we're gonna import concurrent this is like thread pythons thread module oops okay those two imported I'm gonna import one more thing so from kind of current import futures so this is just for you know anything in the language is coming out in the future we want to handle it um import now let's import the two files that we generated so important PB to import ice cream PB 2 gr PC cool so that'll do it for imports pretty simple right now we're gonna define a class and this Python class that we're going to define is what's going to be the handler for our server so if you take a look over here the program that's going to go about saying I can make ice cream you know take requests and sender spawns this was going to be this class so I'm gonna call it class and we do ice cream servicer and if you're wondering why I called it that I just do that for now do you pass so this stays on the side the reason why I called it that is if you go inside of the G RPC generated file there's already a class in here called ice-cream servicer and there's not much in here write this function order ice cream has a air that gets you know raised when it's called and what we're going to do is we're going to implement this class so that G RPC knows where to go instead of this kind of like you know skeleton of one so what I'm gonna do is I'm going to take this classes to find here and have this class inherit it from there if that makes sense so we're gonna do ice cream PB to g RP c dot ice cream servicer so see how it's inheriting it i'm taking in throwing in these parentheses and after doing so we're only gonna define one function here which is gonna be order ice cream mm-hmm and since we're in a class first parameters always self then we're gonna take request that parameter represents the actual request from the client and then we're going to do texts cool and we're only really going to make use of requests because this is just a basic example right so I'm gonna print just say we got something so this is my class super simple right just three lines of code Wow nice you know class is only as good as using it is right so let's do a main function what I'm gonna do now is what you got first time you declare a variable called server and this is gonna be the running program that's gonna listen for requests and you know handle them appropriately so then dude-g RPC don't server and then I'm gonna make use of that futures module and then create threads so I'm gonna let's just do Mac's or it cuz it's 210 so what's going on here as I'm making a server right with G OPC server and then the constructor I'm sending in you know something from the concurrent module so just initially I was like 10 threads to handle it for the next 10 threads okay now I'm gonna do a led gonna LED what am i doing ice cream er PC we're going to add ice cream service and to server and keep in mind that I didn't actually write any of these functions this is from the auto generated files that we automatically created and um you know inside of this I'm going to send the cost that we just made so I'll send an ice cream sensor this is what G RPC knows the points of this class when we were getting requests and then also this server variable that we just made right here cool next I'm going to add a port you know so that we can receive traffic from the network so add let's make an insecure port right just for testing notice how I'm doing the brackets then a colon this is just so that we're listening on localhost we're gonna list listen on the port so 50 51 or 551 sorry so we're gonna listen on this port pretty high put your server started and then you know this initialize the server and then in ordered so the program doesn't exit will be server wait for termination that's it and then now let's call main right and then I'll just attest it google CD dot dot and we have a class we have a main that initializes the class and then I'm going to call on me a Python server and then I scream server not py let's go for it ready I think it worked but uh you know that's just because I hit ctrl C but I think we should add a one print statement on here just so you know it's letting us know saying hey it's working let's get it so I'm gonna do print I'll call it server started listening on or 551 cool save it sorry okay now let's go for it there you go so now our ice-cream server in Python is running the Python side of this solution is done and next thing is we're going to move over to the JavaScript side cool so now what I'm going to do is I'm going to stop the server with control C I'm going to clear the terminal and let's generate the JavaScript files using those gr PC tools that we installed earlier so to do it this time I'm not going to CD into the proto file but instead I'm just going to do pro doc gen to your PC that's s jeaious out and before we just you know say where we're going to output these files I'm going to stay import style we're gonna use common J s that's like the style that we're going to export the JavaScript in if you wondering where I got just commenced from I looked them up online you know just what's the best way to you know go about making these so binary colon then dot slash client those j/s binaries are going to go out to the client directory right because that's where JavaScript closing then gr PC outs we're going to say that's also going to the client I just make sure you spell that right and then specify the port okay kurodo and then we're gonna specify our profile so it's inside of proto and then ice cream that proto let's go boom and now if you take a look inside a client if you ran that command successfully you'll see that these got automatically generated and as it says don't edit it just generated code you didn't write it a lot of stuff here see how cool gr PC is it makes writing these our PC solutions so much easier alright so with that done let's create the JavaScript clients all following those conventions that we have where we're doing ice underscore clean and you guys underscore clean underscore clients J is cool and now let's just handle these imports that we need just like how we imported stuff at the top of our Python file so first one is gr PC so Const GRP C equals using require to require these modules gr PC you know if you don't use the vs code man like you know this stuff autocomplete so much for you and really just a breeze using this kind of stuff messages so this variable again it's going to be required and messages is going to be responsible for you know creating protocol buffers handling them and whatnot so then cons services this is going to be the variable that's going to keep track of all our RPC stuff so require we're gonna require the other auto-generated file which is icecream gr pcpd very nice alright we're done with our imports great now let's make a function main right and the first thing we're gonna do in main is we're going to create a client variable and what it's going to do is basically just bind this javascript client to the Python server on the same port so let's go ahead and do that so cost client equals new services right we just defined services up top ice-cream client open parenthesis and it's gonna take two parameters just so it's easy to read I'm gonna hit enter all these parameters and the first parameter as you can see string so the string is going to be the basically you know address and port of where we need to go we're gonna do localhost because this is just running within our computer's little internal network 5051 and the second parameter gonna throwin is g RPC credentials create insecure that might look familiar from what we did in python we also did create insecure for this server nice we're done with our client now let's create a proto variable so I'm just gonna call it ice cream request you call whatever you want but why not call it what it is so it's an ice cream request request equals new messages dot ice cream requests and capital I keep in mind there remember messages has to do with the proto so I here on this line is we're initializing a proto and we're assigning this new proto to this variable over here now there's some really cool built-in functions that come with proto's and javascript and why don't we go take a quick look so say i want to set the value of flavor or i want to set the value of scoops right we talked about those right here with these requests these two fields now if you check this out let's do control F inside of ice cream PB if you do set scoops you'll take a look in this switch statement there's two cases all auto-generated right no need to fully understand how this works but you can see you have set scoops and set flavor and they both one takes type number and another one takes type string very similar to what we define to your aunt and string how about that so let's go ahead and take advantage of it so we'll do ice cream request that set scoops how about two scoops for now and then ice cream request dot set flavor what's the worst ice cream flavor go ahead take a minute I'm gonna do plain I don't know who in the world would want plain ice cream but to each their own right cool done with our proto two scoops of plain we defined our client now let's put all together let's invoke the RPC let's call the Python code from JavaScript are you ready so real easy client dot the RPC name what's the RPC name order ice cream and we're gonna send in one parameter first which is ice cream request a quick second let's go look at the proto doesn't that look familiar order ice cream and it takes a nice from request interesting interesting just like this we're just going at it right now and after you send that in we're going to have one more parameter which is a callback function and this callback function takes two parameters air and response there you go cool now in this callback function I'm gonna do an if/else block and I really don't feel like typing it all out gonna use this quick snippet boom check that out so if condition' else already typed out for me so if there is an error I'm gonna console.log it so console log and I'm gonna just do this thing broke and I'm just gonna print out whatever the air was how about that and now on the else statement what I'm gonna do is I'm gonna paste this console log error to this line you know just so it's one line block and the else block obviously is gonna be assuming it worked there's no air so I'm gonna do response and instead of saying this thing broke because it didn't write I'm gonna say this balance from Python there you go and this should be good to go we have our main function we have everything we need here I'm gonna do down here just call it right so what you can you know get the output [Music] this looks good all right so uh I'm gonna split the terminal you know do it however you want just have two terminal windows in one window which I'm gonna do on the right side first I'm gonna run the Python server so Python server what do we got in here we got ice-cream server dot py cool it's running let me clear it just so it's a little prettier great so it's running and now on this side I'm going to run the JavaScript server so no or JavaScript client sorry it's a node right that's how we run the files in the terminal another client ice-cream client that J's go for it boom check it out let's observe this notice every time I'm running the JavaScript client something's happening over here but notice something's breaking on this side right so we're seeing some we're seeing activity right something going on and the reason why that we're getting this error is because why don't we go back to our ice cream server this function order ice cream is supposed to handle the RPC right and in our proto remember how it takes an ice cream request oops that was a mistake and it returns an ice cream response are we returning anything in this function or not so why don't we go ahead and return something let's first you know we'll make a variable called response and you know we defined we're gonna use this ice cream pb2 so ice cream you to die skin response and what this is doing is we're going to set a response variable to the ice cream response proto and on this side we can't use get and you know the set fields like we did in JavaScript like we did right here so how we did set Scoob set flavors so on this side what I'm gonna do it just do ice cream response that message sorry and I'm gonna say let's format the string so here you is your insert here I'm gonna say request that scoops scoop and then we're gonna insert the flavor so request that flavor here is your blank scoop yeah that's cool and then I'll just say ice cream the reason why this wrapped around is because I have my editor set that at 80 characters it wraps so you probably have like a longer line here I didn't make a new line here to keep that in mind okay cool this is done and let's return it right return response very cool I think this is done and now we'll go over to Hodgkin client and let's rerun it so let's clear this and run the server and let's clear this and run the client let's go ready ooh ooh no airs see response from Python and then we got this object you know wrapped in brackets here now there's a little ugly right you know this isn't like the optimal what is pivot who knows message-id we don't care about that right what you can do here is you can do response dot get message as a function call so let's go ahead and clear this and let's rerun it there it is here's your to scoop plain ice cream do it again for sanity check boom to scoop ice cream let's get ambitious let's go for three scoops right never consider that all right one more thing what's a decent ice cream flavor you know something better than planes you know treat ourselves we did it about strawberry why not right maybe you have something else but something else but whatever you want ready oh man it's working so what have we done we created our PC we made a JavaScript client and a Python server and we pretty much called order ice cream a function written in Python right here and then we handled the response in JavaScript how about that hope you enjoyed it hope you learned something I had a great time thank you so much see ya

Channel: evan ugarte

Views: 4,260

Keywords: python, javascript, grpc, rpc, protocol buffers, google, protobuf, js, py, vs code, code, 2020, tutorial, programming tutorial, distributed systems

Id: -Mcvbz5J9kQ

Length: 27min 39sec (1659 seconds)

Published: Wed Apr 22 2020