Intro to gRPC: A Modern Toolkit for Microservice Communication

hey everyone we've got a lot to go through so let's dive right in we're going to be talking about and intro to er PC like Matt said I'm I'm Alan tree I'm the creator of and rock I'm in control on the internet follow me tweet at me today we're going to be talking about how we make computers talk to each other more specifically how we make apps that talk to each other and maybe even more specific than that is like how do we build web api's and I think we all know how we build web api is and it is so now it's not it's not so now it's not so it's rest only it's really not rest rest is actually like an architectural principle it's not so much the actual protocols that you're using the actual protocols that most rest api's are based on these days are HTTP and JSON and rest is kind of like the architectural principle of how you structure your API when you're using HTTP and JSON and that's how we build api's and as the talk to each other and thank you wait I have a hot take and my hot take is that rest actually not that great rest api is they are the worst api's except for all the other ones now don't get me wrong HTTP json and rest are great they have I mean there's a reason that all of the web service API is has been built in practically the last 10 years have been HTTP and rest designs and just like the very most basic reasons why like we don't have to go over these things like we all know this but like they're easy to understand right like they're they're a text based protocol you can look it up like we have so many great tools right that have all been built around this ecosystem even before right the fact that this is already how the web works is via HTTP and content negotiation like the fact that we could build api's using those tools using the way that the web already works was just it was great right and it had lots of other great things going for it like the fact that there were really great implementations because of the fact that that's how the web works but I have to tell you something if I never write another rest client library in my life I will die happy man I hate writing rest client libraries and yet when you're building lots of Web API is lots of web services you create an API and you're like yeah I made a Web API and someone's like hey man did you write like I don't want to work with an HTTP client did you write a library in c-sharp like oh man no I forgot to shard and write one in c-sharp and I got okay well I don't really want to use your your library and this is a problem for businesses right who like exposed these api's to their customers but it's also a problem internally when you're building applications they build an application you and you say to another team hey will you integrate with my service and like no you can you can you build me a library for that you're like no I didn't want to build you a library for that anyways you can tell how much I hate writing client libraries but there are lots of reasons that that rest is not as good as it could be and so like I was saying oops sorry that you you one left like the reason that you have to write client libraries is because there is no formal machine readable definition of a REST API it doesn't exist you have documentation right there's their swagger and we'll talk about that later but you have you have documentation you go to a website and says this is how the REST API works and these are the parameters that you can expect but if you want to point a machine at a REST API and say like can you tell me the properties about this endpoint and what parameters it accepts and what types it accepts you you don't have that and so that means that when you want to write client libraries you have to get humans to do it and humans are expensive and they don't like writing client libraries streaming is difficult in REST API is you very rarely see streaming in REST API is because most libraries there are a lot of libraries and languages it's not a good fit for that don't support it well and if you want to do something like bi-directional streaming you just can't so that's difficult there are also tons of like I can't tell you the number of conversations I've sat in when designing an API or someone's like oh man well this operation restarts the Machine like well what do we do do we post to the restart endpoint that doesn't make any sense how do we fit this into the restful paradigm because there are plenty of things that don't fit well into the restful paradigm it's inefficient right it's all text-based which is great for introspection but when you actually like want to do things really quickly and you want to use as little the network as possible like maybe you are on a mobile client that's working on a 2g connection in certain parts of the world then you're in trouble and and even if you did fix all these things like internally you like exposed an API to someone else in your company and they are so very rarely wrestle they are actually just like oh well I stood up an endpoint for you and post to it and here's the body and no it's it's not restful at all there's no like resource that it creates like that's outside that's for like a different set of customers there's the graphical problems we won't talk about those and so the question you're asking is like just G RPC fix all these things and yes it does and I'll show you so let's design a telephony API not like a whole one that that would take ten years so this is a tool F&E API it supports two function calls one is dial and sports to operations one is dial and one is get so initiate a call and then get a call and you can see so there's the service definition calls and you see the two methods and you see that dial takes a parameter called dial roof dial rack and it returns a call and then there's another method called get and takes a get wreck and returns a call you can see below that in the dial rec okay well you specify the two and the from and Tullio style callback that will give you further like a web hook to describe what to do with the call when it connects and then there's the get wreck which basically says give me a sid and i will give you back the call object so this is like a like the smallest slice of the Twilio api that i could fit on a slide about calls and so we can define this and normally when you define an API like you're not used to like writing it in this format right like you get on a whiteboard or something and you write like get slash calls right and you're like okay well let's think about some of the parameters you create like a document about it but this is like a formal definition that can be machine read which means you can run it through a compiler and so there is a compiler for G RPC and so you can run it through compiler and now you have client libraries generated in all nine in nine different water languages oh man I'm so happy I can't begin to tell you how happy that makes me and here is a nodejs line this isn't the generated code that came out of that compiler this is a complete like working like full there's nothing else you have to put on this slide for it to actually work where you import the the stuff that was generated that's the calls GRP CPB saying that you're requiring and then you create a calls client this is like working with a helper library right and then you say like oh well let's create the request object and sorry yeah and then we'll we'll set a sit on it and then we'll call will actually call the get method and send it to that service and then we'll get a callback right with an error in response and the response is the the call object some of you are like oh no oh no I've been here before this is just whistle all over again and and you're right kind of but but not writing other reasons and we'll talk about that later about why this is better than weasel in the soap world but the takeaway to take at this point is that machine readable API contracts are a really great idea and the fact that they got thrown out with the rest of so was kind of ashamed but wait there's more all there's more I love and there's more and the more is that you got server stubs for this service in seven different languages as well so here's a complete go server this is we're implementing the dial and get methods and so you'll see in the dial method we get in a context object and a request object and we return back a call and an error if there is one and so will generate a new identifier will create a call object will stick in at a map so that we can return it in to get stuff I haven't there's a little bit of it to do and fully implement the actual thing to like make calls in PSTN but you get the idea is pretty easy logic to just insert there and then there's the good method right which basically just looks up a call on a map and and returns it you can see like oh if it's not in there then will return an error message that said we can't find the call this is the second part of this is we part where you actually like create a server instance and you basically register the call service the thing that we created in the last slide onto that server and you serve it on a port it's pretty straightforward and that's like those two slides are like a complete working example of that call service and the JavaScript client we generated will talk to that so I've given you like an example which probably want to know like what like well that was cool but like what is this share PC thing anyways GRP CA dot IO is the home page for the project it is a high-performance open-source universal RPC framework on the wire what does that mean well it's not HTTP JSON like what are we doing and the answer is that G RPC is basically a protocol built on top of HTTP - it defines how to actually like write request and interpret the responses that come back from the other side but you're basically just using htv-2 as a transport and the messages that you serialize both for the request and the response are encoded with protocol buffers that's pluggable right now there are no implementations that do anything but use protocol buffers but it is flexible enough that if you want to use JSON or something else message pack or whatever you could do that short version of how this works is htv-2 basically allows you to take a single physical stream and multiplex many logical streams on top of that and every single request and response is a new logical stream on top of it and so similarly this is just mapped even further on top of that where every RPC request is an HTTP to request and the RPC response is an HTTP to response that sets bunch of headers and it serializes the messages into the message bodies and that's kind of this is where we get like the really easy client-side and server-side streaming that we'll we'll talk some more about later this is the best that I could do for visit I've two visualizations because I was really not happy with either of them that kind of give you an idea of like what oh what this looks like on the wire so like you have a physical PCP connection and then here's your the actual like messages that go back and forth similarly this is like more in-depth of what an HTTP to connection looks like I stole this graphic so it's doing things like fetching JavaScript and HTML but just as well you could imagine that there are RPC buffers RPC messages serialized into those requests and responses the actual implementations for the nine different languages that I was talking about basically there are actually three different implementations there is one that is in C one that is in Java and one that is and go all the other implementations are all mapped to the C core as its called they're all just bindings to that there's even one in PHP which is basically a extension that you load into Apache or PHP fpm to get you kind of that long-running connection or long-lived connection one you can usually do that in the PHP world C in Java both use boring SSL is kind of a Google project to strip out a bunch of the stuff from open SSL and the ghost stuff is helped here go G RPC originated out of Google Google internally had a system called stubby and this is kind of the next version of that study was their internal framework for doing essentially this but for Google service this only it was kind of tied to Google and so when they were iterating on the next version of this they ended up coordinating with some other people who had similar problems and decided to open source the project square was one of the major other original contributors and designers of this project nowadays is a free open-source project the back in the roadmap are open their community of people who contribute to the project as well that being said the majority of the development is still is still done by Google engineers and their really great team back to the code so I wanted to take like that basic example and like kind of mutated in a couple of ways because it is just a toy example but when you're putting these kind of things into production you always have questions about like well how do you do I don't know like timeouts for instance like how do you do things like that because there are all sorts of you can you can see plenty of examples of different RPC systems that have been built where people are like oh I'm absolutely that out laters didn't fit on our demo like our homepage so we didn't have to worry about it here's an example this is a Python client for the same service not again not the generated code something that's using the generated code and it you know it's kind of impressive like how similar it looks the fact that it's so tiny like all this stuff has already been taken care of for you and and timeout is just a parameter it's actually supported on all RPC calls built in as part of the system it's even cooler than the fact then that the client library knows about timeouts and you can just pass them in and it knows when to basically return to you and say like oh this call failed actually in G RPC you want to call times out actually sends a message to it actually closes the stream that that RPC was open on so that the remote side the server side actually knows to cancel that request which means you can do really interesting things like if your services way backed up and hasn't even gotten to serving a request it probably won't even see it because the G RPC layer will cancel it so it never actually gets handled which avoids you like doing you've seen services that are like backed up on a whole bunch of work and you're like oh well all of those things are stale now but they're backed up we never had any logic to kill them similarly like the cancel thing if you have long-running work you can actually like cancel it because you'll get notified when that happens another thing that I ran into when I was building stuff on top of G RPC was like hey this is a really cool RPC system but I would like to know you know when calls are being made how long they take what's going back and forth so I wanted to give you an example of that ER PC has these things called interceptors which are basically a middleware more or less if you're familiar with in HDTV context any any sort of like HDTV framework same sort of idea except that you can put them on both the client side and the server side most of the time on on HDTV stuff you you're not doing too much client-side manipulation you do sometimes but GRDC has both of these things built a built in already for you so here's the definition where I create a logging middleware so you can see it gets passed in a whole bunch of different parameters this is the second function on the bottom which I forgot the E in middleware that's cool and so basically what happens really straightforward is we take the time when the request is about to go out to the remote server we log something saying it started we actually call the the method that's the invoker call which is the thing that gets passed in here and then we log when it ended and subtract the time now we actually have a middleware that anytime you create a client to any G RPC service you can basically stick this logging middleware in there and you've got logging around all of your RPC calls in like five lines of code and you can also similarly take the same logic put it on the server or you can take the same logic and instead of logging it push all of that stuff into a tracing service you get the idea so then you're like product manager so you've like got logging and middleware and monitoring setup you have timeouts - you're like ah we're ready to go to production and your PM says hey I've got an idea for like a new API it's really cool tells you the status of a call right so we've seen like the way the Tullio does this right now is with callbacks and the reason you can't do something like what I'm showing you here is because HTTP just doesn't have great support for that sort of streaming API but G RPC does and lets you do really cool things so here's a new method called status and you pass it the same get request that you would have used to get a call so basically just the cid and it returns to you a stream of call events and the call events could be one of three things it could be the status which is an enum of queued ringing answered or hang up or it could be key presses we can stream key presses back as they happen like down the same pipe or maybe we want to stream speaking events I mean you can go wild with this sort of thing but the idea that you can like subscribe more or less to a resource and get streaming updates for you is really cool also you see the one of keyword that's really interesting it's kind of like a C Union but enforced at compile time right instead of like again in your Docs where you're like please specify either one of these parameters and there's a precedence order this like guarantees that this is what's going to happen that your code is is correct and then I want to give you an example of the other kind of really cool API that you can build with G RPC which is a bi-directional streaming API so you know that you can like go through your youyou probably want to be able to at some point like list all the calls that you did and this is another API the toilet supports and what would that look like in Ag RPC world what's something that you could do with it I'm sure you've all like people have thought about like how to deal with listing things and at some point every business or our API designers like halt man well you're going to have a lot of calls you're going to have a lot of anything that you create so we have to be able to page through those results because we can't return to you a million records in one API call actually you can return a million API records in one API call and so here's an idea of how you could actually model that where you could say like the thing that you put into it this example is wrong oh that's really bummer in inside of the list parameter where it says list req it should actually say stream list req the basic idea is that the first message that you send across the client can stream messages as well as the server side so the idea would be that the client streams first a single list req that has a filter and says these are the records I'm interested in and then the server starts streaming call objects and at some point it stops and then the client says gives me more and so it streams another message that says give me the next one alright so you avoid this whole idea of like a cursor or a page number or a next ID and that sort of stuff where you can basically just create a single stream and stream until the client is done so haven't we try like this whole approach that I'm describing like like I was kind of alluding to earlier like haven't we tried this before what about so what about with it wasn't that a really bad idea like wasn't there reason that we gave up on that and there was and those mistakes and those failings that applied to this open was the world like we learned from them and the GRCC team has learned from them and the project has evolved in a way that basically learns from those mistakes so soap and whistle like they were in extra blight XML there was no way to like switch to a new serialization format when or if a better one came along whereas GRDC is structured in that way the service definition format was XML with like xst like to actually validate of that an XML document was correct which is really heavyweight like compared to the service definition that I showed you before where you could fit like four RPC calls like the definitions for them on one slide instead of like multiple pages of XML that you have to like wade through like the actual definitions of G RPC services like the IDL that I showed you earlier is so straightforward and so simple that when I want to look at G RPC api's I just open up that document to look at what the parameters are and what the services what the methods are that I can use you would just never do that with whistle right and like whistle had like it had a bunch of unnecessary like so panel would still had a bunch of unnecessary things like there's two-phase commit like why do you even need that it still like doesn't solve the performance like the streaming stuff there are a whole bunch of reasons you might ask well what about swagger and it does solve the machine readable problem right where you can actually take like a swagger definition and actually generate API client libraries out of it and server stubs and people have built those tools to do that sort of thing it still doesn't solve other problems for you like it doesn't give you streaming so you're still stuck with the inefficiencies of running text based protocols like HTTP and JSON but even the fact that you did get a machine readable contract like it's still it's like the XML one or like this is the definition of a get request for one resource and that actually it didn't fit on the slide it's longer than that right where you can compare that with like the protocol buffers definition where is much more succinct and easy to read and kind of straightforward you may grunting what about thrift actually like search was a really great idea and still is and thrift for you know institutional reasons like kind of got stuck in this place where it was a really great message serialization format and then there was this effort to create a service based contract system around it and it never just quite got there the difficulty in building a system like gee RPC is just the sheer amount of effort that goes into building it in order to really build something that is universal that works across all major production languages you have to build implementations in all those and they have to be well documented and the implementations has to be reliable and they have to be performant and they have to be really easy to set up if you want to get adoption across all adopters more or less and thrifts the thrift project like it just never quite got there so that all sounds really great and you're probably wondering like well it can't be perfect there's got to be some bad news and of course there is there are plenty of problems that you run into that you maybe didn't before the first one is is load balancing which is an interesting problem for G RPC the fact that you basically run all of your RPC s over a single long-lived connection means that you need some sort of load balancer to basically take those are pcs like pull each stream out individually and then maintain back-end connections and those things over and that basically doesn't exist yet I like ended up writing one myself which you know had bugs and I had to fix them it would be really nice if there was one that was just off the shelf and they're getting there it's just not quite there yet air handling is really bad I'm going to show you an example why that's such a pain it's really nice for like the simple demo tutorial case but like I was saying you always run into these problems where you take its production you're like oh no you guys did not think this through and they are handling is one example that you can't do this from the browser that's one of the biggest pain points right now is like the fact that well with a REST API you can call them from a browser right I mean sure you set like the access control headers but then you're you can basically run a JavaScript client that has only talked to rest ap is not there yet the fact that it basically relies on having intimate control over an HTTP to connection means that in order to bridge browsers to talk to this you have to have some sort of like shim layer in between documentation still working on it it's not there and then standardization across languages is a thing that it's not really a deal very rigid is kind of frustrating I don't if you wouldn't remember but basically like you saw and they're just like small things like in Python you say like create stub but in another language you say create client right and it's just it's kind of annoying here's basically what air handling looks like so you get a message instead it's like HTTP that makes sense that's straightforward just like you should be you get a message in a status until you're like oh wait well no I actually have a whole request a response body that I can put stuff into with HTTP as well where do I put that in here oh no you don't and that's really frustrating when you want to add rich air details to an error message so I like ran into this problem where my software I wanted to add basically a single bit that said is this a error message that I should show to the user because they did something wrong or is it an internal that I want to log and basically return a generic response and there's no really easy way to do it you can do it there like things that are called trailers and you can like serialized stuff into them and pull them out on the other side but it's annoying and it should be easy right because you should have this as part of the contract part of the definition of the service and can't do it like I said the dr. poore this is just one example this issue still open it's like open for eight months but like there are all sorts of all sorts of stuff like this that's still going so it's still a project that's evolving it's still not like the battle-tested land of HTTP where these things have had years and years to mature I want to talk briefly about G RPC in production so I run G RPC in production at end Rock when I started and rock I was basically two services they basically just use a single HTTP JSON endpoint to talk to each other eventually got to like five services and I was like aw man I don't want to write client libraries so I switched to thrift and then I ran into a whole bunch of problems with thrift around the implementations not being good enough and causing problems and not coping with errors and timeouts well and that was just fortuitously around the time the G RPC started really gaining steam and getting adoption and so today and rock is 20 plus services spread over five regions and literally all of them talk to your PC to each other doing millions of our pcs every day it's not just me there are plenty of other companies that are using it just a few examples Square uses it for basically all of their internal RPC some of the quite transitioned all of it over there yet but it was it was getting pretty close core OS they're actually people who this is like a really easy thing for people to be like this is what I want to do in production or internally for my services internally but they're actually people shipping this to their clients right like public API is so core OS at CD their public API is is G RPC base and Google as well has actually exposed multiple GRP c.a.p is as well so pub/sub I notice is er PC and speech rec also and I'm running out of time here so and we're on the last slide so this is the future there it's really easy to track what's going to happen in gr PC because it's all developed in the open like read the mailing list go to the github repositories there pull requests for all the proposals about how to change it it is still changing things are you know happening and as far as like where I see it going like I think this is going to see massive adoption for internal RPC like the fact that you can avoid so many of these problems that you see with HTTP especially the the fact that you have a machine-readable contract and you can generate client libraries you can generate documentation you can generate automatic tests around these sorts of things it's going to very quickly become the dominant form of communication for internal services and you'll probably see it like make inroads into replacing existing services when there are things that it gives you like performance or streaming and I think that those those things are actually so compelling that you're going to see a lot of companies start offering public api's just like Google and core OS are as well and Grox next api's are going to be G RPC as well I think you're going to see like this kind of world where you have two sets of API is where you see people basically publishing G RPC api's side-by-side with rest while we figure out like where we want to go and that's about it thank you [Applause] [Music]

Channel: Twilio

Views: 201,027

Rating: 4.8292599 out of 5

Keywords:

Id: RoXT_Rkg8LA

Length: 32min 0sec (1920 seconds)

Published: Fri May 26 2017