Introduction To gRPC

what's up youtube in this video we're going to be looking at grpc what is jrpc why who when all of those questions we're also going to be looking at the architecture of grpc and to some extent it's going to be kind of a bit bit of a deep dive towards the end of this this video and then the plan is to have additional videos specific to dotnet c-sharp where we have uh getting started with grpc and then probably another video after that a little more advanced so today's video is mainly grpc this is completely independent of dotnet c-sharp whatever it's just what is grpc why do we have one who's using it when did it start stuff like that and as well as the underpinnings the architecture of the jrpc infrastructure that gives us some of these benefits and why these things were thought about and things of that nature so i think it'll be interesting for those of you who want to know or learn about grpc independent of anything.net it's kind of like my websockets video before i went down the path of signalr because websockets is signalr is built on offer websockets has built a lot of on top of http 2 so is grpc by the way so there are some similarities there but other than that so think of grpc as an alternative or a competitor or some technique technology that you would use in addition to simultaneously with along with rest rest works for certain situations but does not work for many situations you can have restish services these are not rest in the sense of the true rest followers will say that's not rest correct because the rest almost is good for like crud operations for the most part but there are many operations you have on your services that are not crowd related and then rightly so because services should not be presenting a crud surface that's true that's the whole point of having services is to give more business applications or business service or a business api so very it's very rare at least in my experiences to have a service that surfaces crowd operations and nonetheless without going to that argument rest has this place but rest doesn't achieve a lot of things and the fact that rest is built on top of a protocol and has certain sort of requirements human readability json or xml these kind of things is exactly why we have a grpc and i'll get in the details of that at a very very high level grpc's motivation was performance so what does grpc stand for as you can see here on the slide grpc which is an open source high performance remote procedure call framework stands for grpc remote procedure call it's weird they call it a recursive name or something i don't really care whatever it is grpc the g don't stand for google even though google was instrumental in starting the project that was the precursor to grpc called stubby we'll talk about that the g apparently did not pertain to google google was trying to make this open source and they didn't want their name associated with it so i'm not sure why they had g then over there either ways remote procedure calls are slightly different from restful or rest based calls while rest is more as i said crowd operations remote procedure calls are mainly good suited for at least from a if you're trying to be very particular you can use just for anything you want in anything remote but then the rest people say but that's not rest so it's rest ish it's kind of sort of less grpc rpcs are remote procedure calls this is pretty much anything you want to do calling from one remote application to another remote application we've had rpc capabilities since actually since before when i started my career in programming we had dcom distributed com corba then came xml rpc and then and i had my believe it or not i had built my own rpc capability using raw tcp back in the day because we didn't have things like soap and rest and all that and sockets were not available out of the box you know when uh toolkits like http client is so you have to make your own socket you're gonna make had to make it then from the socket have to make like an http socket or an ftp whatever socket and so i had made my own protocol between client and server and so i was able to do this remoting i didn't call it remoting but that it was i could send messages across the wire to server side and eventually we also took this to cross platform meaning this was in delphi initially and then we looked at we started using c-sharp.net back in the day and so to allow for a smoother transition from delphi to.net c-sharp we had to have the ability to communicate from our delphi applications to the newer c-sharp applications and eventually kind of uh sunset the delphi application so anyway the idea is that there are ways you can have remote procedure calls but remote procedure calls essentially meant pretty much any calling methods sending messages across getting back return messages grpc stands for remote procedure calls some people could argue that rest and grpc can both live together i think that's true to some extent but not because one is rest and one is grpc or one is rpc and one is rest i think they may coexist because of the fact that you might have to support certain clients that are still wanting to use rest but i think grpc is sort of the future right but i can see them very easily living kind of hand in hand or together in the same ecosystem the genesis of grpc started with a project called stubby from google google had this need for a common cross-platform remote procedure called framework that they needed to use within their own data centers and across their data and data centers across their services they needed something that was super fast efficient cross-platform because having a cross common cross platform framework or infrastructure gave them the ability to get benefits such as efficiencies of performance security as enhancements are made to this common framework they saw these benefits across the board behavioral analysis and things of that nature a company like google would benefit from and so they created this project called stubby simultaneously there was another project called speedy which is also internal to google at that time but by this point speedy it become uh http 2 and a standard proposed two ietf and then eventually become a standard and that was like about 2015 and so we had http 2 and the stubby folks were looking at what they were doing internally now stubby had some incredible features but it was sort of proprietary it depended very heavily on knowledge and understanding of google's own internal infrastructure it was not open source it was not uh available it's not ready for open source and all the other benefits that they got internally however because of the http 2 coming out and becoming standardized the stubby folks felt you know what maybe we should rethink our strategy here use some of the benefits we're getting with http 2 that we're not getting this stubby and then piggyback off of that and try and get the benefits of stubby and http 2 as this combined thing and let's open source it and so on and so forth the rest is history i think in 2015 they proposed this new project called grpc to the ietf actually to the cnc ncf which is the cloud native computing foundation project and so that was like the the incubation of grpc and then since then in 2015 it was open sourced and so because you know it was the proposal and things of that nature now this project the cncf project is managed by google and a bunch of other people from various companies who have feel like they want to have a say or a stake in this idea of where this grpc is going and so you have pretty much most companies like microsoft and netflix and others participating in this project it's an open source project so of course you can have pull requests and things coming in but there's something unique about this project the way it's run there are various toolkits that are for the different languages like c-sharp or kotlin or swift go etc all of those toolkits or libraries are managed by the project to ensure that grpc is truly cross-platform so they have a bunch of tests that they run internally to ensure that some library a language a can talk to a library b and all of those tests pass and then vice versa and kind of the permutations all of that i think that's a really cool feature because that's not leaving it up to the library sort of maintainer if you will to ensure backward compatibility the project itself is taking on the owners or the responsibility of saying you know what we are going to make sure these libraries are compatible cross-platform cross-platform in terms of operating systems but also cross-platform in terms of languages right so i think that's a unique cool feature of the grpc or cncf project because it's taking on more responsibility and forcing the powers that be the people that build the libraries to collaborate internally before they make that available to the public right so the public is dealing with some of the incompatibility issues and that's not fair so that's one change i think i know i don't know of any other project that has a similar way of operating so one of the reasons as we heard from stubby's history was to have this high performance low latency infrastructure framework for remote procedure calls and so if you have a need internally within your data centers within your application and enterprise where you need to talk to multiple services using grpc for those internal communications i think is probably the first place to start if you're looking at using jrpc leave the sort of public facing parts to rest in aps but for internal purposes i would say you could use maybe within your own project if you got a bunch of microservices within your own project then your projects could start using grpc within your own amongst your own services and then eventually across the enterprise and so on i think that might be a good migration path for those of us who would like to use grpc okay so low latency high performance cross-platform uh or and portable meaning mobile uh pc laptops tablets and iot devices so grpc works across all of these things and also low latency lightweight etc there are a bunch of large companies or well-known companies using grpc already of course google says they started it netflix square square by the way i think also recently bought title title the online music like the spotify competitor in fact spotify has is also using grpc they've been using it quite a while back spotify also had an internal comparator to stubby called hermes not a competitor but they were they're all trying to do the same thing they all wanted a super fast low latency communication pro rpc communication protocol that was not hdp based on respace maybe you should talk about what's the problem with the http rest based stuff versus grpc we'll talk about that in the next few couple of slides if you haven't taken a look at my websockets video introduction to websocket that's whole signal our thing i'll put a link to my to it over here i talked about http 2 there i'm going to talk about some of it here as well but essentially think of tcp as a layer and i actually even talked about the osa layers osi stack in that video i'm not going to go into details here but just for those of you not watch that yet tcp is sort of at one layer we call it application label as the bottom most part of the sort of the application layer what we call the transport in osi stack this is the transport layer so tcp is used as a transport layer think of that as the surface of a highway just that you know it's the fact that it's a road and http well so let's say land transport versus air and track and uh you know water ships so think of tcp as land transport and then you have http or some protocol on top of tcp such as http smtp pop3 and so on these are so that's the sort of the surface if you will tarmac this is the dirt road so that's http then we have something called a wire protocol so there's a transport layer and then there's the the wire protocol the wire protocol is the so on this highway you have different types of vehicles that can travel from trucks to buses to cars sedans sports cars suvs scooters motorcycles stuff like that so think of the wire protocol to be the the kind of vehicle going across and how how that vehicle looks as it's moving on this highway right [Music] see i'm going to use this metaphor as we go through this talk i think this this highway metaphor works pretty well for this whole thing so grpc uses a protocol called protocol buffers or protobufs for the buffs for short protocol buffers is a few things it is one it is the encoding a binary encoding format of packaging data and to send it across the wire so it's the the type of vehicle you know is it a bus is it a car a scooter the cycle the sports car or whatever the way that data is packaged and sent across the wire so that is protocol buffers or protobufs but the other thing protocol buffers is is the definition of your service kind of like a wizzle or like with knowledge swashbuckle and others in the rest world having a means to describe your service the scheme of your service the data inputs outputs and the different method methods if you will available on your service so protobufs is also a service definition or interface definition language and it's in a dot protocol something something dot protofile and these are all standards by the way grpc is not coupled to protocol buffs or conversely you can use protocol bus for anything you want of your own it's not it doesn't it doesn't have to be only usable with or along with grpc you can use it for your own tcp communications so it's independent in that sense but it when somebody says protocol about support above the meaning a couple of things the the schema idl we call it the interface definition language but also which is a file we see that file sample of that file it's also the the way data is packaged if you will the wire protocol as we call it the way that data is sent across the wire and it is a binary format right it's language neutral as you can see here on this slide here it's it's language neutral platform neutral protocol and it is an extensible mechanism for serialized serializing structured data now that's not to say that you can't cannot serialize unstructured it you can also see like json if you wanted but that's not the intent it's intended to use structured data by that i mean def to finish definitions of objects details models messages they call it so you ace remote procedure sends messages to their remote server and comes gets back messages we would call them models or details they're making it clear that it's messages it's not like object references it's not like instances like dcom and corba literally proposed that they shunted objects across the wire i got your object in my application with the behavior no it wasn't just data rpc was saying well let's not send objects across the wire because they learned their lessons with dcom and karba and they came up with rpc and our pc obviously remote procedure call send messages get back messages and that's what grpc is trying to do as well just stick to messages and so that's the structured ideal structured messages let's look at the upper example of the protofile and then we'll also talk further about some things that you might want to be aware of i'm not don't need to have a full understanding of the profiles today when we go in the next video into building an application in csharp.net probably be a better time to look at the in-depth version there i'm going to highlight a few things here with regards to the protofile so first of all we have is proto version three there are ways in proto files to import other protofiles where you might have definitions in this case the definition of a timestamp type is over here i won't go into the details of that but that's what the import statement is so here's a message type it's the book type these are the fields in the message there's an isbn a field called isbn it's of type in 64. that's that's in c java long and it takes the sort of the first position and these numbers play an important role in the protobuf protocol you can't change them once you've defined them these are the data types available in proto protocol buffers they are not c sharp types these are protocol buffer types and they support mainly the scalar values you can have your own custom types as well as this one this timestamp thing is over here and uh there's another protofile that uses it this one this particularly does not use it but the idea is that you can have specific types your user defined types then you have the sort of built-in types in protocols itself these are typically the scalar types in floats there's no decimal out of the box there's no date time which is what the timestamp thing is a bull is there enums things are nature but then you can define your own and you can have you can share or use common libraries like google has produced additional types like timestamps and things you might want to use that so that is the idea for timestamp is commonly understood across everyone in your app company rather than you having your own understanding of it don't want to go into details but i want you to understand certain things here so these are the three messages meaning these are think of them as the the models or the details the data that goes in and out so that's the the model and this is a book request and this is a book by author request and as you can see here this is the service definition and what it says over here is there's a method called that's called a remote procedure called called get book it takes in a book request as an input and it returns a book as an output that's what the next one is saying it's get books by author it takes a book by author request and returns a stream of books now let's explain these two things here first of all notice that there can only be one argument to a method it could be a scalar type or it could be like a model that you define like the the structure that you define but only one you can't have multiple so that's one of the sort of features or limitations of grpc is that the every method takes only one thing that one thing could be comprised of one scalar property or pretty much nested any num any depth of nested classes not classes structures we call it messages as you'd like so but from a signature point of view it takes this one thing read the one message so and it returns one message can't return multiple things then it can't take in multiple things and the recommendation is even if you have a single scalar thing you need today let's say in 6264 or n32 you might want to wrap that up in a message so that tomorrow when you want to expand on it you're not going to break it's not going to be a breaking change this structure this protobuf format schema does support extensibility and backward compatibility to some extent with some coffee i'll be going to the details of that soon the stream is if you remember my video on c sharp 8 streaming i made a sort of series of videos there c by streaming this is the same idea here so protocol supports the idea of streaming which is not a it's not a collection of books it's a continuous stream of books that are returned as part of the response so grpc supports streaming we'll talk about that a little bit detail but that's what this is implying here that is the stream they also have the notion of arrays they call them repeats there's another set of messages here and you can see here that there's a person message now these numbers again i said are specific they have to be unique and they cannot be changed once you assign this number to that field it can't be modified later and so here's a person that has at the top it has these three fields the four is here so this is a field on the same message person it is of type phone number and that phone number is another message nested inside of this message so in c that will translate to one class nested inside another but that's what i'm going to show you this is just another field like that's number three this is number four this is the type and it's repeated meaning it's an array the four is not saying is an array of four is just saying that's the position for but it's an array that's repeated and this timestamp comes from that namespace that we imported earlier and that's another field so last updated is a field s4 that's five and the type is the timestamp and this timestamp is coming from this included file here so it's okay we don't understand oh and by the way you can also put comments like this in the protocols these comments c style comments those slashes and slash asterisks is about supported and protocol dot protofiles this is a dot protofile all right so just a brief introduction to dot profile i wanted you to see it so that you understand there's a file that's different described describing the schema but is also used for tooling purposes meaning these are machine readable files at this point we're going to talk about i think the next slide here is performance yes so let's talk about just at a high level white grpc or rpc versus rest rest and http especially when a rest was thought about dreamt up this is before like http 0.9 right something like that these protocols http smtp pop3 ftp these are all text based protocols which means this is a cool thing when i learned tcp the i had a blast over time checking my emails using telnet browsing websites using internet actually typing out the protocol and looking at the response and typing out the protocol it's the coolest thing but all these protocols are text based protocols they're intended to be simple and human readable well our desires our requirements have changed the internet is way too big and it's getting very very busy and we just need more speed and so this text-based human readable overly verbose protocols don't work anymore so while rest is okay for certain situations when you need truly high-speed lightweight protocols you can't use http for the most part http has the protocol has two parts to it it's called like header and then there's the body the bodies the data typically that you want to send across and the headers communications that are going on metadata if you will that goes on between the clients and the servers all right so with tcp protocols you know even if you create your own there's always some header metadata information because the other side has to know how much data you plan to send and then when you're done things of that nature so that at the bare minimum the receiving party because of a tcp post needs to know how much data to expect or some way to to tell if you've finished so it knows that you've stopped transmitting it it's come to the end of the transmission the data transmission so what http 2 and then therefore grpc benefit from is the fact that http is ubiquitous as in it is available across literally the planet now and people have a lot of infrastructure and investment and protocols proxies hardware security infrastructure all of that in that and at the http transport there so if anything that rides on top of http such as web sockets such as grpc will benefit from all that investment in infrastructure so that's definitely a cool thing other than that though this human readable aspect making extremely verbose things like xml json these are extremely verbose protocols and so what if it's human readable the problem is costing us grpc at the heart of it is a binary civilization binary civilizations always work out to be super super small so when you transmit that data across the wires very little amount of data is going across the wire and that's what they needed internally in google when they were doing this whole um stubby project we could all benefit from it because you little mobile application would benefit from transmitting less data across the wire that's going to cost you less is going to be faster use up less cpu resources to try and design that data to consume that less memory and so on so it's a win-win for everyone for for devices for networks for cost and for the end users to see better performance everything sort of becomes faster now because binary serialization you can't see the data going across the wire like you're not going to get like a postman well this is actually there's a tool but it's not postman you can't see the data you can't type it in by hand as a dev or during the development cycle now there are tools that support that feature meaning if you can imagine these two points on the wire from here to here when the data goes from here to there across the wire that's in binary can't read it but just before that you could have a tool that kind of converts your thing to binance in the cluster wire right so it's possible to do that and there are tools that support that and you can make your own if you want it but i'm going to get across that the binary serialization comes at probably a cost that is it could make things a little harder to debug wireshark which is a if you haven't you know don't know what that tool is it's like a sniffer if you will network sniffer it can sniff data going across the network and tell you where connections are being made tcp connections are being made and such and there are i think they called dissectors the detector that i would call them dissectors they're like interceptors they're dissecting the data that's going across the wire and so there are dissectors for protobufs so you can see data that's going across the wire you can see it it'll take the binary and sort of reconvert it back to something more human readable the thing is it's strongly typed one of the issues with rest is anything text-based is like there's no way before you start sending the data that you get stopped and you can send what the heck you want and then you get stopped on the server saying i don't know what you're saying i don't understand what you're saying because these are strongly typed protocols at compile time that is for those languages that are strongly typed at compile time you're going to see the problem and compile time you're going to be told hey you can't make this call that the type you're sending across is not correct the strong typing always gives us the benefits that we all love in c sharp it's a strongly typed language as well and so we understand the benefits of a sounding language i won't go too much into that http 2 transport grpc is not dependent on http in that sense it is it can it is a plugable protocol from a grpc point of view but today we have that implementation and so in that sense it depends on http it won't work on http 1.1 it'll only work with http 2. so of course you have some constraints there if you in case you don't have infrastructure that supports http 2 yet one of the core reasons that http 2 gets its benefit is from a feature in http 2 called binary framing so here's my highway metaphor again imagine you are or you're watching a two-way multi-lane highway each side has multiple lanes and this is like a highway somewhere four to six lanes on each side right two-way binary and on a single lane there are a bunch of cars or vehicles there not because there would be like trucks and buses too right so there's a bunch of vehicles binary framing is the ability to but each vehicle on this one plane you are prioritizing and saying okay i know you're going in this fashion right now but this car that's behind is going to end up being higher priority than this car that's currently in the front going that way binary framing is this facultization or the framing as we call it the packets the vehicles and the ability to define priority of these frames as the data is being sent across the wire most of http 2's benefits come from that that core functionality from there you get things like streaming and you get things like multiplexing and stuff like that so one of the benefits that grpc gets because of http 2 is the binary framing feature then there's something like multiplexing multiplexing is this actually comes from electronics at least i understand from electronics background where have you seen those digital clocks that have those segments you know to form like an eight and then there's a clock so there's so many digits but each digit is comprised of all these segments you know the the kind of you know what i'm saying right [Music] when you look at it you think it's all glowing simultaneously but what happens really is every segment is lit up one after another sometimes it's every digit is displayed and it's multiplex meaning you're only feeding power to one entire digit but you multiplex it says that you're firing them here here here here here here so quickly that it looks like they're all on a human eye is not able to perceive that difference in some cases every segment in every digit is lit up independently which is also happening meaning it's both is the digit and the segment both are being multiplex the idea being you can use the same pipe if you will do the same thing with what's it called optical fiber it's the same fiber think of it like a glass pipe and there's many many many many many people sharing that data that that pipe for data so it's multiplexed each of us gets us also our time slot if you will on the cpu side would be called multi-threading in some sense we multiple threads are able to use the cpu but the time share in some sense right so it's kind of like a time sharing mechanism so that's multiplexing multiplexing is a feature of http 2 that grpc relies on because it requires that ability to have multiple conversations on the same see imagine this highway multi-lane highway two-way highway you can you can use the your side of the highway with multiple lanes and this multiple lanes is called multiplexing meaning each lane is this different communication that's happening at the same time so using the same channel the channel is the the highway itself but multiplexing allowing multiple conversations to be occurring simultaneously in one direction and there's also full duplex http which means both you and me can be talking at the same time without missing a beat so you could be sending me multiple multiplexing conversations from from the side and i could be doing the same from that side and we're both completely understanding each other all happening together right so that's the benefit of http 2 but grpc relies on that another feature of http 2 is header compression so as i said this http protocol comprises is comprised of a header part and a body part or content part and headers can be small but headers can also be fairly large and this is metadata and so header compression allows for the compression of this the header part to send less data across tomorrow we used to use gzip or z lib g-zip or z-lib compression for http back in the day but then there was a some form of a security attack on that gzip whatever and so then since then we have used uh forget the name of this new compression technique with the header compression you're sending less data across the wire so between header compression multiplexing and binary framing jrpc is basically utilizing all of those features the http 2 so to some extent it is dependent on it but it's not in terms of the actual architecture we did a little bit about on that is just a plugable architecture the transport layer by using http 2 we are getting three times maybe in some cases more benefit in terms of latencies and performance and approximately 11 to 12 times less cpu utilization otherwise the binary format is so small and simple to encode decode that's less taxing on cpu and memory on the hardware and it's giving you huge amounts of benefits some cases 15 times faster network wise depending how much data is going across the wire too right another protocol that is currently in the works is part of http 3 in fact is called quick quic some people say quick is an acronym it stands for quick yes quick udp internet connections quick again this is a project that was internally started at microsoft these are different groups in microsoft each have trying to solve a problem this is more at that transport layer so it's a much lower level than speedy sorry um stubby speedy was the precursor to hdb2 stubby was a precursor to grpc anyway quick it's sort of now the http 3 protocol it was proposed to the ietf internet in engineering task force task force in 2015. anyway it's becoming a standard that has already become a standard but now it's being ratified this is the http 3 so http 3 and quick are kind of hand in hand it uses udp instead of tcp so it's quite a bit different from http 2 in that sense born out of a need in within google again to get better performance and there's a problem with tcp and especially in http called head of line blocking long story short tcp is a connected or connection oriented protocol and so if some data got if the connection was dropped you've lost that data and if there's multiple multiplexing multiple pipes on the same connection and the connection server then everyone lose their data so it's a bit of a problem for resiliency purposes with udp which is a connection less protocol some of those issues can be solved and so anyway the shorter is that grpc can also be used with quick which is a transport layer protocol again because this plugability of things jrpc could eventually start being used and they've been testing doing some testing so it's not tied to http 2 but currently that's what we have once we get http 3 we'll have implementations of grpc on http 3. quick in itself isn't is trying to replace tcp not even you know it's not like https tcp system that transport layer while still being compatible to many in many cases or in many senses to tcp yet it's trying to make tcp obsolete essentially and it's nicknamed this is a nickname it's not going to be called that but it's the nickname called tcp 2 like not v2 but slash 2 like http slash 2. this is tcp slash 2 so that's the nickname for quick because it's trying to replace the transport layer anyway enough over the performance thing let's move on to the next slide okay so here's the the grpc stack of the architecture of the grpc stack there are different libraries mainly jrpc c core as you can see here c called grpc c core that's the c language core and many other languages use the c core as its sort of underpinning foundational core and then those languages have a wrapper on top of that through this api the surface provides an api and that api is exposed to any language that wants to use the c core for its own purposes and such languages like c plus plus and python your node.js php ruby etc use that now we had or used to have used the same c core in dotnet but now in.net decent.net versions we have our own native implementation of grpc so whereas earlier versions of dotnet relied on this library that wrapped around the c core now this whole stack that you're seeing here the grpc is like all of it is written in dot net core so we don't have reliance on the c core but other languages do rely on the c code and then there are some languages that have their own core the java core and the go core and the dart core and now we have the.net.net five core core so dot net five core the core grpc which is built purely from or completely in.net and it's cross platform and so on so forth so let's talk about the design of the stack that might give you some insight into extensibility and the basic architecture so the core is comprised of two parts let's call it the top part of the bottom part the bottom part the i o manager is everything to do with hardware transport every platform is totally different this platform the io manager platform code base is highly specific to every piece of hardware and every every operating system and so on and so forth so it's the one layer that is tightly coupled to the thing it's running on at that moment in time be it a phone or a machine and whether it's linux the unix of my calls or whatever operating system this piece has got that the code that is going to be tightly coupled to that implementation detail right from that point forward it gets a little better then we have the transport okay we are planning to use tcp http 2 as our transport so i said it's pluggable you can remove that transport layer swap it out put another one in we want to go to quick do that i'm not saying it's simple but it's built in that sense to be pluggable replaceable and pluggable the surface is essentially providing the api portion of it so this is of the public surface of the grpc core and any language that can doesn't want to build its own core can use the surface api to build their wrapper so i think for kotlin the wrap on the builder app around java so kotlin is another language that's supported or uses grpc i don't know the implementation details but i'm guessing since it's java based that they would wrap build a kotlin wrapper around the java core javas grpc java core [Music] then the filters is very much like middleware if you have seen my video on asp.net core middleware i draw this whole pipeline over this going vertical same idea horizontal vertical requests come in and then it goes through different parts and you are able to intercept you can intercept any traffic going back and forth by having an interceptor it has to come to you comes to you and then goes back out comes back and goes back out right so i'm sort of intercepting the traffic so i see the traffic going in and then it comes back this way and then i see it going back out and come back this way and i'm able to look at and change and modify what i in receptor middleware they're all filters so basically it's called filters interceptors middleware one of the same thing to give us features like monitoring tracing security logging you know all these kind of things anything that has to be that has to happen or that you would like or you would like to modify you can use filters as the extensibility point if you will at that level so now that transport lot not at i o but you know at the sort of the application level not your application the grpc application level and so some things come out of the box with grpc like monitoring latency etc but these are where you would extend that grpc supports different kinds of well speed available grpcs support special specific kinds of communications it's a unary communication meaning i make a request to give me a response i make a request to give me a response that's called the unary more mode of communication then we have client server streaming so i make a request and you're streaming data to me and i just keep receiving data until you tell me you're done it's your streaming data it's not one response it's a continuous stream similarly client streaming where the client is streaming data to the server instead of the service leaving to the client and then bi-directional streaming so both sides are streaming now the general mechanism for making a connection or when you want to communicate goes like this as you can see from the diagram here you first open a connection and then you make a call but when you make a call it implies you are sending some initial metadata then you're receiving that initial metadata from the party that you send the data to so there's a sort of handshake communication going on then you send the actual message then you receive a response to that message and then you send a trailing metadata let the party know that you're ending the conversation if you will and you receive the trailing metadata to get the confirmation so when you make a call that's what's sort of happening now while this is happening this metadata here is opaque to the protocol it's like it's used in http 2 scenarios it's used in the header so it's not interfering with the data and that's in essence it's opaque to the protocol the data is independent of the metadata and you can send all kinds of information on the metadata as you want because both parties need to understand what that means but of course within the protocol itself submitted it is exchanged as part of this this exchange protocol and we will talk in the next slide now some of the features we have so let me just go back so the extensibility comes from those filters that i talked about yeah filters interceptors middleware multiple transports can be supported because by swapping out the transport layer authentication comes from the interceptors as well tracing monitoring stats okay some of the other features in grpc is error propagation and if you don't like the error propagation thing mechanism then you can use the metadata features you can also customize you can send your own metadata across and that we can have servers talking back to the class propagating additional information related to error propagation in case you find the built-in features not satisfactory all right so that's also possible that's why i brought up the the communication in the metadata exchange part because that might play a key role in your way you think about grpc you have things like auto reconnect and the reconnection is an exponential back off if you know what that means basically the idea is if i'm trying to connect to you and i can't connect i'm going to try immediately i'll i'll back off i'll try later so maybe the first time i return 10 seconds and i might try in 30 seconds then i might try in a minute i'm exponentially backing off so i don't kill you trying to kill your server so it's all automatic and it's also using exponential back off as a mechanism for retries cancellation timeout and deadline sorry cancellation propagation and deadline propagation cancellation is when if you as a client say i'm calling you and you're calling somebody else calling somebody else calling them the chain of calls this start from the client originating the call and for whatever reason i want to cancel the call because my user decided to do something else if i send you that cancellation you can propagate down that downstream to all the services you've called so all can stop doing what they're doing and not waste time doing something i don't ever need anymore right so that's the cancellation propagation deadline timeout is similar but the idea is that i say okay i wanted to get me this information as a client i'm talking to the server but you got to get give me this back in this much time so i'm setting up a deadline like a manager tells you you gotta deliver this to me tomorrow that's the deadline you're dead similar idea and a deadline propagates as well so if let's say you have to call out the party and i've given you let's say example you know 10 seconds just gonna take you five seconds at most you've got five seconds to give the other person to get the data they need you to get used to that you can de-orbit for the five seconds right so you can propagate five seconds downstream from you and so on the idea being that that gets propagated so that it's not just known you know by the one person and there's no way to communicate for me to tell you that there's a deadline and if you pass the deadline you can just stop because i've already gone away i can't wait around so i've gone away so no need for you to spend your time and effort continuing on only to find out you know an hour later that i have left some of the other features that come out of the box as part of the metadata as well as the filter mechanisms is things like load balancing and they have the standard door balancing like the round robin and the first available they've also introduced a new uh style of load balancing called local side load balancing essentially instead of so with load balance you need to know who's free and so this look aside is saying okay there's something else that's giving you this information about what servers are free here so something else is updating this data here on another server and so you could look here before you make the call you start making the call and finding out you go back somewhere else so that way i in with one year or one i can figure out who to call and then i make the call to the person directly so i'm not going call here to be sent over there because that one is free you know with load balancing you go to one machine and then you send to another machine there's an extra hop there but look aside the idea is that these two can have be happening independently where i can determine who to call before i make the call so it's not part of the request it's there's two hops but that's not part of this one request that i could be making this before i make that a bit complicated all right well i was hoping to keep this julie short and i know i've been rambling on a bit i'm gonna try and cut out some of the stuff in the edit so i don't know what all i can cut out but i hope this has been fun has been enjoyable and i heard well i hope you've learned a few things if you have please give me a thumbs up and i will see you next time

Channel: Shiv Kumar

Views: 3,267

Keywords: gRPC, Remote Procedure Call, RPC, HTTP/2, Stubby, SPDY, WebSockets, .NET 5, .NET Core, C#, XML RPC, SOAP, DCOM, CORBA, QUIC, .NET Core 3.1, IAsyncStreamReader, IAsynEnumerable, RpcExceptions, RpcException, gRPC Streaming, Unary, gRPC Unary

Id: Vsw4saPiTPM

Length: 50min 26sec (3026 seconds)

Published: Sun Mar 07 2021