Load balancing and HTTP Routing with Envoy Proxy

how we all do it hope this is working are you digging this oh yeah [Music] seriously all right I think we can throw that into the background and enough joking around so we're where we are today so today I want to kind of start looking at some stuff with with envoy specifically what I want to do is I want to show you some some routing with envoy and I think like envoy is is fast becoming this like you know central proxy that's used by well everything and I think for a very good reason it's it's a very very very good tool now most of the time when you're dealing with envoy you're gonna be kind of like dealing with things through well another tool I mean you're gonna be using something like console connect are you're gonna be using Sto or or sort of AWS is app mash you know a number of different tools use envoy there's some really really great API gateways data wise ambassador solos API gateway with glue really really really good tools and I kind of want to cover a few of those in this series of all things micro services but today what we are going to look at is just plain old plain old Envoy because believe it or not plain old envoy is actually really really really powerful a lot of the things that you see with modern sort of servers mashes and sort of technologies it's it's just configuration of envoy and and I think it would be a lot easier if kind of I think the Envoy documentation is really good just I think it takes a little bit of understanding of the architecture so that's what I'm gonna show you today we're gonna set up some simple examples we're gonna see how we can do TCP load-balancing we're gonna see how we can do HTTP routing and we're also gonna see how we can do HTTP routing in kubernetes because yeah all the things right so let's dig in so the first thing that we kind of need to look at when you start thinking about on voyeurs so well what is it we'll envoys a an application single applications written in C++ I believe very very very good very popular community around that and if we go over to kind of github and envoy proxy you can kind of you can see the main project that CN CF project I think it's a really really important tool the the architecture of envoy is something that you've got to consider and this'll kind of make sense when we start looking at the configuration but the way that envoy is kind of composed is that you have I like to think of them in terms of inbound and outbound but we have inbound so that's a listener so some think that is inbound so a port and an IP address so envoy will listen on a particular port an IP address a connection that comes into that port IP address goes through a filter chain and a filter chain is it's kind of made up of a number of different things but you can can chain things together you can have basic kind of HTTP connections you can look at things like doing JWT validation and you can you can do some really sophisticated routing with HTTP of so very sort of protocol specific filtering and and also relational databases and things like that and we've also got TCP proxy and UDP and in the latest version 113 so it's it's it's a kind of a very neat little tool now when it comes to stuff coming into envoy it's a proxy so it has to go out to somewhere so the outbound stuff is handled by something called clusters and a cluster is made up of endpoints now generally kind of when you're looking at an endpoint they're provided by things like console services catalog that provided by things like kubernetes the service discovery via ISTE oh and and and that's kind of like how it gets things and a through sort of a this dynamic cluster set up so the EDS assignment will look at that but what you can also do is just use static load assignments so on void configuration is just camel or Jason you can generate it yourself in a number of different ways you could use console template if you wanted to I'm not entirely certain I'd recommend it I kind of would rather you use like something with a control plane like like ambassador or blue but it's worth seeing how because sometimes it's pretty useful so let's dig in what are we got well so I've just kind of scaffold some some basic Envoy configuration but I just want to kind of show you our application setup so what I have in the the application is for TCP to api's I've got API one so two API instances one and two and what I want to be able to do is I want to be able to load balance just using round-robin between these two api's so I'm defining the api's here and these are just gonna be docker containers I'm just using a little tool I wrote called fake service which fakes a service and and I've got an envoy proxy and I'm just using envoys docker container so I'm using the Alpine version of envoy the latest one one thirteen point one and inside of that docker container all I'm doing is mounting the envoy configuration so by default when the docker container starts it's gonna start envoy which is as simple as calling the the Envoy process and it's gonna specify the configuration file which is located in etc envoy envoy llaman so I'm just gonna mount my own volume or which I have here into that location so let's have a look at those listeners so we're defining a listener and and again on voice support multiple listeners but this this is a kind of a very basic listen so I'm gonna define an address so what what do I want the listener to listen on now it could be a specific IP address or it could be all IP addresses in this instance and I also specify the port that I want to listen on and I'm just gonna listen on port 80 now every listener has a filter chain and every filter chain consists of multiple sort of filters Network filters so what we need to do in order to do our network load balancing is is two things one is we need to define our filter our network filter and the second is we need to define a cluster which points to our API servers so let's let's have a look at that so over to the e to the Envoy docks so I'm gonna kind of go straight over to API and I'm gonna look at the v2 API reference and I'll put all these links down below there but I'm gonna look at the listener configuration so if you look at kind of listener configuration here and what listener configuration has is a name that address that we saw then these filter chains it's got a kind of a no thorough other kind of capabilities here we can kind of specify a number of different set up such as the kind of the timeout and there's actually some specific settings we can put on each each filter but we're looking at the very very basics so filter chains a list to fill the chain to consider for the listener the filter chain or the most specific is matched for a connection so as I said we can have kind of multiples of these chains and if I kind of click through here we can kind of see this this kind of setup so what is this what does this look like well well ultimately what you end up defining is a listener filter now the key thing about listener filters is you'll see here that it just has like typed configure it--and and this is kind of when things start getting a little bit lost because it's like well it's a dead end it's just a typed config but the thing is that the listener that you define here in name is of a particular filter type so here's some of the network filters you can see we've got TLS authentication we can do things like simple authentication external call out to a service external service for authentication we can do things like proxying my sequels rate limiting Redis and and kind of and so and so on what we're actually going to do is just look at a simple TCP proxy and a simple TCP proxy is just going to allow us to take that inbound connection and load-balanced it to an upstream cluster one of the things which i think is really amazing about envoy is that there are a huge huge number of system of statistics like out of the box and you can kind of see here all of those statistics that are gonna be available to you you can configure envoy to output statistics in in a number of different formats that's the data dog like you name it there there is a sort of a plugin for that and maybe if you're interested put a message in the comments below and we can we can look at how you you can figure on voice to admit metrics at a later date but for now let's take a look at that configuration of the filter chain so we're going to define this this filter and with with every filter what we were kind of gonna have is well we need to give it a name so I'm gonna call this Envoy TCP proxy okay and with with kind of every filter we have to have that typed config and if you remember the type configure is kind of like it's kind of like a loose interface it's so that you can kind of configure envoy and and we've can't this is really important but it's one of the most complicated bits that most people find so what we do is we're gonna define this this element here we're gonna call it type and then the type is going to be type dot Google API is dot-com slash envoy dot config dot filter yep don't network okay are you lost yet dot TCP proxy dot v2 TCP proxy now the name of that is is kind of corresponding to the the kind of the protocol buffer the kind of the underlying thing which is configured there and if you look at kind of the API reference here we can kind of see all of these these configuration elements is as they're that pure kind of protocol butter and in terms of where you get the name from well it comes from here so conflict or filtered on network dot v2 TCP proxy dot proto and again if we kind of just click through to the documentation you can see here the the protobuf that defines that api the the reason for this kind of you're defining it inside of the the listeners configuration like this as well but you can also configure listeners dynamically so it's possible to kind of set them up as well but let's let's kind of keep keep digging into this so we have that that type and I'm pretty positive that I've I've spelled that wrong and we'll get to that in a moment but what we also need is the the kind of the upstream cluster so I can specify cluster and what is the name of the cluster that I need specify great question because I haven't defined it yet but I am going to define it I'm going to call the API so if you remember what I was saying when I kind of looked earlier at this kind of this architectural overview you can't have this logical separation you've gone inbound which is kind of the listener the filter chain and the filter and then you've got this outbound section which is your clusters and your endpoints so we've defined the listener but now we need to define the cluster so to to define the cluster and again all of this goes in a static resources because I'm configuring it all statically it can also be configured through the the GR PCA XDS API but a little bit too much advanced for this video but let's some clusters okay and then we're gonna give it a name call it API and if you kind of start looking into clusters and let's kind of dig into that documentation again because I think one of the successes of using envoy is it's familiarizing yourself with the way that the documentation works and the way that it kind of ties together with the concepts so if I look at clusters and then I go to the cluster object you can see it's got all of these properties I can define the name the type II D S cluster configuration that would mean that it gets the endpoints from an API that you configure which corresponds to envoy Sun point discovery service but we're kind of just gonna use the very very kind of static static configuration so let's look so we've defined cluster okay alright so what is the the kind of we're not going to set an ETS property what we are going to do is define endpoints so let's take a look at how we do that back over here so we've got a name API and then endpoints so we're going to statically configure our endpoints here and we are going to specify load balancing endpoints and inside of a load balancing endpoint so we we've got a number of areas of recursion here we specify an endpoint so an endpoint has to have an address and the address is gonna conform to envoys standard socket address so this is kind of envoys protobuf object for defining an endpoint and it's it's just this you see it's just kind of that address port value so I'm going to specify address and because I'm using docker I can reference the name of the container so I'm just gonna specify API one dot container dot Shipyard and I'll explain this convention just in a moment the port is 1990 and what I'm also going to specify is that ipv4 so I just want to switch IP for compart true so let's just quickly look at those options in the documentation and not so easy to see because I am so sumed in come on well there we go over here alright so the dress it's a core dress so socket address this is kind of how you learn to sort of navigate through you gotta go thinking that everything is this kind of the documentation is all referring to that protobuf so got the protocol the address I've specified the port file you can use them named port for kind of using specific DNS resolution the resolver name two different options there I can kind of specify things like IP addresses strict D and a so logical DNS we're just gonna kind of use the default which is fine and ipv4 so kind of just to enable compatibility that's gonna bind both IP view for six so that's how we we define a single endpoint if we want to define another endpoint well that's easy I'm just going to kind of copy that endpoint configuration and I'm going to specify it twice there so that should do it so let's um let's let's run our example so kind of just a quick recap there what we're doing is we're defining a listener the listener is going to listen on the socket address 0 0 0 80 so any IP address port 80 and when a connection comes in it hits a filter change the filter chain is going to match the first filter and it's going to kind of handle where that connection should go so the filter I'm going to hit just standard TCP proxy and then what I'm going to do is I've got to specify the type of the object that's used for the filter chain so I'm using this this at type and then I specify the the kind of the suppose the full package name for the filter as as kind of defined in the protobuf so that's kind of envoi config dot filter don't network don't tcp proxy v2 TCP proxy then once we match that what the tcmitch proxy is gonna do is it's gonna forward connection to an upstream defined by cluster cluster here we're defining two endpoints and we're just gonna use standard round-robin load balancing we could do things like maglev if you want to be able to do and of hash-based load balancing so if you're kind of an upstream Redis or you could do random you can do kind of weighted load balancing very very configuration we're just doing round-robin for now but we're going to bounce between these two api's so let's have a look at this these be load balancing and we'll it's work so what I'm doing is I'm just using a tool called shipyard and ship our shipyard just allows you to kind of define cloud native applications so things like docker containers kubernetes clusters and stuff like that and you can kind of just define it as Yama like this so it it came like docker compose on steroids or terraform for local local machines so let's have a look let's see what's going on here well the first thing is that I always write that I did make a typo because my own voice contain hasn't started so let's have a debug and see why so if I do docker logs and I look at the Envoy here we can see what's going on so we're saying exiting the configuration when it's been trying to parse line 26 end of map not found so I love yah Mille as much as I love go mod and 26 it'll be indentation somewhere let's play hunt the indentation actually you know what let's not play on the indentation let's just look at an example and ideally for brevity it right there we go so there's our example let's just run that again kill my own void container I'm gonna run that again all of these examples are in github I will put that in the the link below this time we've got everything up and running so what I have done is I've just I've just used Shipyard to define an ingress which is gonna point to my from my local machine to the docker container which is running envoy on port 80 so if I curl localhost you can see there that I've hit API 1 I curl localhost again you can see I'm hitting API to localhost again 1 2 so we're doing kind of standard round-robin load balancing between those things if I kind of look at the the envoy logs there and you can kind of see that there's nothing much going on so I'm not running in debug mode but everything is is up and running so that's a really really simple example so that's how you would do TCP load balancing how do we do something a little bit more complicated how do we do protocol specific routing right let's have a look so what we we kind of have to do next is we look at routing and routing is again going to use that kind of that setup we this time have got in our applications we've got a front end container in a back end container so we have an application that looks like this so the Envoy proxy sits here then if somebody uses the get with a path of slash they're going to hit the front end application if they use get with a passion path of slash API they're gonna hit the API so kind of pretty standard load balancing so how do we do that all we have is some Envoy Yama yet again so we we this time I've got a cluster and we've got a cluster called front end and the front end is just gonna go to front end but we've also got a second cluster called API and API is going to go to well API but what we need to do is define that filter chain so we need to define what the the kind of the filter chain is going to look like in order to be able to do that so for a filter chain this time we're not going to use TCP load balancing what we're going to do is we're going to use HTTP and the HTTP connection manager let's take a look at that in the docs so again back over the listener in the v2 API section we're gonna look listener there and it's gonna say the fill chain navigate through the filter chain to the specific filters and all of these supported filters so what we have are those support filters now in addition to this we have a very complicated filter which is the HTTP connection manager and the HTTP connection manager I'll look pretty much let you do anything you could imagine so you can you can kind of manipulate headers you can add headers you can take them away you can modify the values on them you can kind of do traffic shaping and splitting you can do routing of course everything is going to give you statistics you can do protocol aware HTTP retries and failures and some really really incredible stuff I just want to kind of look at pretty pretty basic pretty basic routing so how do we how do we do routing and again these docks might look a little bit confusing but you know just bear with them you will get used to it a little bit trial and error but there's there's a lot of great community it's a great slack group as well so again filters what we gonna do we need to define a new filter so we define a filter and it's gonna be Envoy I'm gonna use HTTP connection manager so a filter has a typed config and well again we've got that type because we're going to define the type as it would be in the the pro buff and that's gonna be type dot Google api's comm slash envoy dot config doctor filter network HTTP connection manager dot v2 dot HTTP HTTP connection a juror all right we're there okay so we we kind of do that now what we we kind of need to do is we need to look at that HTTP connection manager around specific configuration but we kind of need to do things like specify the codec so let's take a look at that we have a route we've got too far connection manager we actually want is just the configuration on this v2 API reference there we go so the codec type we can define what kind of codec it's gonna use so you can specify specifically whether you want to use HTTP or yeah everything's nice and brief right HTTP 1 or HTTP 2 now kind of HTTP 2 obviously we're gonna use G RPC and things like that then I'll kind of maybe look at how you you handle that but you can also specify to order order we'll we'll kind of attempt to make a best guess of what the protocol is and if it's upgradable it'll upgrade it to 2 HBT 2 I'm just going to specify a kind of a hard value of HTTP 1 because that's all I'm using so then what we we kind of need to do is we can define this route config so the route config is kind of going to be the the kind of the definition so we'll be able to pick up from the path or the HTTP header or something like that and send it to a particular back-end so give it a name it's gonna call a local config and I've got a specify virtual hosts virtual hosts and let's specify our first virtual host which is going to be back-end so if you've ever configured nginx and you've kind of looked at back-end services kind of very very sort of similar rum similar setup so we've got named back-end now some of the things that we can do here we can do specific filtering on domain name so if the backend was was specifically kind of not path base maybe you want to use host header based kind of routing that API dot my service comm goes to the API and front-end goes to well front-end we can kind of do that we can kind of specify all of these sort of options but but I'm I'm kind of really just interested and looking at handling everything for for this one so once you've defined a domain you specify a route to so that the domain is a filter and then you can specify a route and then you put a match on that okay like so so then I'm going to define a prefix and a prefix can be a a regular expression but what I'm gonna define is I'm just going to specify API so if if I have a route that comes in here and it it matches API then what I'm going to do is I am going to send that to a cluster and I'm gonna send it to the cluster called API we've got this one down here right now if I wanted to find another route well I can just add another one very simple here so I can specify that maybe just slash goes to front end so that kind of help shows me how I can define my my route configuration if I kind of again look over into the Envoy documentation and I kind of want to start looking into that that route config you will see that we've kind of got a number of different options that we can put in there but but let's just kind of look it look at path for now so the the kind of the the last thing that we need to look at on the HTTP connection manager is extensions sorry let me just I've lost my place and the docs fill the chain filled there we go the last thing we kind of need to be sort of looking at and let's just kind of look at the API documentation here is that we can kind of specify a number of different options so we can specify these HTTP filters so what is this all about so the HTTP filters the list of filters that make up a filter chain for the request for the connection manager so when you define your connection manager what you're also defining is the route config but you also need a kind of specify down here in addition to this we need to specify the HTTP filters filters I'm going to do it's gonna be named envoy route and we're just gonna use the default options okay so I'm bound to have made a typo in that so let's just save ourselves the trouble and the codes all err so you can never play around with that but let's run this application so Shipyard run alright and what I'm doing is I'm just gonna run that so I've got my my API here my front-end my back-end docker containers I've got my Envoy proxy same thing I'm just loading the configuration there and this is the configure and I'm using this is my my kind of simple set up there and all of that is running so he'll localhost and what you see there when I curl localhost is I'm just hitting front end because you remember our route config said that just slash hits there but if we do API then what we're gonna do is hit the API endpoint so we've got kind of you can see there if I do API is because it's it's just greedily matching ap he's heading front and it's API so we can kind of set this up and we can configure this however we want it's really really powerful and incredibly incredibly fast say I think the configuration takes a little bit of getting used to but you know once you've kind of built your your first example it's very much much copypasta alright last example how do you run that in kubernetes so we've just been running this in docker containers so far but let's kind of a look at how we would do a similar thing in kubernetes so kubernetes and of course well we're just going to set up some kubernetes services so this takes just a second so let me just run my kunos coaster again I'm just using shipyard run and this will create me a kubernetes cluster and docker and install all of my applications so the first thing that I need to do when I'm gonna run this in kubernetes is I create a service now I don't need to create a service but this is gonna give me some entry point to my envoy and for my Envoy I've got that exact same configuration that I had in the previous example but what I'm doing is I'm putting it into a config map so you can see here the kubernetes config map and that's exactly the same as it was in the previous example we've got our routes there we're using the HTTP connection manager we've got our clusters now in our clusters where we Rob see just using the kubernetes service and because everything's running in the same namespace I can just use front end to reference the service or API to reference the service because that's what I've called them for running envoy itself well I'm just gonna run a deployment so I've got a standard kubernetes deployment I'm using envoy Alpine I'm mounting that config map to a path and then I'm just loading that and it my container so pretty much exactly the same thing that I was doing with with docker compose right nice and easy my application will I have a service defining my front-end that goes to my front-end deployment again I'm just using that fake service and here I've got my API service that goes to the deployment API and again just fake service so if we take a look at this and that's all running we've got our kubernetes cluster there and our ingress if I let me just coop CTL get pods and you can see that I've got my envoy pod in my front end pod and my API so curl localhost and you can see we're heading front end so the ingress is hitting this on voice service which is pointing at this envoy deployment envoy is doing that rooting for me and it's routing between my API deployment or my front-end deployment just depending on what that routing table is so it's it's this is actually a really really simple way of just getting a very very quick and easy sort of HTTP load balancing kind of routing thing going inside a kubernetes cluster I mean you can of course use nginx rhe proxy traffic there's some wonderful solutions but envoy is um is really I'm really very taken by anything it's a very good piece of software and it's just cropping up absolutely everywhere comparing sort of envoy with with linka D and and things like that well envoy is the the sidecar or the data plane so we're kind of not talking about service mashes lolling kadhi is a service mesh quite often you you find envoy will be part of a service mesh it becomes the data plane link ad doesn't use envoy lickity has it's very own proxy which is an incredible proxy in its own right very very fast link ad is the control plane to to that data plane sto is the control plane to on voice data plane and console is the control plane to on voice as is AWS app mesh but that's the kind of the control plane but this is just very very basic proxy when we're not even looking at service mesh or anything like that yet and that's it I well I I hope that you found this useful I hope it's been a a nice little kind of introduction to well like what's going on with with envoy like envoy is powering so much of the web it it's a very very very very great tool you can see it I think cropping up everywhere there's some wonderful stuff which is kind of going into it we're seeing some some really interesting things around like being able to compile your own filters using web Assam and I'm hoping to get some videos on there where we can maybe build a filter for envoy and see if I can bother Christian post her to come along and do that with me but also I want to show you in a couple of videos time I think it's fair to kind of do things and we're gonna look at glue which is an API gateway for kubernetes built on envoy and also data wire by Ambassador we're gonna look at both of those things and we can see how much better it is when you're using a control plane as as opposed to having to configure this yourself but until then thank you as always please don't forget to comment to LIKE subscribe smash the bell button and well if there's something you don't like hit our thumbs down button tell me but for now that's not music

Channel: Nic Jackson

Views: 21,682

Keywords: envoy proxy, kubernetes, docker, microservices, envoy, http routing

Id: D0cuv1AEftE

Length: 41min 43sec (2503 seconds)

Published: Thu Mar 12 2020