Contour: Advanced Ingress with Envoy

hey everyone welcome to this talk about contour and ingress controller for kubernetes built around envoy I'm Josh and I actually gave a talk around this topic at the kubernetes colorado meetup and forgot to hit record so i wanted to record some of the content to make sure those who maybe weren't attending or those who wanted a reference after the fact could kind of come back to it and look it up so along with talking about contour as an ingress controller we'll also be looking at the ingress api and how we tackle some of the more advanced things that one might need to do with ingress as their journey around ingress begins to unfold so i'm really looking forward to getting into this with you so let's jump right in so speaking to contour as a technology just to level set a bit contour is an ingress controller many of you have probably run things like nginx h a proxy traffic all these ingress controllers have different benefits one of the big notes about contour is that contour runs on voi as its proxy technology so when you look at the actual contour pod itself you're actually just running an envoy container and then you have the contour container inside of it which is the actual controller piece talking to the kubernetes api server understanding what ingress it needs to satisfy and then how to build a graph out of that so as a controller it's responsible for reading the ingress objects that if you use nginx or H a proxy you're probably quite used to and also the ingress route objects which is a custom resource definition that has a more capable feature set and a feature set that's a little bit more tailored to Envoy itself in certain respects although it's still very generic and we'll spend a lot of our time looking at why that C already exists as it builds those up it's gonna dive in and start building a what we call a directed a cyclical graph or as I'll refer to in this talk a dag and that just basically means that we're gonna build an object structure that represents the ingress needed for the cluster and then feed that into envoy as far as deploying this there's a couple different ways you can do it and there's reasons why you would do it in different ways some might deploy it as a daemon set perhaps tolerating a taint on certain nodes in the cluster so that you can select which nodes the envoy or excuse me the I guess really the envoy containers but the contour deployment spans across you could also do it as a deployment where you could kind of just drop it in the cluster ensure perhaps that it has an anti affinity that makes sure that it always is on different nodes but then you can set the replica count to ensure it's always a certain amount available and then as far as the networking model goes you might choose to use host port host network perhaps even a node port there's reasons why you might use these I won't get into that in this particular talk there's some around performance and how things would hop across namespace is how IP tables might be traversed based on which option you choose but nonetheless there's many many options at the end of the day you need this pot running in your cluster and with an ability for some type of external lb or even DNS record perhaps being able to reach this pod so that's the overall architecture of contourer now in this demo what I'm going to be showing you is a cluster that's running contour so if I look inside of here for just a moment I have my contour pods inside of a namespace called contour so in this case I'm just running one contour pod and this is going to start satisfying those different resources now like we talked about the resources we need to satisfy are things like the ingress objects and the ingress route objects so let's start off by talking about the ingress object since that's fairly important so I actually have an app that I'm going to use as a sample here and I'll show you this app right now we do get pods for the namespace ping in fact let's put a watch up here real quick so we can sort of watch this and I'm gonna actually make this a little bit better and also include ingress in this list so we're gonna look for both pods which is what Pio stands for and then ingress which is what ing stands for so we're watching for those alright now if I want to have my ping pong beasts accessible I'm going to use ingress to define how to route to it so we're just gonna look at this for a moment and I'll talk a bit what ingress is so inside of the ping app I have an ingress Auriemma file alright and if we look at this ingressing animal file here we can see overall its it's pretty simple right it defines some rules about if this host is being accessed so the request comes in with this host header and it's on this path in this case basically any path including route we're gonna route to a service called ping v1 now what's important to note here those of you who have used an aggressive before probably already know this but we use a service name because the service references or sits on top of the pods which as you know can be ephemeral and kind of stop and start under the hood right so what ingress controllers mostly do is they are going to actually route directly to the pods because if they just de route to the servers in most configurations with kubernetes that's going to go to the service and the service will always round-robin to the pods from there using IP tables or perhaps IP vs and so on but when we actually use the service we can say ok any pot behind ping v1 I'm going to route to those endpoints that way when we want to do more advanced load balancing algorithms like fancier things in round-robin maybe certain weighted algorithms and things of that nature we have the ability to ensure we can satisfy those algorithms based on the fact that we're routing directly to the pots ok so just something to kind of think about and if you're not really sure about that concept just yet it's it's more of an implementation detail and then we have a service port which is how we're actually going to go into the the the port to to access this if you will so I'm gonna go ahead and apply this for just a moment so if we do an apply on the ping I ping ok and then we'll watch up at the top here the change that occurs so I'll go ahead and apply that and then I'm going to get an ingress object inside of here so now theoretically the ping team in my company is really excited they should hypothetically be able to route to their object now which is which is pretty cool so how can we route to this object well we go back to the slides for just a moment we've just kind of applied this ingress object right so in grass again expresses l7 rules for getting traffic in and this is a example of what you just saw in my terminal these are namespace scoped objects so when I applied this ingress object into my cluster I put it in the ping namespace it's scoped in that namespace and it will be satisfied by an ingress controller so right now I'm obviously running contour but theoretically I could be running and Gen X I could be running H a proxy as long as there's not super specific implementation things that I'm expecting this should work across and then there's often times the need to think about the more advanced use cases but let's not talk about this last bullet just yet let's let's just look at the ping app and get an understanding of what it's doing so back to the terminal inside the terminal here I have ping running and I've actually set up a do loop inside of here so just to show you what it's doing let's make this a bit bigger so while true we are going to curl this host just like you saw on the ingress object we'll sleep for 0.3 seconds and then we will be done so let's go ahead and start that all right so as you probably could have guessed this app doesn't do anything too fancy it basically just continues to respond with a ping now my terminals not gonna do a great job of it but do note that this is constantly running it just fills up and then it doesn't seem like it's running because it's printing the same thing but we're gonna constantly run ping over and over again so overall the ping team is super happy their app is running great it's receiving traffic very very good now what about when we get into more advanced use cases with our ingress controllers or maybe we need some more advanced configuration of the ingress a--'s themselves and this is where things get a little bit complicated so the ingress API and kubernetes is really great really generic it handles your basic l7 stuff but then when you think about how do I do more advanced things like selecting an algorithm like I was telling you with with the notion of routing to endpoints right so what if I want to do not just a round-robin algorithm I wanted more advanced algorithm or what if I want to wait traffic in certain directions right well that becomes a little bit harder and what a lot of ingress controllers do and this is truth a true pattern for a lot of different things that provide some type of controller logic is usually they move that more specific logic into annotations themselves so myself and a co-worker or a colleague if you will worked on the first iteration of the alb ingress controller and the alb ingress controller for those of you familiar with Amazon was meant to take the Amazon layer seven alb and be able to feed based on ingress objects its configurations that you could have an al be that routes into different services now the models have been different because the alb lives outside of the cluster but a similar idea needs to create some type of graph based on the ingress objects and then feed that up to an Al B now when we got into the more advanced pieces that we had to consider we found ourselves many many times looking at how to put these items in and we fell on like a lot of controllers do putting them inside of annotations and this is what I like to refer to as annotation hell right so it worked fine and it's great that this gave us more configurability but one of the challenges is once you get into these more advanced use cases you get into scenarios like in the listen ports annotation where you end up putting JSON inside of an annotation inside of a Yambol file and it just continues and continues and you get more and more sprawl and this is nothing against the alb ingress controller it just genuinely is something that when you have a limited API or object and you need to feed more metadata into it you oftentimes find yourself adding these metadata like properties and then trying to parse and read them and they get really messy in and hard to reason with at certain points so an example of where this type of thing might come into play these more advanced use cases would be something like a canary use case so going back to the ping the ping group in our company real quick let's pretend for a moment that ping v1 was a smashing success it's been returning ping everybody loves it in the company and outside of the company but now they've decided you know what we're gonna introduce ping v2 and we're really excited about ping v2 ping v2 s big enhancement is that it's going to return Poong instead of ping now they're really nervous about this because this is a really important application and they don't just want to switch all the traffic over Bluegreen style and have pong start being returned they want to kind of wait the traffic across this now based on what we know about the ingress API there's no native way to do this without perhaps some type of annotation or maybe on the service itself we could put some type of annotation but nonetheless it's it's a little bit messy it'd be nice if in the ingress resource itself that contour is going to read we could define these types of waited rules so going back to our application for a moment here I have ping - up and running ok and one thing I'm going to kind of show you so you can conceptualize this a bit is there's also in contour the ability to visualize what it's dag has formed so I'm going to just run a quick command which you can you can check on the ingress the our sorry the contour Doc's on how to run this in the troubleshooting Doc's how to run this dot command I also have it on my website which will be the blog post will be in the description below but basically what this enables us to do is to see exactly what the contour graph is routing with so we can see inside of here that we have a listener it goes to ping octets comm when it receives traffic it always routes to the ping ping v1 application so very good we'll use this as a troubleshooting tool as we get deeper into this this setup all right so we've got the dot created so backing up a bit let's talk a little bit about making this weighted so we can sort of switch the traffic over so the first thing that I'm gonna do here is I'm going to take what I applied for ping and I'm gonna go ahead and delete that for a moment just so that it can cleanly go away now you'll notice in the buffer in my bottom right there's no more ping requests coming through so it's probably not why is that I've just deleted it but let's say we have an outage for for right now going back to the slides this gives us a good opportunity to talk about the ingress route CRD so rather than going through this model of having many many annotations to satisfy this one of the interesting things that contour created is a ingress route custom resource definition and this is something that we're hoping or at least I selfishly hope that the community as a whole might at some point kind of converge on how to do some type of semi canonical but also like pluggable specifics CRD for things like ingress so as you can see on the right here a lot of what's inside of the CR D doesn't look too different it's just called an ingress route however if you look inside the spec there are definitely some differences so you'll notice inside a virtual host the fully qualified domain name is how we express the hostname you also notice you have more specific fields and I might have some spacing issues here because it's in slides of course but you have a actually I don't think I do so you have a strategy where you can set the actual strategy for this ingress route in saying in this case weighted least requests you'll also know you can go into the routes and this is especially important the service mapping is not too different but notice we now have a weight property inside of here so with weighting you're probably starting to see where this is going you can wait to specific services which enables us to do that more specific weighted load balancing that we wanted to do for the ping application and then there's many many more things inside of this ingress route that we can do and we'll even explore in this video but this just gives you an idea of how you can have a more structured schema validated way to express advanced ingress rules be it's things like TLS pass-through for some of your apps so if you want the TLS to actually terminate on the pot itself perhaps some of these services you want to do TCP proxying - so you want to kind of go back on layer 4 and not rely on the layer 7 notion perhaps you have different balancing algorithms you want to use and weighted balancing as we talked about there's even a feature set that is going to be pretty critical to us in a moment around resource delegation and we'll talk a bit about how these ingress routes can be used to actually in a multi team or perhaps multi-tenant environment so let's go back to our ping app and explore this in a real example so inside of the ping app I have an ingress route resource which again does not look too dissimilar to our existing ingress still going to qualify ourselves on the ping pets comm and we're gonna start out with just a weight of a hundred so I want to start out with saying ping v1 you receive all the traffic and if there's a weight that is present inside of this ingress route and then some other service doesn't have a weight it's going to be assumed that that weight is zero so we can just bring ourselves back to our original functionality so let's go ahead and apply this particular ingress route so we will do a cute cuddle apply all right now inside of our top buffer here I'm gonna update us one more time and I'm gonna make sure that we're looking at ingress routes instead of just ingress so let's apply that here and now you'll notice our ping has started up again because obviously contour picked that up flipped it through the graph and you can also see some of the new information the ingress route which gives us a little bit more details not only the fqdn and perhaps TLS secret and route one of my favorites is the status so something's going terribly wrong we could oftentimes look at the status of that CRD and figure out why contour was unable to satisfy it all right cool and we might as well update the dot to just so we can kind of make sure that everything still looks good cool so aside from the border getting bigger we pretty much have the same thing going on but you'll notice now we are set up to potentially route to ping v2 so let's get some of the weighting in and see if we can solve for this hinari use case that we were talking about so I'm gonna shift 15% of traffic into ping v2 and then 85% of traffic into ping v1 let's do that now alright so going back to here let's go ahead and edit in fact you know if we just edit this directly it might make for a better example so I will do a cube cuddle edit and we will do this inside of the namespace ping great okay so it's not beautiful as far as colors go but that's okay so what do we want to do here well we've got a weight of a hundred four ping v1 we want to bring ping v1 to a weight of C here a weight of 85 cool all right and then for our next one here we are gonna do a weight of 15 like we had talked about so let's go ahead and save that up all right and now you can see inside of here we've got some instances of pong starting to come through so now you know if we took a large sample size of this we should start to see that roughly 15% of our requests are now routing to the v2 of the app which is pong really cool so we have a way to express these more advanced use cases like doing canary style deployments and then of course eventually our team is going to be extremely happy with how this went I think my editor is just set wrong so let's see if I export that maybe you'll be able to see it in color let's edit this there we go so you'll edit this one more time and then if we come back in here after this great success of pong we're gonna bring this to 10 we're gonna bring pong up to 90 save that one more time and then as you could have probably guessed we're mostly going to see pong now with a couple pings floating through every now and then as we saw with our dag as well we have this ping docked it's calm that is routing to these two services so a pretty cool setup and pretty easy to troubleshoot and reason with with what's going on here okay now I have another use case that I want to tackle with you all we talked a little bit about this notion of the ingress route having these delegation like capabilities so what exactly does ingress router even ingress delegation mean exactly so backing up to my company for a moment we're gonna erase this and we'll do a watch for the ingress route and pod in the namespace mount it's great okay so here's the deal I have a an app called mountains and I'm gonna go ahead and deploy that right now okay so in fact let me just kill this watch for a moment so we're gonna apply the mountains app fantastic so the mountains app is a service that our company offers that effectively provides a screenshot or I guess a picture of a mountain so that you can take that mountain landscape and use it as the background of your computer so as you're depressed throughout the day you can look at what the outside looks like and then perhaps be excited by nature right don't we all do that sometimes so so effectively we need a service just like anything else to satisfy this now what I'm gonna go ahead and do here is set up this example so let me just apply one more thing and I'll talk about what this thing is I'm applying in just a moment let's see here we'll do sorry we'll do a cute cuddle fly and I have this other file in here daemul good deal okay all right so I set some things up we're kind of backing up and refreshing here and let's pretend that our company is really excited about this new app that gives mountain landscapes as a service okay so we come back to our web browser here and we want to go to the mountains website so if I take a look at my ingress route for mountains I'll just back up one more time here and we go into mountains mountains let's see ingress route ins see what I've got a call I our mountains okay so we have an app here it routes into a service called mountains fantastic so assume I have that deployed and set up would you saw me do keep cuddle fly on that and for Mountain stock pets calm I should receive a beautiful picture of a mountain range great so we go back to our web browser here and we go to Mount tins octet comm and we go to get that background and boom we get a picture that we definitely worked quite expecting this does not look like a mountain range in fact if you're expecting a mountain range it would probably depress you quite a bit and no offense if you're a big fan of waiting in lines or going to the DMV so we think about this for a moment and we're kind of like all right so what's going on here why is our Mountain app clearly serving the wrong thing so we'll go back into our our terminal for a moment here and let's produce that dot output one more time I'm gonna refresh this and let's see what our company is currently routing so looking at the visualization of our graph we've got our listener coming in at 80 80 and that routes two lines and mountains now what's interesting about here is if you follow the path two mountains both that in lines route to the same service so apparently we have some type of conflict here and as the administrator who has an idea of some of the things running in your cluster you realize oh yeah we have another department in our company that instead of serving up mountain landscapes they serve up pictures of people waiting in lines right and obviously the audience between these two things is vastly different but for some reason the lines come a group or team was just able to take over the mountains fully qualified domain name and just route to that so let's see if we can fix this real quick so we have a mountains a mountains yeah Mille so we'll do mountains like we looked at so mountains dot yeah Mille so maybe we just didn't apply this right so let's see if we could apply this and then get our mountains service back so we will go ahead and do another watch on mountains okay so I will do an apply here for mountains IR Mountains dot animal and it will create but and this is where status is gonna help us out a bit it's a bit wrapped here if you look closely it's letting us know in status hey this is invalid you're fully qualified domain name Mountains is used by multiple ingress routes okay so you can probably get an idea and you probably could even from this graph what our issue is here right we have a system in place we're theoretically someone can just hop in and ask for a fully qualified domain name and contour will respect it in this case lines is effectively taking over the fully qualified domain name that should be owned by the mountains group so what does this effectively mean and why is this an important thing to consider what a lot of folks get into setting up their first kubernetes cluster things go clusters things go really well at first you know they have an initial app but then in some cases where they're not just doing a cluster per application or team in some cases they start getting down the road of multi team clusters or even more multi-tenant type clusters this is where things get pretty tricky because now you don't just have to solve for how do I get traffic into my cluster but you also have to solve for how do I ensure a group that might be the lines Department only has the ability to create certain things that the mountain's group shouldn't be able to create and same with reverse how do I make sure the mountains group can't create things that only the lines group should be able to create so effectively we need to think about this in terms of delegation of resources so in this case if you look at the diagram here we've got some type of ingress load balancer it could be nginx H a proxy contour traffic and as we're deploying these ingress objects you have the same scenario you have team a requesting host team a and Team B accidentally request requesting team a as well even though the destination are two different places so the question is you know what's gonna happen here is it gonna be a deterministic behavior will it just fail all together because the same thing was asked for twice what would we expect to occur and the answer is it's gonna depend on your ingress load balancer choice what what controller did you choose engine acts DJ proxy contour that's going to impact what the behavior is but nonetheless we really just don't want this to be possible so how can we protect against these types of invalid host configurations or just invalid ingress configurations one approach that I've seen in a couple different environments actually think it's a suit for valid approach if you want to go down this route especially if you don't have something as advanced as an ingress route CRD is a lot of folks will create a validating webhook admission controller and these admission controllers if you haven't looked at them are just little services you can run in the kubernetes cluster oftentimes but perhaps outside to some degree but you could run them in the cluster and they're just like a little web service that when the API server gets a request if you have a validating webhook admission controller hooked up it can take that request shoot it over to that admission controller the admission controller could run some rules against it or just logic and come back to the API server and say hey the lines team that's requesting this they should not be asking for this fqdn deny this request and this hasn't become a really popular pattern with a lot of things and even the project which you may have used inside or outside of kubernetes called open policy agent was designed to provide this types of rules engine and allow you to kind of standardize on a language or a syntax if you will I guess a DSL really that's going to provide you with these types of rules so that's one way you can do it and can be completely valid another way is you could think about this in terms of route delegation so what if our ingress route c RDS were powerful enough such that we could have kind of like an administrator name space where we could provide the delegation in one place for which routes folks are allowed to start creating ingress resources in that seems like a pretty interesting approach and that's effectively what I want to show you here so let's take a look at this if we back over to our back up over to our terminal for a moment first let's see if we can sort of fix the the mountains example here by using some some delegation like things so I'm gonna back up out of actually I'll keep this running for just a moment I'm gonna go to the you know what let's let's watch the lines app for a moment so inside of lines you can see the second ingress route that I was requesting to create called mountains and that's where you see this invalid fqdn here so first let's rip this out because obviously it's it's not supposed to be this way in the first place so I'm gonna do a cube cuddle delete for the lines app and this will be ingress trout lines you know it actually need to delete the specific ones so let's go ahead and delete that so I intentionally put two in restaurants in one file just to show this example of it messing up so I'm going to delete in the namespace lines this ingress trout and then that will go away now if we come back to Mountains for a moment here now we can see because of the beauty of CRD statuses that this is valid and if we produce our dot output one more time we can also see in this diagram that now we have you look towards the top it's shifted around a bit we now have a route through mountains that goes to the mountains service itself which on a conceptual level this looks a lot more valid for what we need okay so if I come back to mountains doc Ted's calm and refresh now we have a much better looking picture of Indian Peaks wilderness which is a place in Colorado if you've never been it's it's really beautiful on the way up to Pawnee pass and if I go to lines just to make sure that we didn't break anything with the lines group because serving their background pictures is very important too we can still produce pictures of people waiting in lines if that's if that's more of your thing so great but obviously we haven't solved the problem for how did we get here in the first place and and that's the crux of what we want to solve here so if we back up a little bit for just a moment I want to introduce the notion of a namespace where I can provide this level of delegation now here's what this is effectively going to look like on a conceptual level and then we're going to look at it in the terminal so I'm gonna introduce a root namespace okay so the root namespace is going to be what I'm gonna call ingress system in my in my cluster here and inside of ingress system I'm gonna have these delegation objects and these delegation objects are gonna allow me to say things like for the delegation Mountains I'm gonna give the fully qualified domain name Mountains dog pets comm only to the mountains namespace and same goes for lines lines dot octet Co only to the lines namespace so people in the mountains namespace all they have to worry about is creating their routes and might create one for Mount Evans for Mount Rainer or in the lines namespace same thing you create one for the DMV line but when somebody inside of the lines namespace creates another fully qualified domain name for mountains it's gonna blow back and say hey you don't have the ability to use this this is not something that you've been delegated which is exactly where we want to be now this is going to require a flag and contour so we can tell contour which namespace to look to to provide this level of service now I'm not even sure if this flag is correct but I'll show you in a moment in the actually mo what the actual flag name is but nonetheless there's a flag that allows you to specify this namespace alright so let's take a look at this it should be pretty simple so first thing that we're gonna want to do to make all of this work is obviously we're gonna want to fix up contour so I am going to put a watch on the cube the pods inside of contour so there's our contour right there now another thing I'm gonna do in this bottom buffer if I walk in and go to the manifests for contour original in fact let's do maybe let's do a little dip here so if we do a diff on manna-fest contour original and then contour patched you'll get to see this change that I'm gonna make firsthand so outside of a couple little spacing dips that I messed up you can see inside of here on the left is our original contour and then on the right is going to be our new contour so here's the actual name of that flag it's ingress route root namespaces and we're gonna set that equal to in grass - system so just like the diagram implied with this flag in place contour is going to look to this namespace to figure out whether the ingress resources that are being requested are valid have they been delegated to those namespaces so overall that looks pretty good so we are going to go ahead and apply this one so let's get that in so cube cut' I'll apply great and we'll watch up here because this should force Khan to to stop and start with a new pod so we'll give this just a moment okay all right so as we have contour pending here in fact you know I wonder if now that I think about it let's see real quick how many replicas I asked for cuz I only got a single one that is going to work here so let me let me fix up these replicas real quick so contour patched okay there we go looking a little bit better so long story short I had a small amount of VMs to run contour on so I messed up that configuration and in fact another thing I messed up were my versions so bear with me as a lot of people say don't just use master all willy-nilly which I'm using master here so I'm supposed to be on leave one dot 10.0 so let me just check that real quick so image V zero sorry not one dot 10.0 but 0.10.0 so let's see here we'll paste that in alright sleep that and let's apply it one more time cool so if you are looking for these animals these are also in the blog post that describes these steps - feel free to check that out for the right ones I've just been mucking around and playing around so it broke some of these pieces so contour should initialize in a moment here and two out of two should become healthy if all goes well there we go and like we said contour is now serving in such a way that these delegations need to work so if we go back to our mountains and hit refresh our piece is broken here so I'm still getting contact with envoy but clearly contour is not able to give it its information anymore and if we go back and make another dot output here and do a refresh we no longer have beautiful boxes so clearly we've broken something which is an expected behavior so let's go back and do a little bit of cleanup to start off so if we watch the namespace so let's do a watch fur I think we did mountains last so let's see if we can do that again yeah we'll do lines just to start out so alright so we have lines here you can see now root ingress trout cannot be defined in this me space so it's actually working as we expected it's letting us know hey you're not actually you're trying to like redefine a route you're not using what you've been delegated it's clearly breaking right so if we come back to our terminal and do a cube cuddle delete and we delete this particular object so we will delete the lines object in the name space lines make that go away and then let's do the same clean up for mountains here so inside of mountains we've got the same error we are going to take mountains and delete that as well all right so oops would be helpful if the name space was correct so mountains great all right so at least now we're starting fresh things obviously still won't be working because we don't have anything created or set up so what I'm gonna do here is I'm going to make sure that I have my ingress system namespace instantiated so inside of ingress system basically what I've got going on here is I'm creating the ingress system and then I'm setting up to delegation objects in here as well so let's put a watch on the ingress system here for a moment all right good deal so I have got a delegation setup for ingress route and that's this here and another piece right here now what are these delegation objects exactly let's take a look so taking a step deeper and actually sorry I'm gonna show you it in the UML form here so if we go to manifests and we go to the ingress system for a moment here so obviously I've got the namespace which is effectively what I showed you a moment ago and then I've also got the delegations now the delegations are just ingress routes they're not necessarily anything special or a different object that we need to worry about you'll notice the big difference between these ingress routes is they specify a fully qualified domain name they specify routes just like any other resource would or ingress route would but particularly they have this delegate or delegate attribute inside of here which is effectively achieving what we on the slide so we're gonna say this fully qualified domain name is being delegated to this namespace and we could even give it specific paths if we don't just want to give it route and everything on but for now it's just basically showing off the fqdn then four lines octet comm we have a very similar model in place we have lines octet s-- we have the /d M V so in this case we're gonna provide it only with the DMV path because for some reason we want to reserve the route and we're going to give that to the lines a namespace so now we've got that set up so let's see if we can bring our dag back in order so that it's working again and actually route it now what will the actual ingress routes look like so let's start off with getting mountains back up since that one is rightfully our most popular service so we'll do a watch on these mountains so this will be for mountains all right and in this case I'm going to show you the slightly different ingress route mantains delegated resource so not much different still an ingress route but you'll notice in this case we're not gonna specialist best we're not gonna specify the virtual host specifically the the fully qualified domain name because that's no longer relevant to us we have that delegated and we are going to be able to set ourselves up with just this path and that is gonna wrap to the mountains service inside of our cluster so with the simplified model not having to worry about FPN is quite nice if we come in and instead of editing it we apply it so apply mountains will now create that and it looks like our status is valid again so based on the delegation it looks like we're running this let's start with the dot output which is gonna give us a looks like a rule that looks a little bit reasonable here so mountains going to mountains that's good we'll come back to our browser and refresh and boom we have mountains again which is which is looking really solid all right so let's do the same thing for lines just making sure that we can get set up here so similar idea with lines we are going to go into the ingress throughout four lines will do I our lines delegated all right now obviously lines is a bit of a different beast because it has a specific path that it's allowed to use so we'll wash lines for a moment and just for the sake of testing this out let's assume that lines wants to get back on route again so just lines route namespace all that good stuff so if we do an apply on lines it will now create and just like any good status should it'll let us know hey you're using what you wanted to delegate to but you're asking or what you were delegated but you're asking for /or route and you only have access to the path prefix forward slash DMV which is exactly what we'd expect to happen based on the delegation so let's edit that one more time and we will do something specific will do DMV line one as the resource and we will apply that again and now if we come back to octet calm and do lines we still have an issue let's see valid oh sorry of course that would fail right cuz we just said it should be line DMV line one and there's our line so based on delegation we have successfully been able to set up mountains to limit itself to that fqdn and lines to limit itself to that fqdn as well so again another powerful thing that we can introduce by having this in restaurants CRD in place okay so the last thing i want to talk about is handling TLS which is oftentimes an important use case for ingress controllers a common pattern that a lot of ingress controllers follow is they support the ability for you add your TLS secrets into your namespace or perhaps another namespace and then reference that secret from your ingress object or an aggressor out object a lot of ingress controllers also support the ability to put a default certificate in place this is usually a wildcard certificate that companies are using kind of as a fallback so that if you don't specify your own TLS it will use some company wide wildcard certificate now there are some problems with this model one is the default catch-all is often too much of a catch-all for larger orcs sometimes you want to ensure that only certain groups can use the wildcards server can use subsets of wildcard certs and then also you might not want people who are adding their own TLS certificate in to be able to go in and add their secret and their certificate to their namespace because with improper our back or some type of mistake it might actually expose TLS secrets that you don't want out there so we're gonna examine how to add TLS to contour which is nothing special but then we're gonna talk about how we can use some features to actually bridge the gap between this multi-team wildcard default search type use case we want to be a little bit more restrictive so going back to our application here I've refreshed it a bit I've turned off delegation for routes just to keep things simple and I have a setup that's pretty similar to what we started with I've taken the the lines app out to for simplicity so mountains goes in it routes to mountains 8,000 simple as that now what I've gone ahead and done is I've added a ingress or sorry a secret to the the mountains namespace and I can show you that secret right here so this secret is a TLS cert and a TLS key that are base64 encoded so obviously you'd wanna do more than this if you're worried about people seeing this key and you have it in the namespace of mountains and it's called octets TLS so if we edit our ingress route for mountains and inside a virtual host we add in TLS and then the field secret name we can now come in and specify the octet TLS secret which we just looked at so let's do off test TLS let's save that up and in a moment we'll see that under the TLS secret column we now can see octet TLS and if we do the dot output we will refresh and let's see here so octet TLS is the name so make sure we have this called octet TLS that's TLS in the mountians namespace make sure that I had that applied because perhaps I didn't create it okay that would be very helpful so now it exists which is super important and if we refresh this we can now see ourselves going from mountains to the secret and that will route over here and we also can do things like allow for insecure routing on still HTTP for the service but those are all things you can look up in the docs what this means is if i refresh you can now see I'm on HTTP my certificates kind of silly but it's just for the sake of example I have star octets com so this is actually a wild card cert and I would like to be able to delegate it to certain teams so what could this possibly look like well if we go back to our slides we could implement a model of TLS delegation pretty similar to how we talked about route delegation the difference here is we're gonna have a TLS certificate delegation object and that's going to reference the secret you just saw and it's going to allow us to put a list of namespaces that we effectively want to allow for this wildcard serve to be resolved in so you can see I'm allowing for mountains and trails which means that when people make apps in the or ingress objects in the mountains namespace they're going to be able to resolve this TLS and then people make it in the lines namespace it's gonna push back and say no you can't do this so this is actually something that you can currently do in contour zero ten plus this slides a little bit outdated but nonetheless you'll be able to see this right here and and in dive in and test it yourself so let's try to move this model where we hold the Stila secrets in this administrative ingress system namespace so first things first let's go back to here and that TLS secret just to make sure we're not cheating let's go ahead and delete that real quick okay and in our watch up top let's set up another watch in the ingress system namespace now in this case I'm going to be looking for secrets in TLS delegation which I've already uploaded some of them here but I'll show you them so if we edit that manifest wildcard TLS and we put this inside of the namespace ingress system so we're uploading that same certificate and key so we will go ahead and apply that all right so remains unchanged just to make sure nothing has changed here so that's effectively what we've got here called octets TLS and then we've got this certificate delegation object so let's look at this as well so I have this in manifest TLS delegation and this looks just like what we saw in the slides we have the particular secret name that we're referencing and then we have the target namespaces mountains and trails that we are going to allow to utilize this secret so we'll save that all right and now we're looking good so if we go back and watch the mountains namespace you can see here let's go ahead and just put our other watch back in real quick so that looks a little better cool so you'll notice that mountains is still saying it's valid which it is and it's referencing this TLS secret but if we come in and produce the dot output again and then I do a quick refresh on this you'll notice that the graph no longer looks quite right in fact you'll notice the HTTP path is completely gone we are only routing to the HTTP service through normal HTTP now this is actually expected because if you think about it our ingress route now needs to reference the octet TLS that exists in the ingress system namespace so that's a bit of a change for us so if we edit this one more time okay and in secret name we instead put in the ingress secret namespace so now we're kind of qualifying that I'm sorry not secret but ingress system namespace we'll save that secret up or that change up really in a moment it will refresh again the beauty of the see RDS we can now see that we're referencing this particular TLS object and if we go in and reproduce our dot we will refresh and there we have it so now that the secret exists this is an important thing of kind of how the dag is created right so it's not gonna actually make that route or that part of the dag unless it can satisfy these things you're referencing so now it can and if we go back to our mountains app and refresh it's business as usual we now have mountains in here as well now if we go back and hypothetically apply the lines so let's actually look at the lines real quick cuz I haven't looked at this one in a little while so we'll do ingress or outlines I guess just dot yeah well since I don't have the delegation set up okay cool let's let's test this out real quick so we'll get rid of the mountains because I was just using that for testing cool so we're gonna add TLS in and then we'll put in a secret name okay in this secret name we want to kind be the same thing right we want to be able to reference this same ingress system secret so if I watch this for oops if I watch this for lines now we've got this in place looks good and we will save this and apply it so cute cuddle apply huh good deal and it should show up in here alright now one of the things that it has done is its reference to TLS secret now what will be good eventually and I'm sure this is something that we'll have in a future release hopefully or at least something we'll open a ticket for is the status probably shouldn't say valid I wouldn't think offhand but if we generate the dag for a moment so we will run the dot I will open this up and refresh you'll now see that lines octet has been created on the HTTP path but notice the dag is not referencing it on HTTPS and this is for reasons you could probably guess we have not delegated the ability to use this secret so contour is not going to satisfy that if we go back to our application here and I do a line stock pets like we used to it still works but if I do an HTTP on line not octets it's not going to work and again that's the beauty of this delegation we're now hosting our secrets inside of this administrative namespace we're putting our delegation rule in an object and we're ensuring that lines is unable to do this so you can see kind of the power of this delegation model and that's about it so hopefully this gives you kind of some good insight around the ingress route CRD and some of the things I think you'll see certain ingress controllers start doing like traffic not too long ago released in restaurant CRD as well that I'm haven't looked at it but I'm assuming it does a lot of similar things in regards to advanced functionality if you're interested in contour check out the project on github if you want to get an even more in-depth or at least a different perspective on some of these things Joe Beda has put together a TGI kei episode and the link is right here really cool walkthrough of some of the awesome things for contour and I hope you found this useful if you did my Twitter handle is in the bottom left and you're also welcome to subscribe to this video and check out my blog in the description I'll be producing hopefully a lot more kubernetes and linux-based content very soon so thanks again for your time and I hope you found this useful

Channel: octetz

Views: 2,258

Keywords: kubernetes, octetz, envoy, contour, cloud native

Id: O7HfkgzD7Z0

Length: 51min 39sec (3099 seconds)

Published: Fri Apr 12 2019