Consul Service Mesh: Deep Dive

there's a lot of you folks in here you really come to see that many people want to see my demo fail I know I know you're out for blood look at this matches my shirt now if only I'd spend enough time on my demos I haven't making sure that much my shirt all right so let's go Console service match is is a tool that I absolutely love and I'm gonna tell you why and the reason that I love console Service matches because it's a tool that I wish I had when I was developing systems the last kind of big system that I developed I was working for an e-commerce company in London and we did the whole migration thing we did the migration from monolith it's micro-services about five years ago and it took a long time it took us about 18 months to get the tooling in place to get the process in place to change code to build schedulers to move to cloud and the date came Saturday 9:00 p.m. sitting in the office eating some pizza now I want to caveat this by this kind of was headed to go wrong because somebody ordered anchovy pizza nobody should have to suffer that but we were really excited and we'd planned that we had to do this downtime 12 o'clock was the time we were going to put the site into maintenance mode and then we were gonna do this migration now twelve o'clock came for the site in the maintenance mode everything started a roll smoothly until about 2:00 a.m. and about 2:00 a.m. once we drunk about five cases of redbull we realized that the migration just wasn't gonna happen we had complexities we could not get the system switched over so we had to make the call to roll back and that was sad and the whole team was really easy ejected by having do that but we wanted to try again and try again we did five days later in the office 9:00 a.m. know unsure of a pizza this time guaranteed success any was going good going really good 2 a.m. it came we'd got past the point that we were at before the system was looking good for am still going good 6 a.m. should we make the No Go Go no-go decision yes turn off the maintenance page 7 a.m. getting into the car phone rings CTO sites down like ok I've not slept for 24 hours I can deal with this back up to the office everybody gone eventually we figured out what was wrong was it the deployment was that the new cloud was it the new system no we tested it we tested it again we had infrastructure testing we had all the tests and it still wasn't working because maybe just maybe when we were testing we were using the development credentials for the payment gateway on the production system maybe but kind of where I'm driving at is that we had to do that Big Bang approach because the tooling that we have at the time necessitated that and where we are now we've got wonderful tools which allow us to take very very different approaches and be more graceful in the way that we we develop and we operate our systems give us extra security safety and I love some of these features because I don't want any of you to have to suffer the pain that I did at that point in time so I'm not just gonna talk about that monolith migration I've got a few patterns that I want to run through with you we're gonna look at some of the last not screaming new features a console service match I'm gonna give you a quick run through of some basic configuration just in case anybody didn't read the caveat and the paragraph from my abstract we've said I'm not gonna go into any basic configuration I'm gonna look at some of the traffic routing patterns and I want to kind of apply that to sort of monolith to microservice it's just gonna try and put some patterns into this we're gonna look at traffic resolver and we're gonna look at traffic splitting Multi cluster gateways which I love and service available so let's begin so with console you're very familiar with this configuration the kind of the service stands has been the the mainstay of how you kind of register services with the the console client and nothing much has changed there except that we we added these new blocks which allow you to configure the service proxy and you can do things in there like configuring the port that you want the data plane to run on you can do things like configuring Europe streams but what we introduced with I think console 1.5 is centralized configuration so where is the service configuration is specific to the service instance central configuration applies to all of the services which are defined in this instance as web so here's a simple example of some service defaults I'm defining the protocol of my web service as HTTP if you want to do this as Jason you can also define that same configuration file in Jay as Jason we tried very hard to be able to give you that configuration option and define it as Yama what we just couldn't do it maybe next version alrighty those configuration entries though we've got the proxy defaults so those are global defaults which you can apply to all instances of your proxy things like your metrics configuration which metrics output format you want to use service defaults protocol a couple of other things we're gonna look at service Rooter how do we kind of configure layer 7 traffic routing service splitting capability of doing things like canary deployments and in weighted traffic to different service instances and service resolution service resolver kind of think of that like virtual services in some way and I'll probably get hammered for saying that the way that these work is they work in a very specific order now you don't have to apply all of the things in the chain you can use a service route on it so you can use a service resolver on it so when you're using the centralized configuration it's important to remember that they are applied in certain order so we do routing then splitting then resolution so let's dive into our first example so this is where we are we've got this system here we've got a monolithic web app and we've got a monolithic payments app now we want to modernize but we don't want to do the big bang we want to do a graceful modernization what I want to get to is this situation I want to be able to break out the currency part of my payment service and put it into its own isolated service and I want to do that without reconfiguring or recoding any of the instances in my web application and I can do that using service routing so I'm going to define a service Rooter and it's going to look a little like this so what we have are the capabilities to add various different routes so here I am going to define a very simple HTTP route a path currency is going to go to my new currency service and any other path is just going to go to my payment service when it kind of goes through that resolution chain the service route or inside a console is going to automatically choose the correct upstream you don't need to configure anything changing in your your web application console is going to do all of that for you just by setting up both central conflicts so let's give it a look so what we got so let's first let me just start my example so I've built a little examples for you and I've built all of this stuff in the lowest kind of common denominator so I've built it all just in docker compose so you can take the repo play with it on your own and hopefully it'll be useful examples for you so we've got console up and running in a second there we go and we've got our currency service we've got our payment service and we've got our sidecar proxies so they'll go healthy in just a second once the health checks kick in well what we want to start doing is enabling that service routing so let's have a look is that healthy come on don't make me hang around right while it's wet starting so the configuration that we're going to apply is this rue de so exactly what I showed you on the slide before now we have to also look at the service defaults because what we need to do we're using l7 routing we need to configure our services to be l7 so I'm going to use protocol HTTP for those things so while I'm waiting for that how do I apply central configuration so I can apply central configuration in a number of different ways - well two different ways really one is I can use the command line I can use the console config write with a file and if I could type and the other way is that I can also use the the API so if you taken an infrastructure as code approach which you should be use the API just to be able to kind of as the opposite of right you've also got read so I can do console config list kind service defaults you can see I've got one web contract loaded in my system and I can read that by console config read kind service defaults and and web so that's just coming out the other side there so that's all healthy so where we're at so far if I curl my service what's happening is my web service is calling the upstream and you can see there that it's hitting payments v1 now if I curl my service with my desired path of being able to have currency GDP I'm still hitting the payment service because I haven't applied any of my configuration yet so let's go ahead and do that console config right currency defaults payment defaults and then the Rue de so this file here okay so now let me tell my service route path going straight to the the old monolithic service as I want it if I curl that with my new currency path you can see that I'm now hitting the currency service so in a my upstream configuration for my web service you can see there that I've just defined payments so the web service isn't completely unaware that the service currency exists what I'm doing is I'm rerouting that traffic using a centralized config which i think is incredibly neat next example so traffic resolving so let's take a look at how we can use the traffic resolver why would we want to use traffic resolver well I think a be testing is a pretty neat example this is what we have this is our current state but you know we still doing a bit of a kind of a risky approach we're not really certain that our current system works right I mean my code works right anyway yours but wouldn't it be nice to be able to do something like this where I can actually have both of those things in existence and I can route traffic to a very specific subgroup for my new system that way I can gradually kind of test it I can try out the new features juice maybe testing maybe some doctor ploys and again back onto the discovery chain what I'm now gonna do is I'm gonna introduce this new component which is the service resolver and I'm gonna use it with the service Rooter and with a service resolver what I can do is I can define things like subsets so I'm defining here to service subsets one for v1 and one for V 2 so my v1 service is going to be selected if it's been tagged inside a console with metadata version is equals 1 my v2 service metadata version 2 an important thing to note there is the default subset so with the default subset all of my traffic will route to my v1 subset in the absence of any overrides and that's a very useful thing with my router and we looked at a kind of example of how to HTTP routing on paths I can also do HTTP routing using HTTP headers so in the instance that the HD we had our test group with a value of B is present all of my traffic will route to the v2 subset instance if it's not present I'm going to be using that default subset it's just kind of cool this is the way it's gonna work so I'm gonna set up this system payment or such magical service console comm something if we go the route without the header we're gonna hit the version 1 of our system with the header we're gonna hit our version 2 so we can do that slow gradual transition let's take a look at it first things first let me kill my old demo and start my new one all righty so my configuration very very similar again I'm using that centralized configuration I'm defining my services because I want to be using l7 they've got to be protocol away so I'm H GDP can use GE RPC as well but service defaults my payment service my resolver we looked at that in the slide I've got that default subset which means in the absence of a subset it will always root to the version one version one is defined by the meta data stored in console I'm only using service metadata here but you can actually use any of the filters which are capable on the the health checks and finally my ruder come on console alrighty so you can see my setup there got my payment service payment iv1 in ID and v2 ID v1 ID metadata version 1 v2 metadata version 2 now in the absence of any centralized configuration if I curl those endpoints what I'm gonna get is just standard load balancing so curl it once you can see I'm hitting b2 this time I'm hitting v1 because I've got no centralized configuration at present so let's apply that so console config right payment payment default setting up at l7 configuration currency defaults my web defaults my service resolver and finally my service Rooter so if I curl my endpoint what I'm gonna get is my v1 my service only might be one of my service and that's because of that service defaults now when I apply the header to that what I should see is hit the v2 service so let's add that header right to a local host adding a header test group B forefinger mistake and now we're go ahead in the v2 service so we've we've taken that Big Bang approach we've deconstructed it we've got the ability to run the old and the new system and now we've got the ability to be able to subset and route only very very specific traffic through to our new version gracefully doing that rollout which is kind of neat so what's next surely that's enough that's pretty incredible no you don't like it [Applause] I mean I'll be honest I wrote none of this but I'll take all the credit let's take a look at our next example this is gonna be one of those talks because I've got so many examples you know like a lot of talks anybody watch them at one-and-a-half speed you want to watch this one at half speed traffic splitting so it's all well and good being able to do that kind of that a B type test but you know one of the nice things about modern sort of software practices is we can do these graceful automated rollouts and being able to slowly roll out a version of a new software and to be able to have that have only a very small fraction of traffic while you're checking that it's it's ok because remember your code doesn't work mine does but it's very very kind of some ways it's a bit tricky right but console Connect you can use the traffic splitting level-7 feature and our desired state is that we initially want both of our versions of our software running on version one service and our version two but we want to send a hundred percent of the traffic to version one then we want to be able to test the water let's just check that the version 2 version of the software is behaving as we've expected we can do that by checking the various different metrics and tracing and and other sort of business metrics and observability what we're going to do that gradually once we're confident we're going to increase that maybe do a 50/50 and then eventually we can roll that out so that a hundred percent of all of the traffic is version two and we're gonna then deprecated the version one and we're doing it safely with a minimum of impact and disruption to our customers from a discovery chain perspective we're now going to use these final two elements we're gonna use the service splitter and we're going to use the service resolver service resolver very much exactly the same as what we used in our a/b we're defining the two subsets with the splitter what we can do is define multiple splits so here I've got two and I've got two weights 50/50 so the first split 50% of traffic is going to go through two service subsets v1 50% two service subsets v2 you can put whatever values you want in there as long as the total of all of the splits equals a hundred percentage when the traffic kinda comes in and again console is going to change the sort of the way that things are load balanced to the upstream based on those weightings so I don't need to make any changes or any behavioral changes to my application this is all just gonna happen for me by doing this configuration through my central configurations let's take a look so here we go again let me just start my demo since starting my services now what I'm doing is I'm kind of putting things into my my initial condition so I'm only gonna have my version one service running because I want to be able to configure my service platter on my service resolver before I deploy my version to service so we've got v1 of our payments running here and what we're going to do is we're going to configure our service resolver to be able to define those subsets and we're going to set up our service splitters so first things first let's get that configuration room console config right set up the currency default again my payment service defaults my payment service ruder resolver sorry and my web service defaults so if I kill my service what we're gonna see is we're hitting v1 because we've only got our version one of our service running but now let's deploy version 2 of our service whoo I shouldn't have done that this was going way too well wasn't it you wanted blood you got it thank you all let's just wait for this to start up so we've now got our v1 and of 82 services running because we've got that centralized configuration already applied when we curl our service we should only hit the v1 service so we're only gonna hit v1 so I'm pretty brave and I'm just gonna go straight to 50/50 because I like to Yolo things so let's apply that splitter configuration again console config right payment service helped I was in the right folder console config right payment service splitter 50/50 now curling my endpoint you can see that I'm getting flip-flopping between V 1 and V 2 50% of my traffic go on a 150 percent going to 2 so what if I change that well I'm pretty confident now everything's working great I'm gonna send 5% V 1 I'm gonna send 95% to V 2 again I'm just going to apply that configuration all of these changes that you're making through Center for configuration is going to be impacted with zero downtime because of the way that envoy works envoy is gonna hot reload any of the configuration that we're passing down to it it's not going to cause any impact to your existing traffic 5050 let's curl that so now we're pretty much hitting v2 all of the time this is gonna like just call me a liar but you know if I do this long enough it will hit v1 eventually got anywhere to go to after here you can trust me that that works I want to move on because I want to show you something even cooler so we've done a be testing we've done traffic splitting we've looked at service resolution and canary deployments what if I could do canary deployments with a/b testing that would be pretty neat because then it can be even safer I can ensure that my test group only gets and a select amount of my new service whilst I'm rolling out any updates so Mike my kind of situation is that I want to be able to do this if somebody is in the test group that is they've got the hey gdb had a task group equals B then they're gonna hit the traffic splitter anybody who doesn't is just going to hit the v1 version of the service now I'm actually using all of the elements of the discovery chain so I'm using the route a detect that there's an HTTP header and then to send it off to a subset or a traffic splitter and then to the service resolver to be able to determine which version of my service to select which is kind of awesome service Rooter it's exactly like we saw in the a/b test I'm just going to be using test group with the exception I'm defining a route path now and I'm explicitly gonna send that to my service subset v1 all the things into action but it's an incredibly graceful way of being able to introduce new versions of software into your environment so let's not do that and see how that works so I've got my Rooter all my configuration is already applied so I'm currently running my service resolver and my service splitter so let's apply the Rooter console config right payment service Rudo now if I curl localhost I'm hitting just my v1 version of my service because I'm not in my my test group if I'm in my test group that I've got that header just Group B then you can see that I'm gonna hit the different version version too and just to prove that that is actually traffic splitting I'm gonna switch this back to 50/50 and I'm going to apply that so console config right fifty-fifty localhost still hitting v1 test group there we go see I wasn't lying to you I wouldn't do that so where do we go next well where we go next is multi cluster gateways and one of the really really impressive features is the ability that you can now route traffic between different service clusters so I have my web service and if my web service needs to talk to my payment service which is my PCI protected cluster consoles got the capability that that traffic can be transparently routed regardless of what overlapping networks you may have regardless of any sort of conflict you don't need flat networks console gateways are gonna work and the way that it works is that the web service resolves the payment service to the Gateway the Gateway is aware that the payment service resides in a different cluster so it forwards the request to the dc2 gateway the dc2 gateway then forwards it on to the payment service at no point you need to kind of configure anything specific inside of your your service up streams or anything like that it just does it and it does it securely because the gateways are just transparently proxying the requests they're using S&I headers which give them the ability to detect where the destination is without having to decrypt any of the the protocol all the packets and it's just awesome we need to make a couple of small configuration changes so we we need to be able to Confederate our two data centers so I've got data center one primary data centers DC one I'm setting up my advertised Wan address so that it can talk gossip across the wine and I'm enabling central configuration the to specifying that SDC two primary data center dc1 the advertised address but just it's local IP it's running in the dot six the advertised address one which is the one connecting the two data centers and the retry drawing which is going to allow it to automatically join the other data center Fedder a tall of its configuration security which is required for identity authorization and intentions configuration for that is pretty simple again we can use those service defaults mesh gateway mode local mode remote so what's the difference between local and remote with local I'm always gonna hit my local gateway forward to the remote gateway and then to the destination with remote I'm going to skip the local gateway and I'm just gonna go direct to the remote gateway so I'm saving a hawk you've got to ask yourself is that at the expense of your ingress security fear or the data center so there's there's two ways you can set this up and let's take a look at how this works because I almost as bad at Clint as Clint for timekeeping so we're going to run this all right you of course it has to happen when I'm under pressure doesn't it all right while that's running so what am I going to be doing in my configuration well the configuration differences that I need in order to be able to route that traffic is pretty simple I'm going to define again in my service resolver and I'm going to use this time a redirect stanza block so I'm giving a hint that when I want to resolve payments it exists in data center DC - just to make things interesting I've actually got my currency service inside of DC one so payments talks to currency which is in a different data center and web talks to payments which is in a different data center because that's how I roll looking at the console UI I've got my services there you can see that I've got my federated my federated system got my payment side car running in data center - and I can go ahead and I can apply my configuration so console config right my payments defaults my web service defaults my currency defaults oops the two resolvers because again payments ridgelines in DC to needs to talk currency in DC one web and DC one needs to talk to DC two currency resolver and my payments resolver so now curl localhost 1990 and what you can see is happening is that the there's an upstream for web it's actually heading services inside of DC - so payments running in DC - and payments running in DC - is hitting a service and resolving to DC one and all I've had to do is set up that configuration it's an incredibly useful feature incredibly powerful I really love this thing I could just go on and I usually do but I can't so just kind of quickly recap some of the features that we've introduced you've got the ability to do traffic routing one of the example patents maybe monolith the microservices a be testing use the traffic resolver traffic splitting the ability to do those canary deploys doctor ploys multi cluster gateways incredibly powerful route your traffic between kubernetes virtual machines virtual machines kubernetes bare metal like you name it you can do it and service failover traffic failover between clusters i want to thank you for for listening but mostly i want to sort of direct you to a URL less so all of the demos that i've been showing you I literally just build those up and docker compose because I can cuz it's that easy and I encourage you to you know just clone that repo have a play let us know what you think and if you want to kind of chat about things let's talk about it on the community forum so discuss dot hashey call thank you so much for listening [Applause]

Channel: HashiCorp

Views: 10,604

Keywords:

Id: Aq1uTozNajI

Length: 39min 26sec (2366 seconds)

Published: Tue Oct 15 2019