Designing Cloud Native Applications - Deep dive

hi good afternoon everyone I guess most of us are settled so we'll proceed so hi my name is Rahul and my colleague Aman and we work with Cisco on their service provider solutions and they're two more people who contributed to this but unfortunately they're not here today like so so basically we have a solution which does a lot of things in the back end with the switches and does the configuration and get some services on but what we're going to talk here today is the whole bunch of back-end services that we built and they run or I mean they run on any cloud so OpenStack is one of the supported ones so we started with lot of iterations and we learned a lot of things we tried to we wanted to design our back-end services in a cloud native way and over the time we did a lot of things wrong we did we had multiple iterations we used virtual you know virtual machines where we used to deploy the micro services that were there and then we moved to the container model so lot of learnings in the process and so we would be discussing some of the entire patterns and the design patterns that we learnt out of it so I'm sure most of you would have gone through a lot of them and the ones that we could relate to our work we were going to discuss most of the that content and to start with I'm sure we should have a homogeneous I mean a heterogeneous crowd where people would some other people would be new to cloud natives or just a couple of slides and people who have been working I mean just bear with us so first of all you know what is cloud native application and why are we doing it why do we hear it so much why is the market attracted to it so much so cloud native is basically a software design architecture pattern where you know the applications are designed to run and they're made to run in a way where they can reap the maximum benefit of what the cloud would provide I've tried to list couple of points below like you know being distributed scalable multi-tenant and platform independence so as if your design inherits these these you know things it would be that would personally I would call that as a cloud native application and so it's different from a cloud enabled or a traditional monolith application so how is it different so the internal components of the system can scale well and are inherently distributed it is lightweight and I mean with the usage of virtual machines containers and different kind of you know advanced deployments they try to make the deployment lightweight and you know DevOps and continuous integration based deployments so that whatever you write reaches the production quickly so monolithic versus cloud native so there are so many things that change so why why are we actually moving towards cloud native so that is because we can easily achieve scale resilience and like updates and so many things seamlessly so let's just take an example of a monolithic application under load so you have a application which has so many components and they're very tightly coupled and you have you know like a monolith and under regular load you would have something like this and as soon as you increase the load one of your components get stress now this is the only way to actually scale up so you have to replicate the entire stack and probably put it behind a load balancer and which which we know is not the most optimized way of doing it because it's just that one which is getting stress and not all of them why do we have to replicate the entire stack so this is what you would be doing in a monolithic application and load and you would distribute the load and then things work fine and come back to normal so instead the new cloud native application approach you would all these blocks that you see would be like a micro service will come down to people who would have a question you know do we need a micro service to be cloud native so we'll come down to that but I mean we know it's a well-known approach so these these cubes over here are your micro services and basically components of your application and as soon as you increase the load and you stress the system something something like maybe your auth component authorization with that does get beaten up really bad right we have seen that everywhere so maybe you want to replicate that put it load behind a load balancer connected to the you know messaging system and then you have things up proper now so everything that we have I mean going back I I talked about resiliency I talked about scale I talked about you know if you do more you want to change a implementation of just one micro service you can do it very easily in a in a tightly coupled system like monolithic it is very very difficult and not so easy I mean then again a monolithic applications are of wide category and it depends on how you have you designed them how well you design them here you have a structured approach and changing things this small changing things become very very simple so every technology that gets so much traction has the enterprise and people you know business it makes a business impact by everyone's looking behind it so let's see what business impact cloud native applications give you they give you all the cloud advantages that we discussed in previously right now so apart from that it's very flexible so most of them are designed in a way where you can just migrate your load across any of the cloud providers so none of the companies like to be really tied with a specific hardware or a platform so they want the independence that at the end of the day we want to run applications right that's that's the goal we want to run the software and we don't want it to be very tightly coupled with the underlying hardware or underlying platform or underlying is any of it so this fits in the model very well and it's collaborative and agile as we discussed DevOps and see is part of it so the time taken between you writing the code and that reaching the production after testing is is short so the business like it I mean they don't have to wait a full cycle before that goes to your customer based upon certain guidelines so as I said there are so many guidelines available today like a common one called 12 factor app so if you follow them you're there's a homogeneous kind of design and it's easy to pick up and you know how I mean it takes people a shorter bootstrap time to get on to the to this kind of system we implement replace upgrade as we discussed become very very simple resource optimized and resilient so I mean most of the contain most of the micro services are moving towards the container approach because of obvious advantages and that gives you you know high density virtualization and it can be easily clustered to provide recovery in case of failures so okay so getting into the technical stuff bit so I mean if you ask I would try to design a cloud native this would be a basic primary cloud native architecture that we would have we would have a you know a couple of micro services that could be your business logic you need to have you know log aggregation there because these are so many distributed systems running applications running on so many nodes and so many instances so you need a aggregator where you can actually debug things when they go wrong API gateway for routing your requests internally and health and monitoring because then again these challenges come along with the distributed system where you it's very difficult to monitor so many systems altogether you need a system that could tell you the health and probably even take care of small situations discovery service will dwell more upon that as we go for forward and backing resources when I mean I mean when I say backing resources we say anything that's get connected over the network like a database a cassandra has so that and then you see interestingly we have a REST API on the top and messaging queue at the bottom so why do we have this so this these are the two two systems that we use extensively for communication so any of you know the rest is natively synchronous so any kind of synchronous request would be and all the public facing API is will be usually rest and anything that you want to do like mostly your micro services talking internally or anywhere where you need a synchronous environment you would use the message queue so going forward I mean let's just discuss a point which we discussed earlier so do we really need to be like a micro service always to be cloud native no not really you can design a monolith to be cloud native as well but then it is a big challenge and you know as long as it it gives you all the advantages as long as it scales well as long as you can change things as long as it is portable it is cloud native so but then there are some guidelines which make your life easy we don't I mean it depends on people if you want to go to the market quickly there are some set guidelines we can follow them and go to the market quickly or definitely there's another way where you sit with the micro lengths and try to design better I mean but for the discussion here today we can have we just laying out a couple of points that we felt helped us go on cloud native so one being the DevOps so what is DevOps so DevOps is the collaboration between Devin offs and it's not just tools it's a process and culture as well so some of the tools that you would be using would be your ansible puppet chef then their CI CI would help make sure that your the code that you write reaches the production in a very short time and is tested well third is the containers I mean you can we can always have virtual machines or any other infrastructure service as well but containers as we see are with the evolution of talker and the tool kit that comes along with it docker rocket these have become much better for the application developers to write their applications on top of that and with the low footprint it really solves the purpose that the virtual machines were solving very well so we see that using containers gives you a definite advantage then is the fourth thing which is your your core logic which you would put as micro services so basically all all the other three that we discussed earlier we're part of process or infrastructure and you choose one out of the many available options and micro services the basically that area where you can actually innovate and you can follow this would be something that you would be designing so this is where we would take I mean a deep dive and Amon would be going through couple of anti patterns and design patterns which we saw work good for us in our product and I mean there again there are so many there are hundreds of design patterns but these are some of the patterns that really helped us do our stuff well so thanks round yeah so the first thing is the tool factor app so to design a micro service architecture there are some set of guidelines these are the best practices that we can follow to design a micro service architecture so the actual factor it's available on a 12 actor dotnet web site I am NOT going to discuss all of them but let's see few of them so the first is the code base so the tool factor app says that there should be a version control system being used in designing a micro service architecture like gate or supervision ATC so the idea is traditionally monolithic application also had the version control system like gate or something but the problem there was for each and every component there were different code bases but with 12 factor app the entire application will have a single code base but the deployment can be multiple a single code base can be used to deploy a development environment or a staging environment or a production involvement so this is the idea behind code base I've about build release run so the build cycle says that the code can be bundled into executable then there is a real cycle which can put the current deployment config and then there is a run cycle which can actually go ahead and deploy a different kind of environment based on like the involvement can be different as I said it can be a production or it can be a staging or development et Cie because all the environments are almost the same third I would discuss a very important point is a dev prod parity so what used to happen in monolithic applications is that there were differences between a development environment and actual production environment so developers were using a lightweight database database access while the production cannot use that so with the 12 factor guidelines you have to keep the same development environment as you have production environment you can use the DevOps thing where where there is a collaboration between the development and operations and the basically the difference between the development and production should be as minimum as possible so you can read about that any question we'll discuss after the because I do not have that meaning that much time next we'll discuss about some of the design patterns of micro services so these are not all design patterns that we are going to discuss there are lot many on the web but what we are going to discuss is the design patterns that we saw in our cloud native application when we were working on that we found out some anti patterns as well as some design patterns as well I'll start with them so the first one that we have identified is a fragmentation pattern so it says fragment as you scaled and fragment and the advantage is that fragmentation gives so what happens whenever we see that there is a cloud native application we talk or talk about cloud native we start designing in a micro service based architecture but not all components are worthy of being an abandoned micro service it takes resources it's difficult to manage not that easy to manage if you have multiple pebbles in there you have to manage each and every one more micro services means more stress on your network yeah but I mean traditionally when you start thinking you start thinking you know okay I got so many business units I need and I have these business functions so let me start putting all of them in a separate micro service and then I have logging this that so when when you start you have like 2025 micro services and then as you scale you find that okay this is a concern I need to split this and then you know it's just basically a sprawl and after that you it just becomes management becomes difficult so yeah so the best way to start with a monolithic approach I am NOT saying to write a monolithic application but approach should be monolithic where we need to identify what all services are worthy of extracting as a different micro service so let's see with example so there are four components in an application and the are so going forward we identified that the component two is the one which can be stressed out so the so the idea is to take out component two from the application and scale it independently so this component two can be the first microservice in your application so this is what fragmentation pattern is next is a resource adapter pattern so most of the time we identified that the service has a public endpoint so what happens in this that case anyone outside of your outside of the application can access can directly access the surface this is a major security concern as any malicious user can get access to the surface so here comes a resource adapter pattern where you are not exposing the public endpoint of the surface directly to the client instead you will be having an adapter service in between the request and the service the public endpoint will be shifted to the adapter service and the actual business logic service will have an internal endpoint the benefit is that the adapter service can validate the request coming in and only the legitimate requests can be forwarded to the internal one and rest of the them can be denied it actually it also does some sanity test where the adapter where some requests we should not go to the service logic is handled by the adapter service itself yeah next is an Taekwon pattern so how we all started learning coding in when we were kids we started with hard coding everything in a file then then we moved ahead we started using variables we started using stacked-up variables and then those variables were used in the files similarly with the software development process in the very old days we were using constant files those constant files were part of our code base itself these files were used to these files had all the hard-coded IP ports and everything were there and other files were leveraging the constant file to know more information about that then we moved to Quon file approach so where everything was written in a etc' Khan file kind of a thing so with this approach also since we are currently in a distributed widely scaled environment that the problem is we cannot handle everything with a corn file because there are so many changes everywhere it's it's widely distributed yeah the system so dine make you in a cloud native let's say you are auto-scaling you are bringing up services on demand and then how do you go and configure the con files and so this was a problem when we started going to the cloud native approach and hence we had to get away with that and put as much things in the environment variables and you read so I guess so the solution to this problem is to couple two things first is the use of environment variables so everything would be most of the thing would be running as a friend time environment variable so let's see with an example so here I have two services service a and service B let's say we service a wants to talk to service P it doesn't know how to resurface B so one possible thing is it can have an environment variable where service bill IP n ports are exposed the other thing is there is a central discovery service called console or something which is a distributed lookup service which contains information about IP port and some other metadata as well so service so this console information can be exposed as an environment variable and service a can query the console cluster to know the service B location console cos cluster can reply back with the IP port and what other metadata it needs then directly service a talks to service B so this is the cloud native approach for between services next is a circuit breaker pattern so most of us have used a retry kind of a pattern where in a client-server model whenever we are trying to send a request and there is no response back we try to send the request again after a certain time out but let us think about a scenario when every time you are sending a request after a certain time out but you are not getting the response back so there there is a problem how do we solve that so there is a circuit breaker pattern - for that it says do not try if the request is failing continuously so let us see how does this work so it also uses a central discovery system so let us say service a is trying to send a request to service B but the response is not coming back it does some retry it tries to send the same request after a certain time out but and the request is not coming back so what service a does it notifies the central discovery service that B is down now if any other service they say C is trying to talk to service B it will first query the central discovery service if service B is up or not if we get a reply that B is down wait now the pattern this is the pattern now to recover from this situation you can have multiple approach you can either scale service P or you can replace or whatever that that depends on you but the pattern says in such kind of a scenario your request should not fail if some component component is not working the other application should be working seamlessly but the the application should not the application should keep poking yeah so basically you go to the plan B and if you want to show service unavailable or you want to show you know that a previous version the service works and then you can bring up that and related serve the request for some time so this so many approaches for the firefighting but I mean this that is after this so this is like an advanced retry pattern but yeah right thanks Rob so the next thing is a log correlation and aggregation pattern I would say so what happens in a micro service in traditionally what used to happen so my app was a deployed on a single machine so to manage the logs it was easy it was relatively easy now with the use of micro services we have distributed micro services and a client request can come to the first micro service and then it can the the life cycle of the request can go to different micro service and all so similarly there can be multiple service requests coming from the client side and each and every micro service will generate their own logs now if you want to debug a certain service request how do we do that it is very difficult because we have so much of micro service so much of logs generated out of micro services so what this pattern says you tag incoming service request with a correlation ID and throughout the life cycle of that service requests in between any microt service whatever the request goes the same correlation ID will be carried and the locks which are generated from microservices will also carry the same correlation ID it can be coupled with a log aggregation pattern as well where a log aggregator can pick up logs from different micro service and project it at a single repository yeah so now these these five ways some anti patterns that we see that that they are not normally these kind of patterns are implemented now we will move to some regular patterns so this is the first pattern this is related to design and scalability so this is a leader election pattern so multiple instance of same micro-service should always elect a leader so what happens multiple instances are there they are sharing a same resource so there there is need of someone which can coordinate this resource sharing process so there this redirection patterns comes in so out of these three instances one will be selected as the leader so the leader reaction can happen based on the lowest instance ID lowest process idea there are multiple applications for that so one important thing it is important for all the instances to keep poling the leader because any time the leader goes off this system goes off so in this case our I I so either the non leader instances key poling the leader or there can be approach off for leader and as a sub leader as well next is a queue based messaging pattern so this is related to the availability so it says instead of sending the request directly to a service use messaging queues so what happens a client is sending request to the server there is a micro service which takes the request so in such kind of a scenario the client request is low for some time the request is very low but suddenly it spikes up now definitely the service can scale the micro service can scale but that particular spiked up request is not handled properly so the so in this case we use a message you wear at the input of the message queue even if the request is spiked up the output of the message queue is always flat it may be high the flat the flat thing may be high or it may be low but it's always flat one problem with this approach is that it cannot be used at a point where we need the response to come as soon as possible because this is yeah it is a synchronous it is it's in a queue so if the request is so if there are some other requests to be solved it will it will be served first so this is one problem with this approach but most of the places it works fine the next is query and update segregation pattern so this is related to data management so it says that the database schema for read and write operation should always be different so what happens we have a same schema that for read and write model let's suppose that we have a same schema for read and write operation so the same data transfer object will be queried from the service so this causes a mismatch sometimes because sometimes you are writing something you are doing a write operation you are adding some columns but the read operation is not able to read the added columns or and also it can cause contention issues where the resource is logged for writing and you are trying to do a read operation so to solve such kind of problems we use this pattern where the the read and write model are different and that way the read dto and the write DT are also different depends on the number of requests so the thing is the read model should always be a mirror of the right model because essentially the thing is the same now depend on the request it can be scaled like we have multiple databases multiple instance of databases for read and write but the model itself is different ok so this is the final part of our presentation I let Rahul speak about this so then again we come to a point where we thank someone he he actually took you through all the design patterns that we identified that we're very important with our development and I hope it helps you guys as well at some point in time and so another important thing is how do we split the micro services because this is a very basic use case when you don't thing with the micro-services you might have a cloud monolithic application and you want to migrate it to a cloud native scenario and maybe other use cases let's just go ahead and discuss this so these are the three basic use cases which I could immediately come up with so it's like needed when when we are trying to transition a legacy monolith application to a cloud native we're trying to split a micro-service if it is needed structurally or functionally or we're trying to split a micro-service when I mean with which has components which might be bottleneck at scale so how do we identify that you know where do we actually start how do we split so we we all know that database is one of the concerns but let's start with this so do not compose the entire application at once this is sometimes the most common mistake that we do we take the application and we say you know these are the different logics and let's just disintegrate it into so many parts and try to see them up as microservices but that doesn't quite work very well so it should be always like a chip so you start chipping them on a lip and at the end of the day you can segregate it into many micro services so another approach is let's say you have you know start with building a new my feature as a micro service so if you get a new feature you start with that as a micro service and build adapter around your monolith so that it can talk to the legacy your new micro service can talk to the legacy monolith and identify the areas that would need change in the immediate time frame and pick them up for splitting and then again identify areas which would which upon changing would not impact other areas so these areas are called seams in a monolith where if you change that part of the code that would not impact the rest of the application so this is how we need to identify the split areas and this certain approaches that we take while splitting so database is the most tightly coupled part of most of the monoliths that we've seen so the schema we started we really need to start looking at this schema and when we look at the schema the foreign key constraints are the major deterrents to start with so how do you actually go ahead and take away their foreign key constraints so you have to expose an API so that you can get those values for for the new microscope and we need trying to split it yes there are some of the problems that come along with it like let's say data inconsistency let's say one of your tables has a value missing so these are logics you need to build as you go and because it would be very case-specific and splitting components that share common data so let's say you're accessing a database and two of your components are accessing the same data it could be a static data or something so you have to make sure that you split that in a separate microservice and just get a getter setter kind of micro service which just sells data then the most simple would be like if you have columns in the table and the logic in the code which can be mapped and separated there might be several replications as part of this but then again it's for the greater good and another last but not least the important part would be that you have to handle this in a proper transaction and rollback based model because initially when you had a monolith and if some part errored out you would know it very well because it's all contained inside application but when you have a distributed cloud native application in so many micro services let's say your requests get served by three micro services and then you have a error in the fourth you need to make sure the sanity of the system remains after your you you know rollback so this is an important aspect when you were actually migrating from a traditional system monolith to a micro service based cloud native application so yeah so that would conclude most of the part that we wanted to cover we would be open for question and answer still we have time I guess we should still have some time we have hey go ahead sorry I didn't see ie my name is Michael McKean I work for Red Hat and so you were talking about injecting information into your into your applications using like a configuration file or using environment variables a lot of times problems we run into are injecting like sensitive information into these applications like credentials to access cloud resources I'm curious about your thoughts about like you know you don't use an environment variable or a configuration file on these cases you know what do you guys do to solve those problems I mean do for us while we were developing so you can use something like maybe vault or the secure things to to get a token and but I might not have the best answer right now but we can definitely connect on this yeah that's one valid point in our dev environment is always you know it's easy but for a production environment that's really a concern I think we do use vault for that but I don't know whether that would serve this purpose or not Anika actually hi I had a question around compatibility of your micro-services show um how do you how do you manage it if business requirements are for say say moving in a direction that would require a change across Multipla meant a particular feature is this something where you have to do artificial things to make sure that you're always being compatible or is there a straightforward way to manage those kind of changes across multiple services I think most of this would be taken part as part of CI as well and when I mean you start making changes to various micro services and then you have to you know eval you're integrating you to make sure things roll out well and you can do it in a phased manner that wherein you make some changes in in one of the microservices and still you have the API compatibility and then you deploy the next one and slowly move over to that so if I understand your question correctly that's what I would be doing hi thanks for the present is Christmas English by the way as a two-part I have two questions one is around microservices how do you maintain the relationship between cross micro-services so from business perspective you have a and that goes to the monitoring part of my other question is how do you monitor the system availability from business perspective Thanks oh so I mean we have written down a component that actually queries kind of a component that goes and queries the health of each micro service and we have a small kind of an agent inside agent code inside each of the micro Service which gives it all the relevant data so then again if it is something very general then you can use so I have one point to add on that so it happens on two approaches so one it can happen internally as well so internally the health monitoring micro service can make the as a real talk there are some in it files inside which inside the micro service which provides the health information so this health monitoring micro service can talk to that init file can get information from that init file to know about the health of that particular micro service and then what next it does is to export this health in the health monitoring micro service exports it data collection from inside to outside and then outside there are tools which can be used for the other health monitoring perspective yes so like as we said we kind of give you collective health so I said initially we have a couple of switches that we go and monitor and this is the back end so we collected that's I mean that's very specific to deed employment itself but for our case what we did we exported all the data out of the monitoring micro service and then Club it with with all the like some external kind regime which is taking data from Inter from the switches we take the health and we just give it a collective health only yeah thanks maybe a follow-up on that so you should talk about separating out a micro service it does it reads and writes our data is it still valid to have two micro services say write the same data that's stored by another micro service or is that kind of a violation sorry I can you just repeat the question I am so so so you have a set of data that that two different micro services would operate on including writing them here I got a violation of micro service design or is that okay I don't think that's a violation of micro service design I think that's fairly pretty well and that is one of the approaches that we have to take when we we're kind of splitting the mana right but then don't you have the issue no no so what is an namespace kind of a concept so each of the micro service will have their own namespace kind of a concept so even if the couple of micro services are sharing the same resource it will be isolated it will have its own context different micro service will have its own context on that so ad once you have a dependency on one interacting with the other way we write right so that was one of the points that actually helped us so when you start designing your microservices ground up and you start with so many micro services that sprawl it will eventually go on and happen that is one of the approaches but then each of the micro service the kind of data you sending out is totally under your control so you have to basically make sure that it's a it's a distributed system and it will generate a lot of network traffic so you have to while writing the code itself you have to make sure you write it in a very optimized fashion okay I guess we passed the are if there are more questions we can actually take it offline so thanks everyone for your presence here and clearly appreciate it thank you everyone [Applause]

Channel: Open Infrastructure Foundation

Views: 5,999

Rating: 4.7272725 out of 5

Keywords: OpenStack Summit Boston, App Developer, Container orchestration, application deployment, Project Technical Lead (PTL), Enterprise, orchestration, container management, CB Anantha Padmanabhan, Meenakshi Lakshmanan, Rahul Krishna Upadhyaya

Id: aJq9KwVcMjc

Length: 40min 53sec (2453 seconds)

Published: Mon May 08 2017