AWS re:Invent 2018: Using DevOps, Microservices, & Serverless to Accelerate Innovation (SRV325)

hello thanks for coming out and joining us today I'm David Richardson I lead our collection of our service offerings and like my colleagues Deepak and Ken I've been lucky to be a member of the ADA bs team for quite a while now so we were talking about this and we thought we would try something new today we thought we would share a little bit of our experiences at Amazon and about how we think Amazon goes about building software so give us feedback after it's over let us know if this was something that was helpful I joined the company in January 2006 in the infrastructure and networking group that was a little bit before AWS launched and I've always been a software person so once I joined the company I started looking around and trying to get a sense of how Amazon went about building software and what I quickly noticed was the company had a very different approach from many other companies and it was clearly a conscious decision because it really touched many many parts of the business and it was obvious that the goal of this way of building software was to drive innovation in particular a type of innovation that lets you do lots and lots of little experiments across a very broad part of the business now it hadn't always been that way in the early days of the company as I started to get to know some of the older software engineers there had been one giant C++ hairball that was called obidos and one giant database relational database called ACB amazon.com books and as you can imagine that was definitely not an agile architecture it meant that you know and by the time I joined there were many many products beyond just books if you wanted to add a new type of product say digital goods or a new physical product like CDs or housewares you had to edit the same stuff that was also supporting the existing well-established bookstore and so that was a high risk environment something that making changes to really how could could risk the business and that's not great for driving innovation so sometime in the early 2000s the company started moving to what today we would call Micro Services back then it was called service-oriented architecture now you may have heard that Amazon's kind of a writing oriented culture unfortunately it turns out not a lot was written down about that style of software so I was really excited later that year I think it was in June 2006 the Association for Computing Machinery ACM published an interview with Verner Vogel's about Amazon's approach to building software so that's how I started to learn about Amazon's approach and and you could tell that it was not just a technology approach it was a way to be able to combine you know technology architectural decisions but also organization design patterns and an infrastructure that could support building in that way but most importantly with a with a culture with with a leadership approach and so you know the company started moving to that in the 2000s and it started to take an approach where we would break down components of the architecture into very small services where the idea is a micro service would just be responsible for one thing because that could let you kind of move more rapidly because you could better understand the changes you were making and this was also related to a way to organize teams to build teams and we call that to pizza teams where the idea is you can give a team a Charter ask them to own a problem where the teams no bigger than you know about two pizzas to feed them and these teams have some strong properties the first is ownership and and I'm in Midwesterners I really like this that's one of my favorite leadership principles because I think you just got to be responsible for stuff and so the idea was let's have teams be able to make decisions but then also be responsible for the results of those decisions to be accountable these days I think a lot of people would call DevOps where you know an engineering team is responsible for designing for developing and for operating services I just call it ownership and so that was a key aspect of these two pizza teams where the idea was you could drive focused innovation you could have an idea you could spin up a team ask them to go experiment and to figure out you know is does this idea have some legs are we gonna get some good results out of it can we make some improvements on behalf of customers and so they could go and build the micro service that that would drive that and have the set of technical infrastructure to support it so I think that this type of micro services approach is a natural fit for modern micro services that are built in a serverless way our goal with Cerberus is to really radically reduce the amount of infrastructure you have to manage so you can really just focus on business logic without having to worry about how to compose a lot of these pieces together so we're gonna talk today about some common elements of these I'm going to talk about some architecture patterns that we see work well for these types of micro services and the infrastructure that supports it Deepak is going to talk about the operational model of running these sorts of systems and Ken is going to talk about how you actually build and deliver the software that makes these up now as I go through these these architecture patterns please keep in mind the the infrastructure the software services that are there to support it because the idea is if you build in this way the services from AWS are made to work well together in those patterns and should reduce the amount of non business logic that you have to work on now I think innovation requires change and the easiest kind of innovation to drive is lots of incremental change lots of experiments that's much easier to do than to wait until you make one huge change because you can just get more more iterations you can learn faster by doing more of those and that's where a microservices approach plays well so you can iterate on each of those microservices at whatever pace of innovation you want to drive and you can have teams that also can can be staffed appropriately for the work that you want to do now those of you with a software engineering background may be thinking this this sounds a lot like Conway's law and Conway's law is an idea where you're your org design and your technical architecture wind up looking a lot like each other and that's because really they're both about communication if you think about what a team what an organization has to do to make a decision well they have to communicate with each other and the larger the the team is the larger the organization is the harder it is to make a decision you know there's just a lot of communication that has to happen and similarly one of the key aspects of a technical architecture is how you compose systems together you know how you can reduce the amount of knowledge an engineer has to have to make one decision so I think a micro-services approach is really Conway's law it's really how can you have an organization that can make decisions very rapidly and have a technical infrastructure and a set of you know technical design patterns that can support that so there are lots of different ways to build micro services and we think there are kind of three key patterns that that work well especially in a service way and so I'm going to talk about those the first really are kind of API driven request response applications and I think of api's as the front door to micro services actually inside of AWS when we talk about to pizza teams we often talk about a hardened API as a requirement of a team so we even think about api's at a team level and that goes to that organizing practice of the API is the promise they make to their colleagues in the rest of the company and that shouldn't change very often but behind the scenes we want them to be able to innovate as rapidly as possible to be able to changed the implementation in whatever way makes sense for the business and so that API that front door to the micro service lets them do that now we've built a lot of micro services as part of building out AWS and so we've seen that there are some very common patterns that show up you know truly just in the management of these AP is not necessarily in the API itself but in how you go about building and operating and managing api's and we've tried to provide those capabilities for developers in API gateway where the idea is you know we can take those lessons learned and instead of every new micro service having to re-implement a well-mannered API API gateway can provide a lot of those capabilities so it often starts with authentication so that you can control which customers or other micro services can use the API an API gateway is well integrated with kognito and with Identity and Access Management there's also usually an aspect of resource control and so API gateway provides throttling there's always some some insight some measurement and monitoring you have to do with your API and so API gateway has built-in support for x-ray and cloud watch and cloud trail so that you can manage and monitor your API but oftentimes the API doesn't live in isolation it lives in a network and so you also have to have some controls about call it network reach ability what network environment can your clients be in what network environment are your backends in and so API gateway has kind of a full matrix of that the front end the the front door to those api's can be out on the public internet it can be regional it can be in a VP C similarly the backend can be public backends can be in a VP C and you can even have different VP C's between the client side and the back side and it turns out you can also use Direct Connect and our VPN offerings so the backend might not even be in the cloud it could be on-premise and reached through through those networking offerings but still managed and appear inside the cloud now if you went to Vernors keynote earlier today you probably heard that we're launching WebSockets later in December and I'm really curious to see how customers are going to use this if you think about that matrix of network reach ability we're kind of doing the same with with statefulness you know with WebSockets you have these long-lived stateful connections and yet with API gateway you can implement those in a short-lived service fashion so you can have you know WebSockets that live for minutes hours days and yet use something like lambda to just quickly fire up compute that will respond to the actual requests across WebSockets so we're really looking forward to see how people can expand the capabilities they have in traditional request reply micro services now to long-lived stateful WebSockets but still keep building in a serverless way now if you really get into building micro services with lots of api's you may find that it's not enough to manage each API individually that paper that I mentioned I enjoyed reading back in 2006 from Vernor about how Amazon went about building software he pointed out that even back then if you visited an amazon.com webpage behind the scenes hundreds of micro services were actually firing off to build that single page and so these environments can get quite large and so we've now launched app mesh as a way to be able to discover and communicate with the various api's that and micro services that might make up your application so again in this same pattern of we want infrastructure support that makes it easy for your developers to use simple DNS names SDKs or api's to discover and connect to the api's that make up your micro service and related to that we know that sometimes you have to manage network communication faults or software updates and things like that so we've also launched AWS at mesh so that you can again have a managed infrastructure that we provide for you that you can use to deal with these network conditions and it can be used across multiple clusters across multiple orchestration systems and even across multiple V pcs so that that's kind of you know design pattern number one request reply driven micro services the the second common pattern is aventure than architectures and in a way this is kind of a Back to the Future design pattern you know back in the 1980s many enterprises started moving to message oriented middleware or enterprise integration patterns really for the same reasons that they're super valuable today in in modern service architectures cuz they have this great decoupling capability they make it really easy for you to have lots of different systems that all interact in kind of a reactive way with each other via events and so we have a set of infrastructure that we provide to help you build in this way we have our cloud native queuing and notification service that have massive scalability deep integration with the rest of AWS and very tight integration with lambda in particular to make it easy to build kind of a combination of events and compute that react to each other and it turns out I don't know how many people know this but every time you use an AWS service you can also generate an event in our service called cloud watch events so these combined make it easy for you to build loosely coupled systems where you can have some action that triggers an event that you know has some compute that may trigger some more events and so on so makes it very easy to evolve an architecture the third major design pattern we see people using our data stream oriented ones and and you know these can be what people will call an event sourcing pattern it can be a data flow pattern but basically where instead of events and messages the key connective tissue through your application our data streams and here we provide support through dynamodb streams which is a really neat way to be able to as you drive changes in your DynamoDB tables you know you can have a stream that can then drive computational activities or other things as part of your architecture or you can use Kinesis in a same sort of way you know more generically however you want to start those data streams and in that same spirit of we want to make it easy to use these systems together right before reinvent we launched enhanced integration between lambda and Kinesis data streams which really is just to make it even easier for these to scale up and down with the needs of your business without you needing to do any of the work and to make it you know more performant now one thing that you've probably seen across all three of these these design patterns is it's a lot of systems working together I think that is a big change from more traditional monolithic architectures where you're now going to find you have a mixture of an event or a data stream or an API and some compute and usually some persistence storage together and so another family of tools that we provide is is something to make it easier to manage that coordination together so we have a service we call step functions and its sole purpose is to make it easy for your components to work with each other to manage the workflow through those and so earlier today we enhanced that with integrations to many more AWS services and you know this is something that you could have done before even with step functions you could have written a little bit of code that would be responsible for starting a job checking to see if that job was done maybe retrying it and then eventually moving on but now with these enhanced integrations you can just basically say two step functions what you want the workflow to be and it will manage all of that so the idea is let's keep simplifying the ability to build these API driven or event-driven or dataflow driven applications without needing to focus on on the infrastructure one of my favorite sayings from the early days of the web was small pieces loosely joined and this this was an attempt to you know try to define some of what made the web so magical so able to grow so rapidly and with such diversity and I think it applies very well to these sorts of innovative fast-moving architectures that if you can take this same kind of approach to innovation where you can do lots of incremental innovation lots of experiments via a more loosely coupled system rather than a big tight monolithic system you're just going to be able to innovate and experiment faster and so small architect you know architectures of small components that are loosely joined through things like api's or events or or data flows are going to make it easier to grow like the web compared to large monolithic applications and the big teams that's required to own and drive those so that's a little bit about three architectural patterns that we that we believe you know can help drive innovation and now Deepak is going to talk with you about how how to operate these sorts of systems Thanks and thank you everybody for being here I know it's Thursday after a few pub crawls lots of keynotes there's still one more day to go so David talked a lot about how architectural patterns are changing as we deploy micro services and more loosely coupled architectures in McCallum one of the lessons we've learned at Amazon and perhaps even more so with through our customers as David opted these practices and I started deploying them into AWS is how do we operate our infrastructure how do we operate help help them operate their services with these patterns and I'm going to try and go through some of those lessons and some of the learnings over the last decade or two of running infrastructure as loosely you know loosely coupled services not just in AWS but in general so one of the first questions you see in fact every time micro services are discussed in a place like haka news I'm pretty much the top post on that thread will be micro services are too complicated there's so many things to manage in a sense they're true but it also has to do with how you think about the organization and how you operate your services if you have lots of small teams where each team has discrete and distinct ownership of a small piece of that puzzle they can operate that they can build it they can run it and that's the part that they are responsible for but what you also need as you do that it's starting to build monitoring tools observability system management tools that help you get and Corral the greater sense of the world and a lot of what we've learned over the last few years and what you've seen us launch and add is how do we live in a world where people are moving much faster where there are lots more teams and there's lots more systems lots more small services as part of a larger system one thing that's evolved quite a bit over the last few years is how we think about shared responsibility if you've been at reinvent before if you've been anywhere near AWS people you've heard us talk about shared responsibility and the shared responsibility model what it really means is if you could draw a simple box there's a line somewhere in that box that says the part above that line is the customers responsibility and the part below that line is aw asses responsibility the question then becomes where is this line drawn we give you lots of ways to draw that line let's talk about databases you could run your own database on ec2 in fact a lot of customers do so if you do that you are responsible for running that database you're responsible for patching that operating system you're responsible for figuring out upgrades how do you make it available how do you scale it we give you tools like auto scaling that help you do that they'll give you backup tools they give you three availability zones in many men in most of our regions but the responsibility and decision-making is still yours you can get one step further you could use something like RDS RDS takes care of the database management part of it but you're still running your application yourself there are still some reasoning around how that database runs you have to choose whether you want to run high multi AC RDS or single ac RDS do the decisions you need to make or you could go all the way over to this side but you're running something like Aurora where it's my sequel or Postgres compliant but the storage infrastructure the availability of the storage infrastructure you don't have to think about okay do I now need to run a multi AZ database it's built-in for you or you could go the extreme sense and run something like dynamo DB where you have to make a decision on whether you want to move from a relational system to a key value store and especially some of the features at the announcement dynamo DB at this reinvent the operability of that system that line of responsibility has gone to a place where you are responsible for less and less in AWS responsible for more and more our goal in the end is to give you those choices but continue to deliver features and capabilities where the responsibility part of that box heads more and more into the AWS side so that all of you can focus more on building the business logic and the business value for your companies there are you know you I've heard people say people I know that you know anytime that in a company that they're spending time crawling infrastructure and building scaffolding it's time that they're not generating value for the company and our goal is to try and help you make that less and less there's been one interesting trends many many many years ago there was a service that was launched called ec2 what is he to did and one of the big changes that it introduced was it took the concept of a server and put an API around it you could certainly start it up when you wanted shut it down when you wanted it to use analogy which as someone from India has always been weird you took your service from being pets to being cattle you don't like killing cattle in India just don't give the knowledge it is anyway but I'll use it because I'm here CS ever been used to be a pet you nurture it you've watered it at night you may have made offerings to it especially with other really expensive big iron server but now you could shut it down when you wanted in fact when ec2 got launched before I was at Amazon one of the first things I did was launch about 20 instances which is exactly the instance limited ec2 gave gives you by default and shut it down about four hours later was awesome but something happened we built infrastructure around this to make it easier to operate all of the infrastructure you added things like auto scaling where suddenly adding the second server became very easy to your server was still I was still careful there you know when you've had no work to do it went away then people started running more sophisticated applications on those on those servers you started adding systems like zookeeper or a state management system because now what you wanted to do was I have this pool of resources I want my jobs to land somewhere in that pool of resources but you needed something to manage the pool of resources and have control over it along the way containers came up container orchestration is awesome I spend all day thinking about it but they also made the cluster a thing the cluster suddenly became your pet if you look at how most container orchestration systems work every cluster has a data store every cluster has a leader node which you know how to think about should this node be running in three AZ's how should i manage my datastore should I back it up how do I back it up what happens if one on AC goes down and there's a write going on at that time those are things that over time cloud providers like us provide managed services like eks that take care of that part for you but you still have to reason about the instances you are selecting what applications run in which cluster where they should multi-tenant it at all what's the operational model it's almost like now people are praying to the cluster giving it food giving it what I'm nurturing it it's fine it works really well but I think it breaks some of the operational winds that we got when we went from servers getting api's and the question is how can we get away from the cluster becoming something that comes in the way you've already heard David talk about servers and that is one of the big big things that we've learned is especially even within Amazon is the less infrastructure your provision the less infrastructure you have to reason about the more scaling decisions are made on your behalf the better it is but server lists you also pay for value you're paying for some unit of consumption whether it's a function whether it's a task in the case of dynamodb but the data in your table the more automated it gets in the less you reason about the capacity part of it it's awesome and then availability and security should be part of that system those are not things that you bolt on or have to think about you should just get them by default so add AWS and quite honestly you're missing a bunch of things on this it spans many many different categories of services everything from integration from data stores to compute I'm going to talk about compute today so on the compute side we have two server lists or offerings the first one you were a little bit about that is lambda for the one person in this room who may not know what lambda is you define a unit of code as a function you give it a trigger when the trigger hits something happens as someone who has built services on top of lambda the fact that you can add some new piece of logic you have a new routing service that you want to add or there's an API that you want to add that takes care of a certain action the fact that you can quickly do it in a lambda function deploy it define the trigger that's going to make it active and get it running without having to worry about provisioning without having to worry about what the operational model should be it's pretty liberating and you've seen all the capabilities that have been added to lambda both by AWS and our customer base over the last several years four years since it got announced there are another class of applications where you may want to have a little more control and say over how networks architected services communicate with each other what your packaging should look like in their unit of unit of work goes from being a piece of code to potentially being a set of container images that you've defined and stored in a registry you can define some networking but you don't want to think about how you take that containerized bundle and figure out where and some cluster you're going to launch it and start thinking about what the cluster should look like in an ideal world you should be able to define that and just deploy it and that's what what what AWS Fargate is and it's typically used for long-running services but also for cron jobs it helps you bring existing code over you don't need to completely rethink your architecture and program model but the big part is it takes that pet which was a cluster it doesn't make it cattle it actually just makes it disappear or other than just being an object in on your dashboard and essentially what you now have is a complete it is a recap of what I talked about with the databases but in the compute space you have a spectrum of where that line of shared responsibility comes you can start off with ec2 where you get to build what you want to build you even get bare metal ec2 instances now if we wanted to slap on a hypervisor you could do that all the way up to lambda where all your reasoning about is if is your application code and I think as our customers have found as many of you have found that's become very empowering now cool we've taken a lot of the operational responsibilities of running infrastructure away but what happens next you still have to develop code that becomes an interesting roadblock so one of the areas we focused over the last few years is to make development much much much easier for lambda lambda today is serving trillions of requests every month for you know hundreds of thousands of active developers so as more and more people adopted you go from the early adopters they were happy tinkering around figuring out what to do to more people who just want to get things done and you want to reduce the barrier for them so some of the new things that we've done and I'll talk about this one today and you if you were in Verna's keynote you heard us talk about it is lambda layers now functions can share code you can reference the piece of code another function you don't have to write something new completely it's all in some way that's similar to how layers in a container image are stored and you can reference them and build on top of it it's a promote separation of responsibilities if there's a core library and it's kind of that model that somebody needs to own one person can focus on that and then developers can then focus on the application logic once again your goal is to make it easier to develop to deliver business value and it has built-in support for sharing with the entire ecosystem so again those are things that come by default and not things that you have to reason through the other area where we have tried to make it easier for people to use lambda in high school systems and building really complex applications is custom runtimes who here is happy that we support Ruby now enough people thank you but the way we did it was not adding Ruby support or adding Russ support is to create a custom runtime environment the the adding Ruby support was just a manifestation of the runtime work that we did but it also allows you to bring things like Erlang which partners can develop other people can develop so you essentially now get run arbitrary code on lambda and these custom runtimes are distributed as layers so the layer and runtime parts work in harmony together so one of the beauties of lambda is that it completely allows you to focus on tome but you do have to think about what the architecture would look like it is a different way of programming environment but it takes a big operational responsibility and makes it ours with Fargate one of the fun parts is you can't bring your existing codes you may have been running an application in a container Orchestrator you can keep do it you can just take those images and bring them in you can reason about networks very much the way you have been used to except some of the complexity of managing how you work within the network gets taken away and what did what we've tried to do over the last few years is make containers a core primitive much the way people think about VMs that means adding VPC as a native networking infrastructure for containers if you get an elastic network interface you get flow logs you get all the benefits that a VPC would give you metering at the container level when you're running a container orchestration system you're still paying for the ec2 side of it and and even though we have added cost allocation but you're not metering at the container level with Fargate that's where you think about it and that's how you get built and then you build in things like service discovery you build in things like I'll talk about service machine a bit those all get become part of this ecosystem that this containerized bundle lives in there's a few other things that we've launched in case you haven't noticed we launched secrets management very recently be actually planning to do a second Rev of secret management support we once supported SSM we do will support secrets manager we have dougie in cost allocation for those who want to do chargebacks is work in a company there's a bunch of other features across the entire container ecosystem that we've added I think the big one for me is the deployment part of it and integration into the cdk for those of you don't know what the cdk is Ken's going to talk a lot more about it but really what we are working towards is making that operational model much more developer centric than having to pour all infrastructure and think about it in boilerplate sub Jason or or Yama so you know we've talked about a bunch of features that we are planning to launch I already talked about secrets management are we going to put private link in everything the goal again is make the container a fundamental building block that you can then build on but because it's integrated with orchestration the overall shared responsibility line still stays mostly in our hands so one of the challenges going back to the one of the questions you asked is okay there's hundreds of services running we have customers who are running hundreds of services on Fargate customers like tadhana who run many are many many other services with foggy how do you monitor them how do you get control over them how do you decide which service can talk to what in which service can access what how do I make my deployments safer when there's this big system sure I've done everything to make sure my service deploys safely but there could be some impact somewhere else how do I make sure that that's safe historically the way we did it was when you had fewer services that you built a client library or some kind of code in the application your that library took care of figuring out what you allowed to talk to your code had to understand that library but it also meant well you pretty much had to stick to a single language across the entire infrastructure that's fine when you have a few services that find when you have a few teams but when you have a large system we allowing people to experiment allowing them to run allowing them to write in whatever language they want to do that become the challenge so one of the things that became happened in recent times was decoupling of this operation logic and operational logic and SDKs the way it's implemented is you put a client-side proxy next year code typically as a sidecar in container land as a sidecar container and then your application your little service just talks to that proxy and that proxy is responsible communicating with everything else in your infrastructure again decoupling makes life easy in this world this proxy also gives you visibility because it's collecting metrics and with proxies like Envoy around you've built in a lot of capabilities to give you that keep that to give you the control and visibility into your entire infrastructure there's still one challenge how do you manage the hundreds of proxies you're now running how do you decide how they get configured over time and how do you collect data from them so for us there's many systems out there over the last year or two more and more people have built control planes to manage this mesh of proxies for us it's at mesh which was launched yesterday and in preview and a debris SAP mesh is essentially a control plane for Envoy we're on was the data plane it allows you to define traffic rules it allows you to figure out how your traffic's being shaped over time you'll add end-to-end authentication and the ability to even configure things like Al B's and API gateways but in the end it's a service mesh for all your infrastructure on AWS whether it's running in a container whether it's running on ec2 whether it's running on lambda and being fronted by a load balancer or an ATI gateway so but one of the things that we thought about as we did this was the typical model of developing and delivering this was in a cluster but when in infrastructure spans many many services having a cluster bootstrapping again makes you think about how do you run this mesh management within the concept context of a cluster how do you control it how do you scale it so over time but if and this is something that one I talked about a lot today is just pull it back enabled a centralized control plane that can reason upon reason about everything that happens we focus on managing the availability of that system we focus on taking control of how we secure it how we evolve it what kind of features it supports so hopefully many of you is going to start using it playing with the preview it's up on github and allow you to Corral and get visibility into all this all these little services that you have running around all your applications the other critical part about is security we want to get fast we want to be efficient but we do not want to compromise isolation so perhaps my you know it's it's something that most customers will never see but in many ways it's one of the favorite announcements that I've had at this Raymond is firecracker for those of you who didn't see the announcement firecracker is a little virtual machine monitor the goal of firecracker is very simple how do I run lots and lots of little containers and little functions securely without compromising on how can you run them fast and efficiently without compromising on security and what it allows us to do is scale lambda and Fargate and give you the kind of performance that you want again the in the shared responsibility model you'll see us do more and more of these things to make sure that you can be comfortable there as you adopt these new paradigms everything is working really well so with that comes the next stage you've figured out the architecture you figure out an operational model but now you're actually have to build the code so to talk about automating all the things connects now all right so David walked us through some of the architectural patterns that we see in modern application development Deepak took us through some of the operational models I'm going to walk us through how we actually develop and build software for for this new micro-services server list world I'm gonna try to answer the question of now that you have this new distributed architecture of all these different pieces of functions and micro services and managed services how do you actually develop and deploy that to production for questions I often hear from customers I'm going to walk through each of these four questions first is what's the deployment process look like how do you release software in this new world and how does Amazon do this I managed the process internal to Amazon for how we deploy software so I'm going to share some secrets about how we do this a second how do you now monitor in isolate problems in your architecture how do you do bugs at a system level if there are problems in terms of latency or performance in some component of your architecture how do you find that but the third question I'm gonna answer is not just at a systems level but at a code level how do you edit in debug code in this new serverless world do the same tools work are there new tools what's the edit compile test loop look like and then finally I'm going to talk about how you provision resources and how do you how you model your application and how do you make sure that you're following the best practices as you model your application for this new world all right let's start with the release process David spent some time talking about monolithic architectures and in monolithic organizational structures this is what the process looked like at Amazon 15 20 years ago we basically had a bunch of developers a bunch of developers working on a monolithic architecture and then they would throw it over the fence to a manual QA team a bunch of release engineers and they would Shepherd that out to production and hand it off to operations it was a very cumbersome process it was at most we could do one deployment a week because it was very expensive to keyway the entire thing to push this out to production it was very inefficient and things started to get better as we decompose for agility things started to get better as we broke apart the monolithic organizational structure and started organizing into two pits of teams got a little bit better things got a little bit better when we decompose the monolithic architecture into there we go into a bunch of micro services but we still have this problem how do you actually release software now that you have all these two pits of teams owning their own primitive services how do you actually push this out to production how do you manage that release process we couldn't have thousands of teams trying to do this with a centralized release train model so what we realized we needed to do is we needed to give these development teams their own tools so that they could release their own software so that they could own their own release process and they could Shepherd these releases out to production themselves and this is essentially how my team was born at Amazon over 15 years ago this is the Builder tools team at Amazon and he likes to refer to builders at Amazon this was the Builder tools team and we were established to give developers builders tools that would allow them to manage their own release process at first we were doing a couple of things we tried to automate the builds see I was very popular at the time so you know trying to automate builds connecting up the build system with the source control system at the other end of the spectrum we were trying to also give developers their own deployment tools so that they could deploy to production without impact to customers so that was great but we realized we still had a bunch of stuff in between we sawed a bunch of manual processes that we would take these build artifacts move them across the different pre-production environments do a bunch of manual testing the deployment process was also fairly complicated in manual even though we had these deployment tools because you had to deploy across different AZ's and different regions and all of this was manual so in traditional Amazon fashion we looked at this and said that's the production line that needs to be automated so we set out to automate the entire process end to end from source code check in to the final deployment and we started building a tool called pipelines internally this is over a decade ago and pipelines was this this tool that allowed us to go from source code check in to having everything deploy to production completely automated a person when wouldn't touch it or inspect anything anywhere in the pipeline and this is how we started getting into continuous deployment at Amazon now at first people were a little bit nervous it was kind of crazy that you would have this this this system that would deploy to production and no one would check anything no one would test anything it would just flow out to production but when we started to realize is that anything that a person could do anything that a person could check or manually manually inspect or manually test anything they could do could be put into automation and if we did that we could do the same automation in the same testing every single time no matter how small the change was and that anything that someone could manually verify should be put into a script or into an automated test so we started building up a bunch of you know tests integration tests a bunch of API smoke tests every team started having their battery of tests that would run against their pipelines every single time then we started getting into more sophisticated types of testing already doing automated load testing started doing synthetic monitoring not only against production but synthetic monterrey against pre-production I started doing security scans along the way we started to realize that there are actually good patterns and good practices for how to set up a pipeline and we wanted to make sure that the developers were following the best practices so we did a couple things the first thing is we set up templates for how to set up a pipeline gave the developers at Amazon templates for what a best practice pipeline should look like that you know that define only the contents of that pipeline but also the shape of it and the structure it should have it should have connections to monitoring so that you can set up an automatic rollback you should you know develop you should follow it out to different regions in a progressive way fan out slowly all these things were defined in these pipelines in the pipeline templates the second thing that happened in which was pretty impactful for Amazon is that we realized that we could actually inspect these pipelines and if we could inspect the pipelines we could determine whether or not they were following the best practices we could we could define best practices as a policy and run that policy against these pipelines and if the pipelines weren't following seat following the policy for that business we could block the pipeline we could stop the deployment and this was a revelation for us that we could use this this type of technology to ensure that Amazon developers were all following the best practices for their particular business that we could achieve governance and compliance for all these two Pizza teams they were all had a sense of agency we could control that and make sure they're following best practices through standardized tools so whenever I talk about this for to customers they always invariably ask me how can we do the same thing does Amazon have any tools that we can use that allow us to you know achieve the same types of processes that Amazon follows and the good news is that a couple years ago we started externalizing many of these tools as the a dubious developer tools we have code code commit which is private gate repositories fully encrypted we have code build for building and testing your software we have AWS code deploy to do the deployment and we have a two BS code pipeline to manage the entire release process when we first launched these we were mostly focused on ec2 instances and instance based workloads and customers started asking us how about ECS and Fargate and lambda our internal customers started asking is the same thing because they wanted to use the same tools regardless of what compute environment they were deploying to which made a lot of sense you know whether you're deploying to an instance or a function or a container you shouldn't have to change your tools so we started building capabilities that would have now enable us to use the same tools to deploy to ECS and Fargate and lambda this year i'm happy to announce that we have added support for ECR as a source action in Ko pipeline and we've added support for Bluegreen deployment to Fargate and ECS with Co deploy so that you can use you can use traffic shifting or blue-green style deployments with container based services at AWS we're gonna continue to do a bunch of cool things in this area especially focused on server lists and containers so there's a lot more to come one one feature that I thought was super cool that launched earlier in this year that I'll call out was that we added support for config rules for code pipeline and why this is cool is that it means that you can set up a policy in config rules a tamiya´s config that governs the structure and contents of your pipeline which means that you can achieve some of the same compliance and governance that amazon does you can define policies for pipelines that allow you to control the shape and in contents of a pipeline the second question I said I would answer is now that you have this distributed architecture of all these containers and functions and managed services how do you isolate and discover problems in that architecture if you know maybe you have some errors being thrown or some performance issues how do you figure out where in that distributed architecture the problem is at Amazon we've been using distributed tracing technologies to solve this problem internally with distributed tracing you can follow the entire request path all the way down into the you know different layers of your application the different components to the data tier and back up and you can figure out where the problems are a couple years ago we launched x-ray as the external ization of some of the tools that we've been using inside of Amazon and what x-ray does is it allows you to analyze and debug issues quickly end-to-end through the entire call path of your application through IV every single component of your distributed architecture there's a couple of new features I'm gonna call out that I think are super cool one is we launched x-ray root causes yesterday and what this allows you to do is do custom alarms based on filter expressions in x-ray that you can use to identify an alarm on whatever whatever custom thing you want to alarm on for your distributed application you can use filter expressions to create these very granular types of monitoring the the third the four the other thing that we support for a couple of weeks ago is API gateway we're starting to add support for a number of new services in AWS but support for API gateway was super important so that we could support server lists so we now support lambda and API gateway and if you use the API gateway you get x-ray by default alright so that's how you monitor and debug things at a system level how about at a code level huh what's the edit compile test loop look like for for editing and debugging code last year at reinvent we announced AWS cloud 9 and cloud 9 is a cloud-based browser-based IDE that allows you to develop completely in the cloud it supports serverless really well so if you're building a server this application I highly recommend using cloud 9 you can do debugging through cloud 9 you can manage a server list application there's a lot of great things to be coming out soon for cloud 9 so so stay tuned but this morning inverters keynote we also announced support for three new ADA Beals toolkits and the reason is because we want to meet developers where they are we've heard from developers that you know IDs are these are very personal thing and and everyone has their favorite IDE so we want to make sure that we're meeting you where you are so if you're a developer of IntelliJ vs code or pycharm we now have tool kits that allow you to do serverless app development using these IDs these are being developed completely and open source so you can go to github give us feedback contribute code if you want to and give us give us some ideas of what you'd like to see next initially we're focused a lot on the server list experience but we're gonna be adding support for other use cases as well the pipe charm there we go the pycharm plugin allows you to create a lambda or serverless application a new project it allows you to local step through debugging in your IDE and deploy your application straight from from straight from PyCharm which is one of the jet brain projects super cool so check it out and give us back the fourth and final question I wanted to answer for you is how do you model your application how do you provision the resources you have a lot of you know different pieces now when things were all in one server one ec2 instance you might have provisioned that using a web GUI or a CLI but as you have more and more pieces to your distributed application things get a little bit more complicated so you need a new way to do this and that's where infrastructure as code comes in there's a ton of reasons to do infrastructure as code not only does it help you define the desired state for your application and it's something that you can reason about in code but it's also something that you can revision control that you can inspect and look at your architecture in time it's something you can code review there's a ton of reasons to do it but one of the pieces of feedback that we always get from developers is that it's still a little bit hard it's still a very low level you're dealing with lots of JSON or ML code that tends to be fairly low-level and map to service api's one-to-one so it exposes all the configuration options a good example is if you're trying to set up a VPC there are so many configuration options for a V PC that you end up with thousands of lines of JSON code just to set up a VPC using cloud formation or terraform so what developers want is they want a more abstracted more opinion opinion ated way to set up a to view us in infrastructure as code and preferably to be able to do it in whatever languages they want and not just in HCl or Yale or JSON so we've been working on a new project called the AWS cloud development kit or cdk and what the cdk does there's two things I want you to remember the first thing is that it provides language bindings for cloud formation so that you can authorize formation templates in imperative languages so you can alter cloud formation in Java or typescript or c-sharp or Python and then it compiles down to cloud formation so it allows you to do whatever you want to do in whatever imperative language you want we have more languages coming but you can take advantage of inheritance and in loops and if statements and all the that you would normally do in a regular imperative language the second thing I want you to know about is that it has a higher level set of components we you know not just the low language not just the low level bindings but also a bunch of higher level components that we call constructs and what these constructs do is they provide opinionated patterns for how to implement things sometimes this might be a single service implementation like the V PC so rather than the thousands of lines of cloud of CloudFormation JSON or HCl and terraform you'd simply have a few snippets of code in typescript or Java for how to implement a V PC in the best practice in in the most common way you can still you know configure it however you want but it gives you a default best practice way to do it or we would have constructs for implementing common patterns across services maybe you want to implement a pub/sub pattern using sqs and SSNs we have a construct that does that so super cool stuff that I'm excited about it's being developed in open source as well check it out on github related to this is Sam the ato server list model which is a sister project that allows you to do some of the same things it's higher-level opinionated ways to implement infrastructure as code but in a declarative format so if you if you like using a declarative format Sam is a perfect solution for that allows you to have opinionated ways to model your application and as Deepak mentioned we now have container constructs for the cdk as well and this is super cool because it allows you to have opinionated ways to model a container based application using for gate or ECS and it not only generates all the CloudFormation that you need it generates the i.m policies and all the other configuration files you need for docker and for builds and everything else all of it is managed by these constructs all right and with that we've come to the conclusion I want to finish by saying that here at AWS we are building a platform a cloud the best supports for modern application needs and we're innovating across the entire stack from the hypervisor layer to the application construction layer and everything in between I want to thank you for coming we are at the top of the hour so we're not gonna have time for questions here but David Deepak and I will stick around in the lobby afterwards for 15-20 minutes if you have questions please come up to us and we're happy to answer anything you also can give us feedback using the mobile app please take time to give us some some feedback thank you for coming and enjoy the rest of the day [Applause]

Channel: Amazon Web Services

Views: 16,149

Keywords: re:Invent 2018, Amazon, AWS re:Invent, Serverless, SRV325, AWS Lambda

Id: eXl6Bumksnk

Length: 57min 0sec (3420 seconds)

Published: Fri Nov 30 2018