Terraform Module for Cloud Adoption Framework Enterprise-scale Walkthrough

hello and welcome to dev radio uh we're talking more about the cloud adoption framework today here with wayne meyer kevin rollinson from microsoft and james anderton from hashicorp uh kevin is going to get us started by deploying the enterprise scale foundation uh for the landing zone with terraform did i get that right kevin is that uh what we're doing you did and uh thank you for the quick introduction okay thank you so we're going to go into a brief introduction to what enterprise scale is but first let's step into a quick demo and just show how quick and easy this can be enterprise scale is a reference architecture consists of many hundreds of resources what we've tried to do with the terraform module is try to break that down into reusable components the customers can deploy into their environments using as little code as possible so here we see a diagram representing the core components that we deploy through the module and this is what we're going to quickly demonstrate now we then have an additional section for management which we'll show later on today um and just kind of get started um this is just the readme from our module i'm just going to copy this code i'm going to switch over to the um azure portal where i've got a cloud shell session already open i'm going to go into a folder which i've pre-created and it's empty i'm going to launch the code editor within here copy the content in i'm going to save it just call it main.tf just as a starting point and then from that point i'm just going to run a couple of commands to get us started so i'm going to initialize the environment which will go away and download the latest version of the module and then the next step i'm going to do i'm just going to go straight to a terraform apply i'm going to auto approve it so it can run on through and i'm going to set some parallelism on this to get things running a little quicker for us today but that is basically as simple as it is um and just to quickly show that we're not cheating on anything this is my environment here at the moment where you can see we've just got the tenant group management group and a single subscription and this is the environment that we're deploying into now and it's as simple as that matt so uh back over to you okay and i handed over you in well fantastic that's good and uh so we kind of wanted to kick this off at least so things are running and then we'll kind of walk through and do a little bit of a positioning on how on where this all fits in um as uh as we've done in the previous sessions you know we're gonna cover off the cloud option framework just give you a quick sense of or at least a you know cover off what the actual framework is um where this particular module fits in we have done previous sessions on the adoption framework where we've spoken about enterprise scale deploying through the uh deploying through the different um through the interface and uh through through the portal uh jeff mitchell kind of you know walked us walked us through that it was great that was a great session we've also done other sessions where we've kind of given some more of the theory around what what enterprise scale is and so if you go to ak dot ms forward slash caf series calf series you'll see we've got six six videos in there the last one is uh the actual uh module that we did deploying and then session action number two was actually the one focused on just talking a little bit about what enterprise scale actually is um i'm glad that we've kind of got this kicked off with with kevin and and james because i think this is one that we've seen customers asking a lot of questions around like hey i'm using terraform today how do i go and stand up my environment how do i get my my basic kind of azure management groups and structure kind of established and i think that this is a this is a really good addition to kind of that the icd and their infrastructure is code that we can go out and run i've physically been involved with specific customers in doing this um and what's great is that you know we had i think it was literally two days worth of conversation with the customer um and it took them 45 minutes to an hour to get everything prepped and up and running and so we had this you know this their environment stood up using terraform ci cd they were over the moon that uh it was really that simple to continue to to deploy um to talk a little bit about the adoption framework you know we you know we really kind of think about this in in three ways we've got this business focus where we've spoken about strategy and planning what the importance of building out that plan having a cloud adoption plan um once you know what your backlog of work is and you understand all the assets and all the work that you can actually drive and deliver into your azure environment we need we need to establish that platform the platform is really around that landing zone and making sure we have some governance guardrails from from the outset enterprise scale is our series of reference architectures that we have today these reference architectures are built with the right you know best practices in mind where we can actually from there from the outset to deploy at least a minimum viable product for for governance so you know thinking about locations and thinking about types of resources thinking about um things like types of security elements that we that we put in place making sure that we are monitoring and logging in the same in the right way we have a common set of tools set up we also enable the environment that if you were looking to do things like hybrid management and hybrid environments using arc that is that is established there's principles and rules applied to do things like security monitoring using you know azure security center and sentinel and so when you think about enterprise scale it helps us kind of take remove some of the hardcore decision making that we know really holds customers up we've got these reference architectures in this best practice in place that we can actually go and deploy this code now customers will look at deploying these things in different ways as i said we've got enterprise scale using just the portal experience um connection into you know with github actions but the terraform module which we're walking you through today is absolutely brilliant and one that we get as i said before kind of many customers are asking us for questions asking us questions around hey i've got terrible today because i'm doing doing you know using this as my as my platform for ci cd and doing my infrastructure as code how do i use enterprise scale and so you'll see how we can kind of build that um as you would move down down the path within the adoption within the adoption framework the workloads are really the areas that we've spent some time on talking about before which is patterns and practices and repeatable ip around hey how do i do migration or how do i do some sort of innovation scenario or data you know a data project we've built some of these into various azure devops templates but you there's lots of rich guidance that we've actually built into the adoption framework itself that if you go in and have a look at um under the adopt pillar of the adoption frame which i'll show you in a minute you've got a series of you know areas of guidance around how do we do linux migrations or how do we do kind of sql sql migrations or kind of thinking about taking those open source databases that we might have how do we get those up into azure or right down to even doing things like hey we want to make sure that we are um you know building out some sort of analytics platform or embracing you know various uh devices and you know doing things like iot and so on we've got all of that documented for for you as well the last kind of two parts of the adoption framework already around management and and having the right guidance around architecting those applications of those workloads in the right way and providing guidance as practices on how to do things you know to build for cost efficiency build environments that are reliable secure um and can be operated in the right way the well architected methodology is absolutely phenomenal we've got some really good assessments within this space this is an area that we continue to invest time and resources into building it being able to go out and do those do those assessments getting recommendations on how to improve your applications and then the architecture and design of those applications is absolutely is absolutely key kind of thinking a little bit about enterprise scale for a minute um you know i think we have covered this off in previous uh in previous sessions but really is our best practice and an architectural approach to kind of the implementation of your landing zone now landing zone is kind of matrix right you're going to have that all up kind of foundation foundational platform that you stand up so think about your connectivity your management your identity making sure that the right structure to management groups where your subscriptions reside but then you can also have more kind of tactical landing zones which are going to be focused on specific types of workloads or specific types of applications um that we call these construction sets and so you will have things like hey i've got an analytics construction set which i may need to kind of deploy into or a wbd environment or an aks environment or even an sap environment and so when you think about your landing zone what we're going to be building today is that all our foundational environment and the all our platform but then making sure that we then tee up for later on downline where you can actually start to bring some of these construction sets in because you may have a very specific requirement to go and stand up something like an sap environment we've got code and we've got a you know reference architecture to be able to go and drop in what we've tried to do with the enterprise scale is actually provide a design that is that meets you know the bulk of customers we know that there's going to be certain decision points that we think that customers need to make once uh we've tried to make those help make those decisions um providing you you know guidance on what are the best practices um the whole point of building out enterprise scales to help drive that velocity this is proven it's built on a number of large scale migration projects it's built built on a number of customer engagements we continue to take feedback and we continue to build that in the other really interesting thing here is that we this is aligned to kind of engineering engineering principles and focus and future and so as the platform and as the environment the engineering environment continues you know we continue to evolve this will continue to be updated as well and then i think the last piece around this is making this prescriptive um and this idea of being out say hey when you want to stand up your environment whether it be a green field whether it be a brownfield environment you can actually use enterprise scale to stand up and this you might have an existing environment today that is running that has got workloads in we can actually stand enterprise scale up in parallel and on site and you'll probably see even in kevin's scenario today like he's got an environment that he's not standing up in the site on the side it's gonna start building and so you can apply this the enterprise scale principles think about this as being this maybe it's an evolution of what you've been building over previously that you just don't feel is quite up to you know up to par once that once this new enterprise scale environment has been landed and deployed hey we cannot start moving subscriptions and workloads across into it so it picks up these new rules and processes and policies and and so on and so from an i.t operations perspective hey your teams are now managing that infrastructure and those policies as code and the platform and the platform as code you've got your developers being able to build into these environments um i think we've got you know a good um well-rounded well-grounded kind of environment that is meeting that is meeting those uh those design those design principles that have been set out um we've actually gone out and we've built out four key reference architectures from a foundation perspective to get you up and going um i think the last time we spoke there were probably only three of these we had enterprise scale foundation at this hub and spoke we had virtual wan there is a new uh reference architecture that's just landed which is enterprise scale for small organizations and so think about this as being foundation plus some level of networking and connectivity that has been built in there as well each of these specific reference architectures are available in github you've got the code that's there or you can do a straight up deploy to azure we're going to be spending some time today as we've mentioned already on the terraform module that's that's available it stands up exactly the same kind of reference architecture the same the same structure that we have um obviously the steps to get there are are slightly different um it's uh i'm a complete newbie to terraform so i'm going to be the guy that's asking all these newbie questions to to to kev to kevin and james but uh you know as we as we go through this you'll see that it is pretty straightforward as i say sitting through a couple of customer engagements going through this it is super easy um and it's great to see the stuff operational and running once uh once it gets deployed so with that i'm gonna quickly just show you where to find this in the adoption framework itself uh we've got some uh i'm to give you a quick view and let's go to the adoption framework uh environment ak dot ms ford's adopt when you we jump into the environment go down to explore the framework and let's jump into the docs platform you'll see that under landing zones starting with enterprise scale here's your terraform module for for all things that you need i have put a link into the slide went into the slides as well so that'll take you directly to the hub repo but uh this will give you kind of a you know a step by step and some guidance and some background around around the terraform models go and spend some time taking a read through it's pretty it's a pretty easy read uh giving you some examples around uh the various examples that are there um the code that's available and then as you want to kind of you know get into it there is the get a wiki jump into that you'll go and find the code and follow the steps that that kev is going to kind of walk you through so feel free to continue to use this and we will continue to update um if you've got any questions or any feedback drop it into the comments window um or get hold of us at uh on the dev radio channel and we'll be able to kind of help out and uh and fulfill those needs so let me jump back to you kevin and see if that builders actually got to a point and we can yeah thank you and great introduction and overview of enterprise scale so thank you um so yeah so we're nearly there um i don't know if you can see my screen again um in fact literally as we flicked on there it has just gone to apply complete um so as you can see you know in that time that we've just been doing the introduction to talk about what is enterprise scale we have now deployed the foundations for enterprise scale um actually if we had done the management landing zone at the same time it would have actually completed in the same time because that usually completes in parallel with the management groups being put into place but you can see here just from that process there we've added 189 new resources into our environment so let's have a look in the environment and see if we can now actually see this in practice and this is where we have to wait for the refresh of the portal nice new logo okay indeed so here we can see now we've got an enterprise scale management group within the hierarchy so the subscription is still where it was before but what we've now built out is the base hierarchy if you compare this to the reference architecture that we've got published on the ems docs site you'll see there's a lot of kind of familiar names here for now within the terrible module what we've tried to really focus on is the platform elements um so the specific examples for landing zones of online sap and corp we've omitted from the module they're perfectly easy to add but rather than kind of pushing that on to a customer from day one uh we've left that open and what we've tried to do is really put in the bare minimum of what we consider our prescriptive setup um if we actually start to drill into here we can start to see what else is done because obviously 189 resources there are not 189 management groups there so what we've actually been doing in the background we've been actually pulling in a whole set of policies and these policies are actually added into the module dynamically on a so fairly frequent automated update process we pull them from the native deploy to azure templates which are used for the native arm experience that just deploys everything through the portal we actually pull them into the module and then we make them part of the internal library as part of the module functionality we use a thing called an archetype definition which is basically a way of defining the scope of you know what resources to deploy against the given management group and here looking at the route management group um what we've got here is a set of policies that you can see have been assigned and they're still waiting to actually assess the compliance if i actually drill down into specifically the definitions now and let's filter them down to just the custom ones what we can start to see here now is actually a list of all the different policies and initiatives that we've put into the environment now going back to the whole thing around prescriptive architecture what we're really trying to do here is encourage customers to create this custom route and within that custom route this becomes the point from which all resources sit under two reasons for that really is one it allows us to have somewhere that we deploy everything to that doesn't necessarily long term require full ownership over the entire tenant um the other reason as well of course is the fact that you want to be able to raise exceptions so if you want to be able to stand up for another management group at a different scope or pull the subscription completely out of the enterprise scale architecture and compliance you can do that still but within here we've got all of the policy definitions and initiatives and this makes sure that they're all available at any scope so we're not necessarily assigning them all here but what we're doing is we're making them available for assignment and if i just scroll down through here you can kind of get an idea of some of the examples so controls relating to network access controls relating to enforcing ssl connection settings uh we've got some rules relating to subnet management so enforcing things like putting an msg on a subnet or a user-defined route um and you know some other ones which kind of more core capabilities so things like turning on as your defender and controlling the azure defender settings for your security center to ensure that you've got better compliance over your resources and all of this kind of comes together to be able to help build up your environment and make it sort of highly governed if we now sort of just look at the assignments you'll see at the top level here we've created a handful of assignments now these are more to do with the broader visibility so think visibility before control and what we're trying to do here is really turn on diagnostic settings and make sure that whatever's going on in your environment all of that information is getting pulled down and stored into a central log analytics workspace if we now actually go back to the hierarchy and we go a little further down and i'm going to look specifically at management because this is the area that we're going to look to modify shortly you'll see here we actually have a different set of assignments so we've not only inherited the assignments that have come from our enterprise scale root management group we've also got this additional one here for deploying log analytics and this is actually here in the normal enterprise scale we do a lot of policy driven deployment but obviously terraform we have a slightly different approach with this and i'd say this is possibly more of a cultural difference um but terraform we're a lot more used to the idea that we actually deploy and manage the resources with terraform and as true infrastructure as code rather than letting policy do the work for us so what we've tried to do within the terraform module is actually bring those two pieces together so the module is designed to really ensure that the resources that we deploy actually are compliant with the policy um so there's actually a bit of a handshake there in the background within the module that ensures that those two things match up and we'll demonstrate that in a little bit so i guess on that point um let's go back to the cloud shell and we will modify the deployment and we'll actually now set up some of the management resources and while that's loading i'll just go into here another moment i just have a set of generic resource groups that don't have anything specific to enterprise scale yeah sorry a bit difficult to see at this resolution um you see there's various different resource groups in here but these are just my sort of general testing environment and this also gives us an opportunity to now actually look at the code that i deployed so if we look into the main.tf um a lot of this configuration is actually fairly standard setup so we've defined the block for our required providers just that that you know our minimum version we want is 2.41 of the azure rm provider and this is actual minimum supported version at the moment and that's based on the various different resources that we deploy and how we've configured them you can't actually go any further back than that otherwise you will get errors but some of these enforcements are also built into the module as well so if you didn't configure this the module actually helps enforce that for you um we've set up a fairly standard provider here which obviously in the current context is just using my login um so it will be running under my current user credentials um as part of the model um this is something we use quite consistently within our examples but i actually use the client configuration for the current provider session and that's quite useful for us because that allows us to pull out values that are not dependent on running terraform apply so we can pull out things like subscription id and tenant id and actually use them as inputs into the module without having to do a scoped deployment we've got two variables defined just one set of route id and a root name this is what we would kind of recommend as the minimum starting point for a customer fundamentally root id keys into pretty much every single resource you create so if you change this particular value you're going to basically redeploy absolutely everything this module creates because everything is in one way or another pinned back to that value so if we're ever going to talk to customers and talk about what is the one thing you want to get right from day one that is probably the most important bit for you to consider the root name this is just to set a more friendly name on the actual um the customer root management group and that can be whatever you want within the limits of the validation and on that point it's probably also just worth mentioning some of the limits that we put in place on the module you know we often get asked the question why are you constraining it to these limits when this resource type supports this naming convention and the simple answer is about insuring standards across multiple resources because we use this value and we join it with other things and we actually do other stuff with these values we need to make sure that we've got a set of rules that are much stricter than you would usually consider on those individual resources so that we can then take those and easily reuse them across multiple resource types within the module and when it comes to the actual module itself this is the module block doing the declaration so you can see it's literally as simple as declaring the source from the terraform registry with the version um setting the root parent id um now this is the only mandatory setting you could actually deploy it with just these three lines in the block and the purpose of this is just to give absolute certainty of where you want to scope the deployment now usually we'd recommend doing it against the tenant id which represents the tenant group group but if for whatever reason the customer wants to be able to actually deploy it at a different scope you can actually set this to the value of any existing management group that already exists as long as you're not going to exceed that limits of sex management groups within azure and then obviously we just set in the root id and root now so there's the the starting point um before we move on to making a few edits to this um do you guys have any questions yeah i think uh just the thing that android's hoping we can kind of go through this maybe towards the end again is just the the setup and uh where we initially went to i think just it'll be good for us for folk just to kind of see so maybe we'll go back and just kind of show that um i love the fact that you've kind of pointed out these you know specific areas of things that you should change and things that you know have that have those decisions on i think that's going to be that's going to be critical um making sure that as we build through this we don't have any specific gaps um and then i think the point at the end around the management groups and the different levels and you know that's that the the depth of your management group structure so being able to deploy this into any existing management group is great but be aware of you know of that of that six that's six uh six level depths i think is is a really good call out um i have got some other things that i want to kind of talk about a little bit about later on but let's let's keep going unless james you've got anything that's top of mind and james yeah we are we're lip reading right now this has been a great walkthrough so far cool thanks cool well on that note then let's let's go to your first question then about actually looking back on what i've actually done here and where i've pulled this from so for us all all parts start at the telephone registry you know this was a really critical part of what we wanted to set out and do here um and actually in the early days of when we first started discussing this we had a few internal conversations about how we wanted to approach this and we actually initially looked at whether to write this as a module or to even actually develop this as a provider and the reason we were making those considerations is the vision for this was really about building a data model and obviously to build a data model and work for the data model it's much easier to do in a full programming language um but what we didn't want to do was create something that then conflicted with the azure rm provider we wanted some of the complement complemented that um we're also thinking of it from the lens of open source contribution so one thing we really wanted to do is to be in a position that you know while we are trying to provide a prescriptive solution specifically for enterprise scale this is an open source project all the source code is fully publicly available and actually we want to encourage public contributions into this as well so you know in that regard we've actually kind of gone down the module approach to help with that because we feel there are going to be more people out there who feel a bit more comfortable reading and understanding terraform code rather than go it also helped us in terms of development because we had more skills within our team that could actually write terraform code as well um go was a bit more limiting for us in that regard and it's also then a bit easier to actually understand so people who want to consume the module can kind of really understand what the module's doing if they choose to look into it and of course that can all be done through github now in terms of where i've copied that code from admittedly from the github page essentially within the home page of this we've got the diagrams explaining what we're actually building with the module and what the options are this is all very high level information just to kind of get you started and give you a high level overview we've got this simple example within usage um so this is the example i've just run today but we do have more examples and actually to do that we turn to our wiki um so actually if i go to our examples page on the wiki which is linked down here but we've also got links further up this actually takes us to our github repository and specifically to the wiki where we have an examples page and within here what we've tried to do is we're trying to build up a library of examples starting off with some level 100 basic examples which are kind of like the really easy get starting examples and then we go through some level 200 which are doing a bit more customization on the module and then level 300 is really starting to get into some of the more complex scenarios that you might want to cover and obviously over time as we get more requests on the issues log and we kind of start looking at what it is that customers are actually trying to do with the module what we're going to try and do is we're actually going to pull that information into a um we're tracking all of these issues and we're flagging them as documentation requests we're prioritizing them and then we're going to actually build that into the wiki as well in the future um so yeah i mean on that note if if there is anything that you feel you want additional information on please click on that issues link um click raise issue and provide us with your requests that's cool i mean i just i love the fact that you know we we're looking for people who are doing stuff and saying hey maybe i've got a gap here or i've got something that is not quite meeting my needs that we can continue to even continue to finish that out so drop those links in cool thank you um yeah because one of the limitations we've obviously got with the module is it's very difficult for us to get any telemetry on this and to understand how customers are using it and what they actually want to achieve with it um so for us the issues really is a really important kind of communication channel for us to be able to understand what our customers want from the module hey kevin just something that's top of mind for me right now so i'm fairly familiar with the caf enterprise scale reference architectures how do we see and that you know i've seen and i've seen the terraform kind of piece in in play and obviously what you're walking through we get to a very similar outcome at the end of the day with the hierarchy and the management groups and your subscriptions and so on can you talk a little bit about that relationship between enterprise scale and terraform and do these run on two separate parts how do they do their potential come together help us understand that relationship for a sec so the real intent behind the module is to just create a terraform version of enterprise scale um so today i said we're building out the core resource hierarchy um we provide the ability to do the management landing zone specifically uh we have plans in place at the moment i'm currently working on one for connectivity so it's really about taking terraform and building a solution for that that enables customers to accelerate their journey to deploying enterprise scale so we should have parity irrespective of what of what you're using so whether you choose to go down the path of arm templates using kind of the enterprise scale modules or whether you're using canon terraform hey it doesn't really matter the parity is there correct that's right and from a team philosophy point of view you know actually one of the things that we talk about quite a lot with our customers is the thing that we really care about our passion is actually the architecture and the recommendations behind that so if you actually look at our documentation on the cloud adoption framework one of our real kind of focus areas is the critical design areas yeah that is our real focus and those are the bits that we feel passionate about how you actually deploy that you know that that's less important to us you know we want to make that journey as painless for you as possible and whether you choose to go down the route of using arm-based templates um build your own solution go with a third-party provider solution from one of our partners or the terraform module you know actually to us what matters is that the end result is customer success yeah cool so i've heard this other i've heard this other term the terraform cloud and i don't know james i don't know if this is something that you want to be in on or kevin if you kind of have a perspective on it like where where and how should i think about that in in what we're building here so i'll start and then i'll pass it off to kevin um so terraform cloud is our opinionated and managed platform for uh teams to be able to collaborate when building and deploying terraform so today you've got terraform and you saw how he was able to do it on a single person scale with just the run time on the cli and terraform cloud is the managed collaborative version of that so we have the um vcs driven workflow the version controlled system driven workflow where you can commit your code you know and you you and your team can work in that repo and then as you commit your code as it gets approved then it sends a web hook over to terraform cloud chair from cloud picks it up runs a the init and the plan and the apply for you and that's just the base you know the base model we have different tiers as well so you know if you if you have the governance tier we can also then apply some sentinel policies to be able to provide guard rails for those developer teams they don't want to have to worry about the corporate i.t security and all that they want to just write their their code and move on and deploy and so you can have those applied you know for them in place so that's kind of our opinionated management platform got you got you and so kevin and then so for you what you're deploying today guess anybody anybody could pick that up right from i guess what i'm hearing that you know uh that james was saying is that this is the you know it's based on the cli it's just available to anybody like there weren't any prereqs that were required in order for you to be able to go and use this right no the prereqs are quite simple on this and this was something that we tried to do with the module um and it actually shaped a lot of the decisions that we made because during initial development and i continue to do this today for my personal testing when i'm making changes to the module um i actually use terraform cloud myself my whole reference implementation is run through terraform cloud and one of the benefits this gave me was the ability to remove a dependency on my machine as to which versions i had installed i can actually use terraform cloud to set some of that stuff up i can securely store all of my secrets within there so things like my credentials to the environment i can store within there but i can also start storing things like some of the values for things like the root id and the root name so it just makes it a lot easier for us to get up and get started quickly some of the vcs integration points as well are really useful for us because it allows us to push out a new commit to our git repository and then that will auto build we can then use the terraform cloud to actually take things like the uh approval workflow which allows me to not only push those changes out quite seamlessly but also to actually review them first before they actually go out to the environment so yeah we see a lot of benefits from it um also just thinking about the module design um it allowed us to kind of really focus on what it is that would work that keeps the module simple um so actually we've managed to achieve what we've done to date without anything in the way of third-party additional providers or tooling or anything like that and working on terraform cloud was a really useful way to kind of help shape that because we were able to work within the limits of what terraform cloud could support giving us greater confidence that no matter where you use this module you should be good to go with literally a base install of the terraform executable cool that's super cool one one other great feature with terraform cloud is you saw that he's storing his module in the public module registry terraform cloud also has the ability to create private module registries inside of your organization so teams that have additional restrictions and regulations and don't necessarily want to just grab public modules they can also clone these modules into their registry and use sentinel to be able to lock it down so you can only pull modules from your private registry so it works really well gotcha cool that's neat all righty kevin what else have you got for us cool so i'm just gonna try and copy a few bits across here and hopefully not break things too much um so what i'm going to do now is i'm actually just going to set up some basics for getting started with actually configuring some of the additional resources and you'll guess from my formatting that i did have an extra line in this block um but what i'm going to do now is i'm actually just going to specify a subscription id for management which i'm going to just take the subscription id from the current context now this is because i have a single subscription but actually you can you know you can configure this module in such a way that you can deploy it kind of in a decomposed structure one of the real key design points of this was the data model and i keep mentioning the data model so i should probably elaborate a bit more on this um we take a very basic set of inputs from the user and there are some that are very simple and there are some that are a bit more complex um but ultimately we've built a data model that allows us to have a built-in library of templates and the customer can then bring their own library of templates and that's really what's given us the flexibility to make the module sort of not just customizable but fully extensible um so yeah we often get asked all sorts of questions around oh can i change this and i think so far there's there's not a single thing that a customer has asked for to change that we've not been able to change with the current design of the module it's just knowing how to do that so actually it's more a case of uncovering the documentation requirements behind it rather than the technical requirements um yeah when we take all the defaults you know certain things get pulled in so the subscription id what this is going to do this is not only going to identify the subscription id as being the management landing zone um it's gonna then use this as the subscriptions target for deploying in the management resources because it is the current subscription but it's also gonna take that id and it's gonna place it into the management group configuration for the management area so it will move this into the right management group within the hierarchy it will also build up all the data model for things like the policies so we'll actually see some of the policies get updated here as well and what we'll do is we'll go from having a placeholder value of all zeros within the log analytics workspace id to actually having the valid subscription id for our management subscription and that's really where some of the kind of the heavy lifting is done within the module to make the customer life a lot easier um because they no longer have to worry about making sure that the right settings are being pushed into those policy assignments because the module will now do that for you for the built-in policies that have a relationship to this so i'm just gonna save that i'm gonna do terraform apply again um i will set parallel ism again just because i know there's going to be a few resources changed on this i won't auto approve it this time though so i just want to kind of quickly review what this change is actually going to do quick question whilst you're going through this kevin um this is i get this question from a number of customers uh both on the enterprise scale side and then also on this as you were saying hey there's 189 resources or so that get deployed is there a way or an easy way for customers to understand what the potential financial impact is of what get of what gets deployed like what are billable services versus what i'm not great question um not an easy or simple answer um i i guess the shortest version of it is if you go with all defaults the cost is generally pretty low unless you start layering in lots of resources so by default we go with a lot of configuration defaults that match to our recommendations um so i think the main area to watch out for is things like azure defender so our recommendation is to turn on the standard profile for azure defender um in itself that doesn't actually cost anything it only costs things once you remediate resources that as your defender are then actually configured for so in itself the module doesn't deploy much that costs anything i mean management groups don't have a cost all of these role assignments don't have a cost um they're just part of the kind of the core offering of the platform but yeah if if you put this in and then start bringing subscriptions in and then you have resources going into those subscriptions then you can start seeing some costs okay cool to chime in here shameless plug for terraform cloud um we do have a tier where we have cost cost analysis and cost management and so having run this i actually can say that he's right it doesn't really cost anything i've got a couple of empty subscriptions that i was able to apply the uh landing zones to and also assign my subscriptions to those management groups and i think i paid a whole 39 cents well there we go great answer great um so so what we've got going on here um if i just quickly talk right here you see some policy assignments are being recreated here um these are all policy assignments where actually the subscription id plays a critical part in one of the input parameters um now obviously this is a kind of a that's called a limitation of policy assignments if we change the parameter values of a policy assignment we have to destroy and recreate it and that's what's happening here um you know the policy assignment in itself because these are all deployed policies it's not actually going to affect your governance at the point that you're making these changes um so you know in theory no one should be able to bypass the compliance rule just because you're running an update at this point um but what this is doing is making sure that those policy assignments are up to date and they're going to be fully aligned with the specific log analytics workspace that we're deploying here but it's much more than log analytics workspace we're actually layering in the automation account and linking those two together and then we're layering in some solution packs as well as your monitor which allow you to provide some additional governance over things like you know vm updates or change management and we've got a couple in there specifically around sentinel as well and so all that stuff gets turned on for you as part of this setup and if you want to change any of it that's absolutely possible as well so you can so there's a lot of granular control specifically within the management configuration settings um you can kind of fine-tune a lot of the core settings and what we try to do is align that with the same experience you get through the deploy to azure experience but if you want to do more heavy customization you can do that and this is where things like the custom library come in which is documented within the wiki right those resources have deployed so let's see if we can now see some of this okay so here we can see i've now got a new resource group here called es mgmt this is the resource group we've just created and here you can see the configuration of all the defaults that we've layered in so here you can see we've got a log analytics workspace called es bash la got the automation account here if we actually go in and look at the log analytics sometimes this takes a few moments to actually update with the link right but we're already there so you can see now it's got the link associated with the new es automation account as well and yeah as well as that we've kind of turned on a lot of the solutions as well so you can actually go into here and start looking at some of the things that have been set up in preparation for you and so here we've got things like um update management um again this is just taking a little while to update but once it's all updated this should all be configured for you um and that kind of gets you started on that side um if we just go back to the management groups quickly and i'll go down to the management management group you'll see now the subscription is showing as under the management management group whereas previously it was still under tenant route group and so that change that's one of the things that that's supplied and if i go to policy and assignments and i look at the assignment for deploy log analytics you'll see it has settings in here that match all of the pieces that we've got deployed there so the automation account is called yes automation um the resource group called the sm gmt yeah so you see how by using the module and deploying these things together what you actually get is that synergy between both policy and the resources that you've deployed which ensures that once you've actually done a compliance scan on this particular policy then you'll get a nice green tick telling you that you're compliant um i think just show another one as well um hopefully get this to the right scope let's go for the root level and again into assignments so we look at things like um deploy vm monitoring and you'll see here now um previously this would have had a placeholder value in it um i perhaps should have shown this as part of the earlier demo but um you see now you've got a valid value in here specifically for the log analytics workspace that we've just deployed um but what we've also tried to do is make sure as part of the customization if you want to provide an existing one because we realize there's a lot of brownfield solutions out there if you want to bring an existing log analytics workspace into the solution we actually support providing this value as part of that and then that will ensure policy gets updated for those correct values yeah this is to kevin i just you know look at customers that have actually had to build this from the ground up right and so you can just see you know five seven things that you've shown like some of those decision points are just so onerous that it's just taking customers so long to be able to make those decisions and like being able to have this kind of code built in as a reference architect architecture says hey here just go if you run this and land this and put the right you know these couple of key variables in gets gets everything deployed for you from solution packs to policies um i think that that is huge that is really really powerful now this is a this doesn't mean like we're still ncis compliant or miscompliant or or some other kind of regulatory meeting some sort of regulatory frame that's something we need to address after the fact right these are based on what we believe are core best practices um but you can now go and layer on some of those other or cis-based policies over and above is that is that accurate yeah that's the first statement i mean i think how we try to look at this is these are a set of policies that we recommend everybody should have yeah and then when you start talking about things like cis compliance or hipaa high trust you know pci combined you know all of these are things that the customer can come in and add on later and you know just know that's a really good segue into talking about some of the customization and extensibility of the module because what we actually do is we provide the ability to go and add on your own definitions so we talk about creating custom archetypes but what you can actually do you can also extend existing ones as well so if i just go to this example um what we can do is we can actually use a library within your module where you can define additional files for extensions and exclusions and this allows you to actually modify the policies that are assigned by an out the box archetype so in this particular example what we're trying to do is extend the es landing zones archetype definition to add a policy assignment for deny resource locations and it's as simple as defining this template within a folder within your root module and then you just declare that as part of your main.tf and in that case it's just setting the library path so you just tell it where the library is and the module's got the logic in it to actually go and find that do a name match on the fact that this is extend underscore yes underscore landing zones and it will know what to do with that and how to do that extension similarly if you want to create your own completely custom architect definitions you can put them in the library but also policy assignments policy definitions role definitions and policy initiatives are all supported within here as well and that allows you to really sort of start customizing your environment and building it up using a common framework if you like um for deploying not only enterprise scale but your broader management around your policy and governance that's really cool that's really really cool so one question i had about uh this whole thing was i've i've had several customers ask me you know there's a module for cloud adoption framework and then there's this enterprise scale terraform landing zone module can you tell me like what's the difference and what size companies should use which one well that last bit's a great part of the question um so the the cloud adoption framework module is a broader kind of module covering sort of the broader scope of parts of azure and deploying azure under the cloud adoption framework design um it also provides a kind of a recommended reference way of running terraform as well so it's got a built-in capability called rover which basically is kind of a ci cd pipeline ready um docker container that contains everything you need to kind of edit set up and run your deployments it actually uses a hierarchy based approach so the team that developed it put a lot of time and effort into making sure that you could layer in the various different layers of the architecture and so they start off with a base level where they set up your kind of ci cd environment with a devops capability they then move on to start layering in things like the landing zones and they actually use this module and the enterprise scale module as part of that deployment as well so you can actually deploy enterprise scale using the cloud adoption framework module it provides a lot more but it's also a very prescriptive way of running terraform so you know for customers wanting to do enterprise scale they have a choice and i wouldn't say it's so much to do with the size of the organization but more kind of a decision around how they want to run and operate terraform okay yeah that that makes great sense actually so a much more prescriptive way of doing it it's kind of like us with the difference between the open source and terraform cloud okay that makes great sense yeah like the i like the distinction that kevin of the not around the size of the organization versus the operations um because even you know you could be a really small a really small entity but your operations you know could be would be very could be very mature um so it's not necessarily that old size and ultimately that's where i mean i think a lot of the things that i talk about when we talk we talk the cloud option framework is helping customers change change the way that they operate and change the way that they think cloud is that that model it's not a hey i'm going to take some vms and throw it up there like what's that as a great solution and it is a very valid kind of a real scenario for many customers you know there's other things that we can that we can do and how do we manage and optimize and streamline our environment through the way that we do modern operations and modern management so one other question that my customers been asking me is how does this work with the brownfield how do we integrate this when we already have subscriptions and management groups another fantastic question um so the enterprise scale architecture is designed to work alongside brownfield um it doesn't have to be a fully greenfield environment um what we tried to do though is actually look at the migration approach rather than the deployment so both the native and the terraform approach are both designed in such a way that you can actually deploy this alongside any existing infrastructure because of the way management groups work until you move something under a management group it's not affected by the management group so we talked earlier a bit about the depth limits um you know when we talk about management group limits i believe the current maximum for management groups in a tenant is something like 10 000 management groups so unless you have a very very large number of management groups you're not going to run into any problems with limits so what we try to do is encourage customers to actually deploy this alongside whatever they have today scope it to the tenant route group as per the recommendation and then start looking at the policies you can do an evaluation of what you've got today in your current environment and look at a way of migrating that across into the new hierarchy and then once you're comfortable that you've got all the right policies in the right places and you've planned out your landing zones and then what you can start doing is start taking a subscription migration approach and this is really just taking a subscription that's in one management group and associating it with a new management group and theoretically speaking the only impact should be a change in the policy profile obviously most policies only apply enforcement at the time that you try to make a change so if you're introducing a new policy that's maybe going to deny a resource that already exists but if that resource already exists it can't deny it so you're not going to stop things working but what you're going to start doing is raising red flags on things that are non-compliant and then you can start using some of the newer capabilities within policy assignments like exemptions and so that if you have specific resources that you're going to want to exempt from the policy you can then do that on a case-by-case basis um but it you know it shouldn't have an impact on running resources that's great that's a great uh clarification because i know that that's a lot of that's a lot of concern that i get a lot of the times that customers will say hey i'm moving things in what are these policies going to do what's going to break um and so that that's another kind of just i guess another roadblock that we have to kind of overcome as we go through these things yeah and the main one would be so where we've got deploy if not exist policies just ensuring that there's no kind of conflicts on resources but i think the likelihood of that is pretty slim in most organizations and most of that stuff that you can work around and to be honest if there is a conflict then chances are you've already deployed it so deploy if not exist is not going to kick in yeah speaking about deploy if not exist is there plans for automating creating and assigning subscriptions because i know that's kind of been a sticky point with some of my customers do you mean as part of a subscription factory or in terms of when you're moving subscriptions having the dyn policy kick in so you have to assign a a subscription to your management groups or landing zones for these policies to be applied but say i want to create a new subscription and apply it to the landing zone all at the same time is is there plans to be able to create subscriptions on the fly okay yeah so not within the module um and the reason for that is the subscription creation processes so we feel kind of has its own unique set of challenges we see that really as an input into this module so you know when we think about terraform itself that can create subscriptions um i'd have to double check the exact setting options for that but you can with the subscription api put a subscription straight into a target management group it doesn't have to go into the default um which obviously usually is tenant root group but you can configure that within your tenant to be a different management group um the only thing you have to watch out for though is because this module is enforcing governance um one critical part of that governance enforcement is the fact that the subscription should be under the right management group for it to inherit the right policies um we also enforce the placement of subscriptions in the right location so if you start dropping new subscriptions into the management group hierarchy without making the module aware that those changes have happened then you'll find that the module the next time it runs will try to pull those subscriptions back out and send them back to their default home does that answer your question yeah yeah it knows cool so i know we're just about at time um kevin is there anything else any final thoughts or anything else that you want to show before we wrap up i do want to make sure that we take an opportunity just to do a quick recap on what we've seen and what we've shown um so that kind of go away and kind of no way know what the next steps are sure um i mean in terms of sort of just general um you know one thing i would ask is take a look at the module on the registry have a little bit of a look through it um give it a try you know as james has mentioned the actual cost of running this is very low um and we can run this in an existing tenant alongside other so resources without having any impact on those resources um so give it a try let us know what you think um please provide any feedback via the issues and uh yeah if you've got any questions then reach out to us there as well cool and do we one last question do we have any training kind of on a docs on the learn platform around this or is there any other training that uh the hashicorp team have actually put up that can actually help that somebody can go and walk through the microsoft side i don't think we've got anything lined up for that yet um i think it's a great point i think sort of dms learn is something that we should start considering and put that on to our plans uh for now we're putting everything within the wiki um so yeah just take a look at the uh the documentation we've got there and if there's anything you have a specific question on then yeah to the issues cool i think kevin uh so james anything from from your side on the hashtag side any kind of key training or learning material that folk can actually go and take a look at um there's quite a bit of material on the learning site about how to integrate modules and and how to just get started writing terraform um there's nothing specific to the enterprise scale module but it it's pretty easy to to take our module integration tutorial and and use this go ahead so i was going to say something else just worth mentioning um not so much training material but we are actually in the process of developing an accelerator repository and so it's going to be a template repository that will sit alongside the module repository and it will have a reference implementation in it one which you can either run locally from your machine but we're also going to use a lot of the um the lessons we've learned from our test framework um because actually within the module we do a lot of automated testing within ci cd pipelines using a combination of github action and azure pipelines and so we're going to take some of that learning to give a reference implementation for how to run terraform in a pipeline on github or ado um but that will also give a much better insight into some of the customization and how we would recommend setting up your root module to deploy enterprise scale and that's going to include doing things like alias providers so you can actually do the whole multi-subscription deployment and that should hopefully make things a little easier to kind of get started at a more advanced level then yeah that'll be a big deal i know a lot of people will be interested in that part fantastic so we've really covered a lot of ground today from kind of setting up showing kind of the initial the initial build um updates applying kind of updates and changes um obviously all of this has done through cloud shell you could you know i think it's obvious to say that this could all be done through kind of vs vs code or something as well so um i think really really powerful kind of walkthrough thank you um thank you both um with that i'm going to kind of hand you back to matt who can do a quick close out i know it's slash cafe series we'll keep pushing people there matt any final thoughts on your site no this has been super helpful stuff uh kevin really appreciate you taking the time to show us james thank you for joining and i hope to have you back and uh can't wait to see the the next thing that you have to show us so thanks everybody okay see ya

Channel: Microsoft DevRadio

Views: 6,606

Rating: 5 out of 5

Keywords:

Id: 5pJxM1O4bys

Length: 65min 17sec (3917 seconds)

Published: Tue May 25 2021