Deploy Infrastructure As A Service with Azure Resource Manager Templates by Will Anderson

good so my name is Willie Anderson I am the incoming CEO for the DevOps collective we are 501c3 nonprofit organization that runs this conference primarily I'm also a cloud solutions architect for a company called core tech services out of Michigan no the Canada hoody wasn't a mistake I do live there but I am American so it's a little bit confusing I'm also a cloud and data center management MVP with primary focus on PowerShell and Azure so I do a lot of Azure automation I write a crap-ton of azure arm templates so that's why we're having the talk today and then if you want to find me out on the Twitter's or social media pretty much anywhere just gamer living well is my handle for pretty much everything you can even find me on xbox so the azure vision you know since they moved to the the azure RM model off of the classic model everything has really been geared towards star is code regardless if it's configuration infrastructure even governance policy you're starting to see a lot of the stuff as far as like our back you know role based access and controls stuff like that a lot of that stuff is designed to be automated through templates and it's not quite there as they're adding new technologies and you know the the templates kind of follow in short suit but yeah they're constantly iterating through that so you're seeing a lot more of those things coming into play so why templates well they combine the benefits of underlying you know armed with the adaptability and readability of JavaScript object notation this makes things very simple is anybody ever tried you know creating like an azure VM purely through PowerShell to give you a little bit of a difference in the amount of code required when I did a blog post a couple years ago when as your classic was a big thing and you could actually deploy a VM with like five lines of code nowadays I know that they've come out with some new commandlets that make the process a little bit simpler but up until recently with Azure RM they required more than 70 because you had to define how every single object was configured this is so the the JSON templates actually made things a little bit simpler but the language is a little bit kind of intimidating too and that's that's kind of what we're gonna be working through today but it allows you to deploy topologies and workloads consistently you can make things item potent so once you have a JSON template you can stamp it out and basically any region globally if you're doing it right and it's going to look the exact same way across the board you know you manage all of your resources in an application together in resource groups we're gonna be talking about some of the best practices around that you can apply our back and then use tagging to to streamline things such as building billing and asset management which is highly important one of the things that you need to do when you start working on Azure templates is to think item potently so you know being able to go from that one-to-many model and you know this is very much that pets versus cattle mentality where you know the when it comes to template design you really want to be able to create a template in a way that not only is it easily easy to rapidly deploy across multiple regions but also be able to destroy that instance and rebuild it with greater ease than having to go through and following a bunch of manual steps to do so thinking imperative lis is another advantage to Azure templates so you know arm templates are designed to define the goal state of an application how do you want it to look regardless if it's configuring load balancers virtual machines traffic managers storage configurations and stuff like that applying des configurations you know everything all together is empirical you know exactly what you're putting into the environment you know exactly what the dependencies are the order of operations everything has to be very well outlined so getting into some of the best practices when it comes to templates one of the things that I bring up with a lot of people is choices versus decisions when you're talking about parameters and variables these are the choices versus the decisions parameters our input thing you know things that your end user is going to be inputting and then the variables underneath are the decisions that you make based off of those user inputs this is very important if you look at is anybody like built a VM in Azure and then downloaded the template if you notice in there it's got all of these things in the parameter fields like the computer name and the IP address and the user accounts blah blah blah blah blah blah blah the more choices that you give your end user to make the more likely they are to screw something up so one of the big goals that you have when you're creating these templates is to provide as few decisions to your users as possible or choices to your users as possible and then based off of those choices make decisions for them computer naming conventions what automation accounts to use how many VMs to spin up for dev QA and prod environments things like this can all be made as decisions in the underlying template basically what I said one of the other things that is very much key is user accounts so working having done a lot of work for fortune 500 companies where they were basically environments where they gave subscriptions out to their developers you wouldn't believe the number of instantiations we saw where basically the admin account was admin and password was password that's bad so those are things that you want to be able to take away from from your users and be able to make the right decisions the azure key fault this a really good job but I'm a little bit more partial towards as your automation account creds primarily because an azure is are you guys familiar with the two I'll show you guys this a little bit later but actually I could show you right now but basically what it comes down to is as your key fault will gives you the ability to an obfuscate the password not really something that I want to be able to hand over to my admins to use oh hey a connection to the Internet is a good thing okay who's got the Wi-Fi access called Homer because that's awesome waiting to login yes connect to Wi-Fi okay so let me just drag this over here real quick all right so if I bounce into my automation account here did you go oh stop I'm a little click happy this morning I've had too much coffee I guess I think it's going off-screen let me fix this real quick where'd my scrollbar go here we go thought I had a key bulb in here anyway with the key fault you can actually infuse gate the password whereas when you're using as your automation account credentials those passwords always stay completely hidden away so even though I'm the owner of this subscription if I go and I look at a key you know I'll jump in here and you'll see the password is obfuscated and I have no way to be able to take that away so kind of a big deal what it gives you is the ability to have like your sequel administrators or your app administrators anybody that kind of uses a service account you could give them the access to the automation account to be able to apply those credentials and then as an administrator you can use the credential to deploy the environment without necessarily knowing what those credentials are so it's a little bit more secure so that's you know pretty important so moving along here one of the one of the big things that I get into is decision-making with complex variable constructs is anybody kind of playing around with this right now excellent so who here is familiar with a hash table or actually better yet can configuration data in DSA okay so essentially what a complex variable is because in an answer template basic variables are just key value pairs right essentially what you're doing with a complex variable is you're creating an object you're defining those objects based off of key naming so you can see here like I've got my VM configuration NP denotes non-production PR denotes production and then I have these tables underneath that define like what I want my web server to look like given a certain configuration now the way that this is called and I'll be showing you guys the scan code the way this is called is you have a variable that references the complex variable that you want to grab and then the input parameter and then when the JSON template runs it looks at the input parameter goes through this list and says you know for say for example if I'm in a production environment okay I need to look at the pr table and then I need to grab the object references for that and then you can dot source this in your template this allows you to do a lot of decision-making on behalf of your user saying maybe for your non production environment you only want like two web servers and a load balance configuration but for your production environment maybe you want four maybe you want bigger servers in the production than the the smaller one or vice versa so this allows you to actually make a lot of those decisions underneath clears mud yet awesome one of the other best practices that you want to follow is keep naming simple naming standards simple I know that there's a lot of people out there that want to have you know some randomized you know string of characters or they're like you know ABC this many zeros and then denote it with a 1 or something to make it completely you know unreadable or you know yeah I've seen some weird and crazy stuff but you want to keep it simple and standard across all of your resources the the reason why is because you want to be able to easily identify what resources are connected to what so if you have a computer naming convention where it's maybe a couple letters to define what application it is you'll have like dev QA or prod in there so you know what what environment level it is you might have the region in there denoted by a couple of characters so that way it's easy to identify what region you're looking at without having to look at the resource group and then maybe some numbers but then you'll take that base naming and you will apply it to like your network interface for that VM so you have your computer name Plus like NIF you might have a load balancer and just call it lb you'll have a public IP and just call it pub IEP but this allows you a standard naming across the board one of the other things that I tend to tell a lot of my customers is that you want to avoid hyphens and special characters the reason why is because now you're creating you know VMs can take hyphens but well is it I think storage groups can't yeah storage groups can't so now you're having to create different naming conventions for all of your different objects you're adding complexity and as we all know complexity breeds catastrophe you also have to be cognizant of naming limitations so Windows VMs can't exceed more than 15 characters still even though we live in a 64-bit era don't know why but that's the way it is storage accounts on the other hand can't exceed 24 characters so you need to be cognizant of that when you're making these rules up you also have to be aware of things that require globally unique naming storage accounts public IP is Web Apps stuff like that and you know the global naming is not just to your organization its global period I remember the first time I went to create an azure RM storage account and I thought it would be really you know cool and nerdy and call it vault 101 nope somebody already took that wasn't in my organization with somebody else globally but apparently there's you know other nerds that are faster than I am template parameter file versus template object this is hugely important guys I see this all the time actually if you go to the Azure QuickStart templates you see it rather frequently where a lot of people have parameters dot JSON files that they put all of the the parameter inputs number one you're using a static file so you've already thrown idempotency out the window number two one of the most common mistakes that I see is that oftentimes there will be something in the parameters to pass credentials so what do I see in those parameters JSON files username password great thank you for putting it in plain text file that anybody can read so in you know adding additional layers of security on top of these parameter files it's just a pain in the butt you know if you want yeah you might have to store it in a blob store and then you have to deal with sass tokens and then you have to create the constructs in order to be able to generate those sass tokens and then you're managing I mean it's a complete pain whereas the other way that you can go this template parameter objects which are fantastic so if you're doing deployments via PowerShell you can actually do that in a parameter object and you know deploy it on the fly I'll go ahead and pull up my visual studio code here and show you how that looks as soon as I clear all my customer stuff off the screen just go ahead and drag this over here so Wow this green stuff is really cool can you guys read that in the back by the way okay so basically what we're doing here is I'm setting up for an account deployment kind of using that same template that I was showing you in the screenshot and you can see here that I have a parameter object that I'm generating and the parameter object is actually doing a couple of things for me it's grabbing the location by getting the resource group that I'm deploying to it's got the environment you know that NPP R that we had defined for the complex variable constructs and then I also have the automation account like registry key in URLs that way I can go ahead and assign this thing a node configuration and desired state config and I can do this on the fly so you know as you know maybe I have some this automation code in my CI CD or whatever to do my deployments for me then basically it can just go off and say okay I'm going over to West US so we're gonna create a resource group there grab the object import that into the parameters and then go ahead and do our deployments and then also maybe have some automation to define what automation account that I want to hook up to if I'm doing maybe multi geo automation accounts so this allows me to go ahead and do this stuff on the fly instead of having to generate a bunch of static files that I have to to go ahead and build once again clear as mud excellent moving on so this is a huge conversation one of the probably the most frequent questions that I get asked on a regular basis is what about nested templates are linked templates and this is kind of one of those things that I would love to have a conversation with you guys about so the the the basically as your templates give you the ability to link to child templates so maybe you'll have a standardized template for how you want to BM the look or the network guys have a standardized template for how the network looks I've never actually found in my experience a valid use case for those linking of templates one of the one of the most common questions or one of the most common statements I hear as well our network guys want to be able to define the the template for creating a virtual network they wanted to look a certain way and have certain controls and etc etc yada yada yada which is great I think that all of the organizations should definitely be involved in maybe creating the standards for how those templates are built but not necessarily creating a template and the reason why I posit this is because what if you're relying heavily on Azure templates to do your deployments and you're pulling these linked templates in and then maybe your network team or your storage team or somebody like storage decides you know what instead of v1 storage we're gonna go to v2 storage and they go ahead and update their template then any template that relies on that as a child or is a link template is going to have to be tested and validated or if you're doing automated deployments they may just all of a sudden blow up on you now that you have all these things relying on these link templates anytime that wants to make a change anyone wants to make a change all of the subsequent templates that it relies on have to be reviewed and now you have a review panel that is going to be far far worse than your change control board that could ever do so I tend to say don't use link templates the the what I usually say is have your organization define the standards of how things should look and how they should be built and then make sure that the deployment teams are actually following those set standards is anybody here using child templates awesome that's that's what I like to see no handsome so secrets and source we kind of talked a little bit about this don't do this instead do this this is where the automation account credentials come in very key to play because yeah basically when you're when you're calling that into a template it's pulling it as a PS credential it's all handled at runtime so if you're doing it through actually ideally through an azure runbook because that is done in that instance and as soon as the automation runbook is finished that vm goes away and then anything that could have potentially been captured in memory is gone but definitely don't leave it as part of the template file because anytime that you have a static file anywhere I don't care what kind of security security you're laying down on it anything can be hacked API version is a variable this is the one that drives me nuts I I will slam your laptop lid down on your fingers if I catch you doing this but it's also probably the most common thing that I see in the Azure QuickStart gallery here's the problem API versions if you start looking through the schemas are typically specific to the object for which it is written so an API version for the storage account is not necessarily going to be the same as VM or anything like that I've actually grabbed QuickStart templates where the API version was hard-coded it is a single variable and passed you all the subsequent objects and go to deploy that template and it completely explodes on me because there's only one object in there that it actually goes for so quite path yeah and this is this is the the most aggravating mistake that I'll see API versions you don't necessarily need to update them for every single iteration that comes out one of the things that I usually tell my customers is that if there is a you know like if you're if you're doing an iterative code change maybe you want to do a deployment with the latest version of the API is to make sure that it works with your application or maybe there's some features in the new area a P I that you want to be able to leverage but other than that you're not going to be modifying or updating it that often so you know just put it you know set it as part of the resource leave it alone unless you want to update it another thing that I often see especially in the QuickStart gallery and you're gonna hear me picking on that a lot is people using concat instead of resource ID really this deals more with behavior with templates so essentially what it comes down to is at runtime when the measure environment is compiling the JSON template for deployment the way it behaves especially when it comes to depends on is that if you have another object referenced in your template and depends on as a concat it expects that object to already exist whereas if you go by resource ID it's going to look for that resource ID in the list of objects that you're defining going okay it's not created yet but I know that you know once it's created I can start working on this whereas with concat it's going to throw a bunch of errors and say it doesn't exist and then your template deployments going to fail so be very cognizant of where you need to use it when now I will tell you that one area where this kind of starts to go off the rails is in the networking constructs especially with like load balancers where you're actually calling the resource ID and then concatenating it with other string information in order to be able to be able to reference the full string I have no idea why they do that instead of just dot sourcing the property but you know that that's one of the things that you have to look out for now I'll be walking you through some of that stuff unnecessary dependencies so my animations aren't working there we go keep it clean and simple don't over complicate things basically the the way resource dependencies work is you know it's your order of operations right so and I've seen a lot of templates where or people get frustrated because they start building a resource dependencies and kind of start creating like the circular logic errors where you know they go to the deploy the template they don't know why it's failing but then you look at resource dependencies and they want to like create a V net and the network interface is reliant on the the V net and they want to attach the the public IP so they make the public IP reliant on the network interface but they're also making the public IP reliant on the V dot well that's nice big circle and it's not going to work so try to keep it simple and clean this does get a little bit muddied especially when you're talking about VM deployments because say maybe you might have to have an ro DC in the environment so you have to build the objects for that ro DC maybe apply the DSC configuration but the way the like the DSC extension works is it returns a successful when the extension is installed not necessarily when the configuration is completed so even you know so what I'll often see is people are stamping out the machines and then having people you know having the subsequently machines waiting on the creation of the ro DC VM and then having their subsequent DC configurations wait on our ODC configuration well then there's no domain controller there because that DSC configuration hasn't started you know started processing so often what I do oftentimes what I do is I don't have those VMs I actually start creating until the DSC configuration or DSC extension installation for that ro DC is complete am I actually going to be showing you some examples of how that works but try to keep your dependencies to an absolute minimum and keep it as streamlined as possible because that's oftentimes where I see a lot of thing mistakes occur so how many people have heard you know people online say oh don't create your templates from scratch you know go ahead and pull it from the QuickStart galleries and then just modify it as you need it stuff like that show of hands yeah really great if you're good at coding or if you're already familiar with JSON if you're like me you weren't when you started and where we're all you know admins and engineers and stuff here right we're used to understanding the nuts and bolts of how something works so I always say that basically use the QuickStart gallery kinda as a starting point don't use the the QuickStart gallery as production-ready code but most importantly take the time and build the stuff on your own because you're not going to understand the behavior of these template resources until you start playing with it and experimenting with it if you're relying on these QuickStart galleries if they work right off the bat which I want to say about 65% of the time they won't you know you're not really understanding the underlying constructs and it's very very important that you do because if something breaks sometime down the line you need to be able to read into it understand what the expected behavior is and then be able to trace back why is failing also like I said the QuickStart tell templates tend to not follow best practices they are almost never item potent and so if you look through the QuickStart gallery you're gonna see templates that are gonna have 50 billion parameters no I don't want my user to make decisions so you know try to build them on your own try to make them as an idempotent as possible authoring best practices I won't necessarily slam your laptop lid on your fingers for not following these but you know my I might twitch a little bit if you don't use camel case so this is one thing is PowerShell people we're always taught to use you know capitalize the first letter or if it's an abbreviation make sure those three letters are capitalized not really the the kind of standardization for authoring templates goes it's you know the first word would be lowercase the second one is Kamp capitalized the this is more just so you don't arc the programmers don't specify locations so templates can't be used to deploy resource groups there's something that you use to deploy to a resource group so a lot of these objects unless you're reaching out to something that re exists are going to be pulling into that resource group you get more flexibility and more item potency if you use the resource group that location object because now I'm not reliant on a hard-coded resource group location it's whatever the location I'm deploying to it's going to automatically pull that location then specify at my objects tags number one allow automation to create your tags on the fly so you can use the you know you can do this a number of different ways you can pass it in as parameter objects and then have your CI CD do the decision-making for you you can go ahead and build tags on the fly using automation but most importantly tag your resources I can't tell you how many projects where they've paid me stupid amounts of money to go through and come up with some kind of automation to go back and tag resources because they didn't know what or to build what objects to so you want to start getting ahead of that right off the bat use outputs so this is super super important you can actually there's a construct called outputs that output certain data so like if you have public IP addresses or you have like asset management and you need that computer name or those object names you can actually leverage that in the object output and then be able to pass that to additional automation in order to be able to register that information wherever you need it to go so I Pam or asset management or what have you I'll show you how those constructs work so before I get into demo time anybody have any questions I know it's Tuesday morning and we had a lot of drinking last night but come on somebody's got to have a question there we go so that gets into the whole child templates discussion and I don't like to do it primarily because the purpose of a template is to define what that specific environment is to look like so you're painting that picture right that picture is going to change between instance to instance so like you know I might have one application that needs to be painted a certain way another organization or another group is going to have their application paint in a different way and then you have to manage what happens if one of those child templates changes and that creates a lot of unnecessarily complicated unnecessary complication like I said it it will make your change management you know process look tame because anytime you want to make a change to that child template you have to go back and revalidate everything that it's dependent on so I say you know it used it to kind of like a DSC configuration where you know you're painting the picture of how you want a box to look the arm template is how you want that specific application environment to look so in that way you know if they decide to update the standards then they can go deal with the app owners to update their stuff but they don't it's not incumbent on that specific organization to go and harangue everybody or they can just say hey going forward this is what the standard is leave your current environments alone so it's a lot easier to manage from from that perspective anybody else the template that I have will create its own domain so it's kind of one of those things and I'm a little bit proud of it creates its own domain controller and stuff but if you're talking to like connecting as you're a was it as your Active Directory domain services I don't have a template for that yet that primarily because it just went in like GA not too long ago and I haven't had any time to play with it yeah and I'll show you how I do that any other questions good work it depends and I'll show you the what the the examples look like my template actually let me go ahead and start queuing that up I have you guys until town right 10:45 awesome even better I don't have to talk fast to make your eyes bleed I will be making example code available to you guys by the way so the first thing I'm going to do is I'm actually going to do a live demonstration for you however I'm not going to guarantee that it's going to work okay and here's here's an example where I'm actually grabbing a admin password and user from my key vault basically I'm gonna just kind of walk through what we're doing here so you can see here I'm you know giving a base name to my environment so this base name is actually going to be part of the decision-making going into my naming conventions and whatnot but the user doesn't actually see it because the base name goes into the resource group and then the automation knows to pull that base name off of the resource group name and start generating stuff you can see here I'm grabbing my automation account I've specified my automation account name and resource group and then basically the where is it you can see the automation account name here so I'm going to be getting that see actually no that's old code deployment oh okay I guess in this case I am using a key ball so we'll be seeing how that polls now interestingly enough and this is going to be getting into like the automate or the azure DSC stuff the azure DSC stuff actually grabs from the credential store so that that's something that I can show you a little bit later if we have time but I just wanted to show you how I'm pulling all this information then it's all passed in as a hash table and then it's going to be given over to the new Azure RM resource group deployment as a template parameter object so I'm gonna go ahead and kick this off real quick hopefully this doesn't explode it's gonna probably ask me for a key habit it just its habitual from you know way back in the day so I'm gonna go ahead and trigger this real quick but you'll see here I have verbose tagged on to these part of the reason is because if you don't use verbose you don't actually get a return until anything's completed and even then you get a null output so like if I do my test right now cuz I took out verbose oh if I had taken out verbose you just get a null output at least with this I get some kind of a return nice warm fuzzy feeling that it actually did the check especially since I'm using the s code which sometimes gets a little bit flaky on the PowerShell output and then when I passed it over to the new Azure RM resource group deployment this is how I look busy at work and you also just realize that this is being recorded so now my boss knows that this is how I look busy at work but basically what's really nice about the verbose output in here is it's gonna go through and tell you exactly what it's doing and where it's at in the deployment so it more importantly than just looking busy at work it gives you some semblance of where during the the configuration deployment you're at whereas if you're not using the verbose output you just get a hung console session until it completes and then returns its output data so I definitely recommend using verbose so let's go ahead and take a look from a template perspective as to what we're doing here okay so actually I think that extending my screens is causing some issues so let me go ahead and just fix this real quick okay that's better can you guys still read that up there or do you want me to expand it a bit better awesome okay so you can see here that the input parameters have you know some of the decisions here so I'm looking for the location I'm looking for what environment here I'm looking for my automation account registry keys and URLs what my admin user and password are much like you'll see oftentimes in one of these QuickStart templates the difference is that because I'm using some level of automation I can actually add a few skate a lot of this so maybe I only want to give that user like an input location and what environment level that they have and then everything else is passed via my automation so there are a lot of ways to obfuscate a lot of that stuff so you only parameterize what is absolutely necessary try to automate as many of those parameter inputs as you can and then give as few choices to the end user as possible because we know end users we humans humans make mistakes and mistakes are been going through then we start getting into our variables you know I have a base application name and this base application name is actually going to be part of my base naming convention then you'll also see here I've gotten in what I call an environment name instance this is something that I actually kind of add as a standard to all of my templates because when you start getting into global deployments and you start dealing with like as your Germany as your China the base URLs are different so sometimes it's good to have those tables when you're doing those global deployments so you know hey you know this is this is something that I need to add into my decision-making but you'll also see here that I'm taking like this East u.s. West US and north-central US and I'm you know basically converting those into smaller character sets that I can actually into my naming conventions this is really important because I embed that location information into things like my VMware I'm limited to the 15 characters so and you can see here the difference between what a regular variable looks like and that complex variable in the way the complex variable works is I have this thing called environment reference and you can see that basically what I'm doing is I'm saying look at this variable table environment name instance that I have here and then any table that matches this location input I want you to grab that as an object so it's going to look through and see okay I'm deploying to like for example East u.s. and then I have then automation knows to use that instance name in that region URL base as part of its decision-making process it's that cool and I can actually embed all of these decisions in into the code and the user never sees any of this stuff and get a little bit crazy here I start using like base name instance and putting together all of my naming conventions in a single thing this is because I'm lazy and I like to forget things so basically if I can just call out base name instance dot storage I know what's gonna use that base name operating systems I do it for that once again I'm lazy and like to forget things so all I have to do is just remember what operating system I'm calling and then put it all together from there I have Linux in here just to kind of show that you can actually call out different operating systems in the same complex variable set so if you have a mixed environment where you have some Linux servers doing some stuff and some Windows servers doing some stuff you can pull from that same table I'll show you how that works in the the automation as well I create my domain name on-the-fly so you can see it here and then of course this is that VM config that I was telling you about so mine on production servers I have everything separated out by role and then when I start building those resource objects I just dot source whatever property I need it's a role server web server that VM size dot name dot DSC configuration I don't have to remember it because it's up in here where this is also advantageous is oftentimes you're having to use those same values 50 million times and your different resources well now I only have to change it in one spot if I have to update it so it makes your code a lot easier to update just don't do it with the API version go for it max now so what this does is it assigns the particular Moff that you already have in your Azure automation account so you can see here I've got composite config domain controller basically what I did was I did a composite config had all of my role servers in there and then it generated all these different moths called you know composite config dot roll and then I just assigned that particular configuration to the box so and actually I have a webinar on how to do that will do here you can see the production instance so basically the same thing it uses the same kind of reference variable constructs so you can see here I've got a VM config reference to call out that complex variable that I created and then based off of the environment it's going to go and pull that particular table and then I start dot sourcing those resources later in the configuration and we're going to walk through how this works here's my DSC local configuration manager stuff if you had and and it's a complex variable yes but if you had a requirement say for maybe you want your cyber critical servers like your domain controllers and stuff to get monitored as much as possible you could actually create a complex variable to say for my critical servers monitor it you know run the consistency check every five minutes blah blah blah blah blah and then have a second table for your non-critical servers and have it run however long that you want and then dot source those same things so the complex variable constructs are critically important to making things impaired in item potent so this is something that I highly highly encourage that you you start experimenting with networks once again is a special child I find often times that you know I won't put my network stuff into a complex variable set because a lot of the resource objects are actually circularly dependent on a lot of those configuration settings so when you try to create a complex variable you can run into a lot of surgery circular logic issues so that's why you'll kind of see those call that as their own things but you'll also see here too like for example I have my V net subnets called out you can see I'm calling out that resource ID and then I'm concatenate it with additional string information and this is something that's particular to network objects like I said why they don't just allow you to dot source those particular properties in subsequent objects not 100% sure and to be fair I haven't played with it in some of the newer API versions so they may allow you to do that but if you try it and it explodes this is why and you know you'll have examples on how to fix that I actually have information on OMS instance so if I want to maybe have different workspaces I can go ahead and call that out in the same complex variable constructs and then my load balancer stuff I will tell you is anybody actually built a load balancer with an answer template oh you're in for a ride so load balancers are probably the one of the most finicky ones and do a lot of that weird concatenation stuff with the resource IDs and everything and this is where it gets circular so you'll see here that I've got a web server load balance resource ID that I'm calling out and then I have a load balance probe configuration reference that calls that resource ID and then I have back-end address pools that are called out and then I have oh where is it yeah the the backend address pools are actually I think concatenated later in in the configuration so and this is where all that circular stuff came came about not yet I'm actually trying to write a book on authoring Azure resource manager templates the networking stuff is a little bit of a challenge so yeah it'll get there eventually hopefully maybe we'll see but this also comes back to another another point about the whole authoring versus just getting examples I personally whenever I create a new object with an azure template I have a master template file and what I do is any new object that I create I will add it to this master template file the template file is completely functional in the respect that I could actually deploy it and all of the objects will work it's going to just be a bunch of random crap that goes into it but the reason why I have this master file with every new object that I create is because you might only do it once every so often but when you do you don't necessarily want to recreate that wheel so instead of using the QuickStart as your reference guide for when you're creating these objects create your own go ahead and build it learn how it works figure out what you need or what it needs to to create that particular resource and then added into this master reference so that way if you ever need to go back to it it's the code that you've written which makes it familiar easier for you to read and easier for you to recognize what you need to do when you're creating a new one and it's also a lot easier to be able to just scrape that code paste it in the template and then make the changes necessary as needed makes sense cool so oh did they break the schemas again they broke the schemas again all right this is gonna be fun so unless I haven't referenced wrong give me a second no the Brooks scheme is again yay this will happen on occasion and I've gone round and round with a the azure team about it you can see here it's saying the value must conform to exactly one of the Associated schemas occasionally they'll make an update to the master schema file when they add a new resource and becomes available and sometimes for a couple of days it makes all of your stuff get green squigglies because that reference file isn't quite working properly actually the biggest advantage and I'm sure this is going to be fixed at some point is that when your dot source saying objects in complex variables you get linting errors in vs code so it's not yet designed to recognize when you dot source stuff and actually I think Visual Studio itself doesn't necessarily reference that your dot sourcing and object property and a complex variable I think it just doesn't care so and I'll show you what the difference is between those two in a minute now you can see here that I've got the API version hard-coded per resource because obviously we're not using the thing Oh actually that might be why one moment come on give me my intellisense there we go this is the other reason that I really like going through is because when you go to set it in the particular resource it will highlight and yeah it works in vs code - it will highlight what API versions are compatible with the particular resource this goes into how you start modeling your resources number one start with type first most of the examples that you see often will show you API version first the reason why I say type first is because when you call the the type into the end of the template then you can call API version and it gives you the compatible API versions for that resource if you go the other way around so I'm going to just go ahead and create a resource here and we're gonna say API version intellisense yes and tell us since yes there we go go intellisense is a little bit slow when you're pulling a foreign schema it's gonna pull you a bunch of erroneous api's in actually when it's fully working all of those would actually be black and highlighted because it's basically saying hey you're calling a resource but no type it's compatible with everything yeh so you you want to go with the type first because it'll actually call out the appropriate ones so you can see in this list where it's like 2014 Oh 401 to 2016 Oh 201 so if I do the storage account type so type Microsoft storage and I do API version here you can see I get a different API version list so always start with the type first because it gives you all the compatible information yes so right now I'm getting a little bit of a linting issue because I think they updated the schema and it's probably messing with vs Visual Studio right now but by default if you call the API version first it's going to give you a complete list of all the api's and all of them will be compatible now I'll usually do type first API version second location third but location shouldn't matter because what are we doing calling the resource group that location this is what it looks like in code and this gives me the ability to stamp it out in any region and it's gonna go okay what resource group I'm in am i in okay use that location we're good the next thing I do is the the name now basically the the rules kind of get a little bit fuzzy here I'll use either the name or I'll use copy index and I know this is one of the things that somebody wanted to cover so I'll just go ahead and talk about it real quick so you have your base naming and you can see I've got a copy index here this is where you start getting into looping and using the same resource for multiple deployments in a single template so say I need two web servers behind a load balancer right I don't want to call those out necessarily you know called the same configuration out twice in the in the same thing so what I can do is I can use a copy index the way this works is you need a number makes sense right and II don't know how many loops to do I call this out in my VM config reference so if you go here you can see instance count my web servers I'm saying I want to but you can see as I go through the go through the template a lot of the stuff that I'm using I still use the copy index even if I'm only calling one resource because if my developer decides to come back and say you know what instead of one box I want to all I have to do is change the number and redeploy so you need a number easy enough and then you need to be able to create the constructs so naming convention is pretty easy yeah you want to be able to make those numerical so what I do is I put a zero in front of it and then copy index and why do I have the one there I think programmatically because by default when you start programmatically the first number and in index is yeah so I add that one in there so my first box is going to be number one but you also need the copy construct the copy construct requires two things it needs to know where to get that number and this is where you can see how I'm dot sourcing so there's my beer my VM config reference my role server and the property I'm on pretty cool now I just keep stuff in tables and all my decisions are made there and I don't have anything hard-coded if I want to change it I had a new table I'm good but the the copy also needs some type of a loop name the loop name really only matters to the instance it's not going to be called anywhere else and this is really for the the record for the deployment so you can call it really whatever you want just keep it clean because if anybody decides to audit the logs and sees you know the f-word loop you might get nailed the name is really just to because the the copy loop in itself is its own object so it just needs a name to reference that so it really has no impact outside of the the loop itself yeah you can see here I've got my tags in here so I've got like my subscription display name and then parameters for the environment no big deal and you can see here that I've got a crap-ton of dependencies so why do I have this order of operations and requirements I'm using Azure Diagnostics which means I need a storage account for my Diagnostics that has to exist before my VM does I need a network interface now if I look at the network interface for the web server that has its own dependencies okay this is where we get into that linear logical order of operations I need to have my availability a set exists before my VMs do and then my configurations so you can see here I'm building a domain controller for the environment I need that domain controller to be up and running before I even deploy this box so that way as the domain controller configuration is finishing then this box starts kicking off and it's applying its configuration and that time wait for the active directory domain doesn't have to wait very long so that's where I start referencing all of this stuff part of me so you do have actually I can't remember if it's still being managed or not there is let me get out of this real quick armed visio used to be really really good tool basically what it allows you to do is it allows you to load a JSON template and then it can trace back the dependencies caveat is the bigger the JSON template gets the harder it gets for it to be able to track a lot of those dependencies and then it just kind of explodes after a certain point so and I believe it was actually written by somebody at Microsoft but I'm pretty sure that the project was eventually abandoned because I haven't seen any updates come out of it in a really long time it's a good good starting point to kind of start getting that direction of where your template is going and then once it gets to that size where arm business already exploding you've already got a good idea of you know where you're going with it and how to set your resource dependencies because honestly after the first one a lot of my depends on for my different constructs is going to look really similar so if I go to like the next VM config you know I I have this one where it's waiting on the domain controller blah blah blah once again you know my dependencies are I'm waiting for the VM configure efforts for the domain controller so but you'll see you know a lot of kind of similarities where it's like okay these are what the dependencies are and then you get onto that track so really creating those first initial resources is where it's a little bit more critical to back check your depends on constructs after that everything else is just kind of sauce so type API version name copy okay is usually kind of the the construct I set tags optional but highly recommended depends on order of operations then you get into your properties all right properties are absolutely 100 dependent of 100 percent dependent on type and API version okay because what it's going back is or what it's doing is it's going back and reading the schema does anybody know how to actually look up the schema for a resource manager template so there is a github repo that Microsoft manages that always has the latest up-to-date master schema now the master schema calls out all of the it's basically a library of all the schemas available for the different resources right if you want to see what the latest version of perhaps your network one or storage one is or yeah let's do the storage one because I just updated that we'll go back here and it's 2017 1001 so we'll go in here find 2017 1001 and you can see that there's three schemas you've got one for cache container registry network power bi and storage this is a little bit of an odd duck because you've actually got five different reference types in a single schema update typically if I pick out one you're gonna only see one schema type in there so this one's particular to the network if we open this this is what the schema looks like and it's just a JSON file it's got the exact same things what your input parameters aren't and your input parameters in this instance are gonna be the those you know objects that were referencing so it's going to be API version it's going to be your name going to be type location tags blah blah blah blah blah blah blah then you can start getting into the requirements here in your properties so you can see here it's looking for one of these things so it's going to be either be the application Gateway stuff or some type of an expression here and you can trace it back and read what what that is looks like this is actually a particular two application gateway application security groups so these are the the properties that you would be filling out for that particular object Wow okay I got to stop moving my head around the location inputs and if we start getting down a little bit further in here you can see what the required properties are and then you know by inference what the the optional properties are etc etc etc so this is a great place to go back and kind of read through exactly what's necessary but I rely heavily on intellisense and tell me what I need to put in vs code does really well with this like I said the only real downfall is that when I'm looking in here you can see all that red on the on the side what's it doing red on its go looking at the dot sourced objects so I don't like red so I don't necessarily use the S code for writing this stuff right now kind of going back and checking the progress this is what I was talking about the object creation if I go through you can see here it's a right now it's built my demo domain controller looks like it's applying the OMS extension and applying the DSC configuration so this is why I keep that in for both sand you know aside from always wanting to look busy any questions so far go for it what's that for copy index yeah all you have to do is just not call a number that's where you start getting into operators I have an ad added that part is as part of the talk yet because I haven't had time to experiment it and totally understand it but yeah so basically what you're asking is you're looking for like some kind of an if-then reference so if parameter X says blah do this so that is now available in Azure templates somebody just wrote about it recently I can't remember who it was I will tell you that dan dan are you in the room Dan Iverson so talk to Dan Iverson I I know he was asking about it and I think he's been playing around with it but yeah that's not quite in this talk yet but yes you can any other question no okay are we you guys getting this awesome dude you going a little bit further here see so we talked about dot reference here's where we start getting into the networking stuff so are you guys clear on the the concat versus resource ID okay lame let me go back and just kind of reference that real quick so when you grab a VM here this is really where it becomes most crucially important so you can see and depends on I'm calling by rest resource ID in and everything right so if I was to call concat and it depends on when when we go through and do the initial template deployment right did you see where I did the the test deployment command and it said template is valid what it's doing is it goes kind of does a pre compilation of the JSON file to make sure everything is validated a little bit wishy-washy as far as the the stuff that it checks to primarily check syntax but it will also check object order of operations it also checks if you're calling concat in dependency as to whether or not the object exists when you do a new Azure RM resource group to deployment if you're running in verbose the first thing you see as template is valid okay if it's correctly configured so it's running that validation check when you do the new deployment now here's the thing if you have concat and it depends on it's going to assume that that object should exist before you do the deployment if it doesn't it will fail the template validation all right so this is where it gets really important as to where you want to use resource ID versus concat resource ID is to call an object that may or may not exist at the instantiation of the template concat is essentially looking for a string it doesn't know whether or not that string is going to exist so it makes this something that it should so basically it's just going to check the URI that you're putting together and say does it exist yes or no now we're where this gets a little bit funky is when you start talking about network constructs where's my load balancer the network constructs Rui Rui dependent so you'll see that it's got an reference ID here and it's concat Aang the resource ID for the web server load balancer now here's the funny thing we're in the load balancer object but it needs the resource ID but it's looking for the URI of something so you can see I'm Khan Khattak ID and then I'm adding that back edge backend address configuration reference which if we look at it is the rest of the string to complete the URI this is where all the circular crap and the network objects gets really confusing so I would say if you want to bludgeon yourself mentally but get a really good understanding of how the underpinnings of an azure resource work build a load balancer otherwise don't do it unless you absolutely have to does concat and resource ID make a little bit more sense and how it gets applied and how you need them okay awesome so getting back into here and once again this is where the dot sourcing becomes exceptionally crucial so these are these are properties that you're gonna keep using over and over and over again so you can see here I'm using that webserver dot name here I'm using web server dot name again for part of my base naming for part of the network construct dududududu DNS address boring I start getting a little bit lower into the VM config you can see here this is where I'm calling out my operating system information I don't have to remember what the publisher SKU and version is I can just go ahead and call it from that table and then what image on my am I using I always want the latest one because I like aggravating my developers getting a little bit further in this is where it's taking that credential information that we grab from a jerky vault and passing it in its passing it in as a PS object and then going ahead and parsing out that information from what I can infer because I've never gotten a straight answer on this it looks like it's probably taking the PS credential and then using the was a get Network credential property to pass the password in plain text so it's kind of one of those funky things about how it breaks things down but because we're doing it in automation it remains obfuscated especially since we're pulling from the key vault once again VM size because I'm pulling from that table I can dot source that and then that pulls from that complex variable construct getting a little bit further down into extensions so you can see here I'm actually the this is something that's a little bit funky with extension so I see a lot of examples online where you can actually create child resources in the VM objects for the extensions I don't do that primarily because where if you have a dependency on that extension coming out with the reference for it is extremely difficult so what I usually do is I create the extension separately and then have it depends on dependency for the VM to be created then go ahead and kick off that extension so if I have another dependency on that extension I can call it it may have been fixed in later iterations but my previous experiments have showed that when you have a child extension it becomes exceptionally difficult to create a dependency on it so that's why I have it on the outside by going into it you can see here this is where I'm calling in my DSC configuration manager settings I will tell you that if you have problems with the latest version of the DSC extension the only way that you're going to get around it is this way the was a edger are MVM DSC extension commands do not allow you to be able to specify the type handler so if you want to go to a previous version you got to do it through a template I think you can also use the not the DSC extension command lit but like the VM extension commands to special specify type handler version but then you have to remember what the type name for the specific thing is and it's a pain here's some of the properties that you'll see in here so registration URI that pulls it from our input parameters where we grab the automation account VM config reference this is where I'm getting that configuration name so all I have to remember is the web server you know the role and tell it to look at the DSC configuration in that table configuration mode so that's going to be you know applying auto correct apply only etc etc reboot node if needed that's another thing that I have in the tables and that's so that way you can actually set these up if you need different configuration settings for different web or server roles you can go ahead and create that in the table and then it just gets referenced off of it here also this is another nice thing you can specify Deb EMF version in the DSC extension I mean had some security people might be no we're only using PowerShell five and not 5.1 unfortunately it's against most HR policies to take them out in the backyard and you know bludgeon them until they start agreeing with you so okay yeah you can go ahead and specify those settings in here that's pretty much where this is at I wanted to make sure that I left about half an hour for questions because I know you guys yeah some of you guys have been expressing some I was thinking of you in general so yes let's go ahead and talk about that part of me by default I think it's complete so it's gonna go ahead and check through the configuration and if anything's kind of drifted out of config it's going to go ahead and kibosh it incremental looks for any of the objects that don't already exist and only creates those correct now I will tell you that using the default mode options one thing that can be affected is custom script extensions they will react reacts acute I found that out when somebody had put a script in without telling me and went and deployed and buy so yeah the the custom script extension will reacts acute any other questions there you go so that's passed as a parameter let's go back here here that's the wrong one where's the play basics they're real so that's set here the the definition of what environment I want so that's passed in an automation through my parameter object that is called in here all right then for the VM configuration reference I have the table built so just go ahead and fold some of this stuff so we can get NP and PR together I have my table definitions built on that NP and PR parameter input okay and then I have to create those reference objects underneath and the tables like so so web server I've got that instance count VM size name DSC configuration etc same thing for my domain controller now the way the JSON template puts that together is here so basically what it's saying is I want you to look at that VM config table and then that environment input parameter I want you to grab that object I'm referencing so I think it was set for NP on my deployment right so it's going to go ahead and say okay anything that matches NP I want you to grab that table out of this config so it's going to look in here and go okay I need these objects all right now where it gets applied is basically wherever I need it but for our example we were primarily looking at the web server so if we go here it's looking at that VM config reference variable that pulled those objects then I'm gonna say the the role server object that I created the web server I'm gonna go ahead and look at that out that object table and then pull the instance count so then it pulls that reference back and says okay here's my web server there's my instance count cool awesome and this is really where we get into all of that decision-making and I think about all of these decisions that we've taken out of our users hands and we've made those decisions based off you know all these decisions for them based off of one input our computer naming convention our object our object naming convention how many objects that we need how the configuration gets applied all of these decisions were made off of one input so now my user is much less likely to do something stupid any other questions so primarily cuz I yeah I work as a consultant so basically what what I'm often tasked to do is create the template and then hand it off to their guys for the CI CD stuff maybe provide some sample code that they would build for the automation I myself oftentimes use straight powershell for my demonstrations i've actually been working with a logic app to try to automate these so because I'm lazy and I like to forget things so I've been playing around a lot with logic app where it reads my Google Calendar so whenever I have a session scheduled like half an hour before it goes ahead and kicks off a run book to deploy it so there's a lot of different ways that you can actually do that like for example if you're using was it not scorch what's the portal thing that Microsoft has in the system center that you can actually have users go to what's that not devtest labs it's like system center service manager or something yeah it's been a while since I system centered you could actually create a run book off of that where it's like I want an environment and then just give them like what's the the environment level and then have your run book actually go out and pull the automation account names and all that stuff for them and then deploy so there's million different ways that you could actually build this into your your runbook automation any other questions because I've actually done it a few times and I have my reference configuration it's gotten a lot faster I would say the first VM config I ever built was about five days and then coming out and adding the DSC extension to apply a DSC config the first time it's probably another two days so it and this kind of also goes into the design thing to number one work iteratively don't be like oh you know I need a VM and I need a load balancer and I'm gonna need these network interfaces and I need the this V net and then I need these storage accounts and blah blah blah blah blah and execute you're gonna get a lot of red you know start simple start with the storage count deploy it validate it start with you you know then add your V net deploy it validate add your public IP is deploy validate any time you add another resource deploy invalidate to make sure that it works appropriately before you add another one then start adding in your dependencies you're gonna save yourself a lot of frustration and banging your head on a table part of me I do a lot of that stuff with desired state configuration so DSCs got some great resources for bandaging like local local group policy local security policy creating those users you know I'll use oftentimes the automation account credentials for creating like the base user credential and then I disassemble the credential in my DSC configs to create new ones so if I have like a series of users that I have to create but I know that they're all going to have that same starting password I'll pass that credential in and then break it down by the users and then recompile a new automation credential so that that long ensure use desired state configuration and apply those so number one intellisense is key intellisense is going to tell you flat out what you're missing in you know in your new object creation so basically you're going to get that green little green squiggly until you have a complete object and oftentimes like when i when i create an object property so let's go back over to my load balancer because load balancers have 50 million properties are you yes okay so if I do this and and I do my little blue quotes give it a minute because intellisense slow Network it's gonna tell me what additional properties I can actually add to it and then I can go from there so use something that doesn't tell us sense if somebody's like I only build my Azure templates in notepad well I too like to get frustrated on occasion but that's not how I like to do it yeah vs code like I said vs code works great except for the dot sourcing of object properties so it's a good free way to start or get your employer to pay for Visual Studio you know in the grand scheme of things with the time is that you're saving and the standardization that you're providing it's worth the money yet somebody's got it yeah and if you're dealt with developers are telling you that you're not gonna get it without prying it out of their cold dead hands well I can't make recommendations but they've given you instructions go for it part of me now I haven't I keep hearing about it it's just you know I I go with the tools that I know there's tons of really awesome tools so I don't you know push these kinds of things on them but like if terraform has the ability to read the schema and provide you like actual you know usable relevant information on how to create the the resource go for it anyone else go for it is it the Palo Alto thing yeah yeah yeah yeah so number one I have people at my work that sell palo alto so number one palo alto sax yes I said that you can quote me on it and I'll tell you what worked - yes so and let's let's talk about this this is where the the built-in templates in the marketplace are actually really awesome so if we go to create a resource and I type in PO you know I'm going to get I don't know pick one for me alright I go here I know we're gonna call you Bob put in my super secret password great new there there's we're getting to purpose on here so store everything's Bob know somebody already took Bob okay globally unique sure we'll call it that yeah that works buddy-buddy bah so before you buy there should be a link that comes up for the azure template yeah here you go now here's here's where the super hyper critical information comes in I'm not quite sure what this is deployments okay that that looks like just some generic crap what you need to do is you need to grab the template and kind of scrape the information that you absolutely need where it becomes super critical is with like the VM information in particular so you'll see here image publisher version blah de blah de blah oh good it's referencing variables I don't know why they do that so if we go to what's called image publisher Palo Alto Networks actually has their own image and then here's the SKU in the image offer you can pull that and then build the additional resources including the public IP and attach to your own network interface and it should work that's weird yeah we might actually have to take a look at your template and see it I I would suspect that maybe you have some kind of it depends on going on in there that it's looking for and then throwing the error now here's here's something that's a little bit weird with the Azure templates is that when it throws an error typically with an error you're gonna get a line in a column line in the condom column doesn't actually always match where the error is in fact taking place I think it's how it compiled like how the formatting is when it compiles so sometimes you have to like scroll up 10 or 20 lines to figure out where it's at but I'm thinking you probably have some kind of depends on yeah yeah but when it comes to when it comes to third-party custom images I think you know it's one of those things where they they can provide support but it sounds like there's something in your template that is calling it and I'm I suspect it's a dependency yeah yeah well so and this is why I think it's in the template is that the VM itself is just an image it's an image reference so unless there's a specific parameter that it's looking for which those parameters are going to be called out in there I am inclined to think that it's probably some type of a resource dependency thing that's going on or they're using some kind of a kind concatenated reference that's calling that variable and you're not seeing it so yeah I definitely like to take a look at that with you maybe during lunch and well well hack that out yeah yeah because typically there's not going to be any type of a return in the VM image that's going to affect the rest of the template any other questions what time is it yeah me for nine more minutes anyone anyone No oh did it survive it's still building we're in yet this is a really big build because I'm building like a Citrix environment here yes so you want to see it like what the configuration looks like I know I have some in here I'm just trying to think yeah here we go so know this so this is what the composite configuration looks like if you're familiar with composite configs you'll recognize it but basically I'm applying a base configuration to all of my systems then I have all everything separated out by role so you can see web servers just basically got domain join I actually have it building PowerShell web access because you know I like that as opposed to RDP here's my domain controller config where I'm actually building out the DC but I'm also creating my Citrix administrators I've got my sequel server here where it's basically just got some nitty gritty stuff for the the sequel server my license server so basically when this comes out compiled it builds out all of the different roles and then getting a little bit further into the nitty gritty you can see here where I've got my Citrix environment and like my director server schema where I'm just basically installing the binaries and all the additional stuff so all of this gets loaded up into Azure automation so the composite configs will be loaded as modules and compiled and then you upload the primary config that we saw so this will actually get deployed through a script that basically passes some of those configuration settings basically what I'm setting here is what's my domain name what's the automation account that I'm using and then the admin name and then in that composite config you can see here I'm using get automation PS credential to pass that information in so it's actually going to the automation credential grabbing that data and then throwing it in at compile so that's that's essentially what it down to I would love to show you my automation account except for I have customer configs in there right now so I don't want to peel that away all right any other questions yes yes so you do I won't be sharing this template I'm gonna just give you guys a basic template my boss is probably a little bit mad if I gave them I gave away my Citrix configs so first off please do do put in as mid-session rating because you know we rely on those and you know even though I'm the incoming CEO you know I still am contractually obligated to beat the crap out of myself if I do badly so yes please do fill out those surveys also here so codes the slides once again a gamer living we'll all the things but if you want to go out to github just look up gamer living well and I'll have by the end of the week my slides and code up there also cool graphic that I did well I actually I didn't do that I stole it but also follow me on twitter at gamer living will and then linked in a you know gamer living well Xbox Playstation 4 whatever just yeah Google that all right in that case get out in my room we're done [Applause]

Channel: PowerShell.org

Views: 2,149

Keywords: powershell, windows powershell, techsession, powershell summit

Id: fY62tqENNw4

Length: 105min 28sec (6328 seconds)

Published: Wed May 02 2018