GitHub Actions, terraform, SQL, EF Core

hello everyone welcome back again i apologize for my absence i have been a little afk from a lot of projects especially this last week i got pulled onto a project that requires me starting work at six a.m and i am not a morning person so i have been consuming extra mountain dew and lots of coffee in order to stay going through the day so i'm a little unprepared a little tired probably gonna cut things off a little short tonight um just a heads up so stream probably won't go quite as long as it usually does um i have not uh i've not forgotten about uh posting the recording of last week's stream to youtube that is still forthcoming um there was a couple things i wanted to actually um clean up and fix about about it before i actually got it posted so uh it is it is not forgotten i know normally things drop the following monday and it's kind of intentional that it didn't come out this week but rest assured the recording will go there um you can still find the recording back on twitch i forget how long they stay up there 30 days something like that so it'll be up on youtube long before those things time out for tonight i wanted to go back to my project that i was working on last week about the sort of trying to build something that's kind of full stack enterprise like-ish um out on github because i i found a lot of tutorials and they all seem fairly small and targeted and rarely is there a complete end-to-end solution that you can fork and plug in and start running out on your own subscription so i want to try and get a little closer there so if we recall let me flip over here this guy this one this is what we were playing with before so i've got an org called tuna waffle for um those people unfamiliar with it um i get made fun of a lot because i enjoy actually making and eating tuna waffles you can feel free to leave me your comments on your feelings of that idea but for those people who have had like a tuna melt sandwich or a tuna salad sandwich it's basically that subtract the bread replace it with waffle some people think that that sounds absolutely disgusting i love the little sweetness that the waffle brings over the bread love it love it absolutely love it so i'm weird um and this only further approves my point um so i've got this um kate's demo here and bundled inside of this is a bunch of stuff i've got an infrastructure project that has some terraform uh right now the terraform stands up let's see uh container registry uh it stands up a couple aks instances or aks clusters and then uh sql server is going to be where we head uh next because i'd like to to show setting this up and going um managed identity from the cluster into the sql server to be able to retrieve that data back and forth because managed identity is awesome uh for those people unfamiliar with it let's see measure managed identity uh yes spelling counts manage identities for azure resources these guys so anybody's interested in checking them out um you can almost conceptually think of them as accounts that uh almost like service accounts but service accounts you don't necessarily need to manage so there's an automatic uh password cycle built in and so you get this uh i guess better user account management and as the name implies it's managed for you managed meaning azure manages it you don't have to manage it which is great um so leveraging that in there and then i've got a ton of github actions set up um so i've got codeql scanning turned on i've got several infrastructure uh projects in here and then i've also got this this toppings api is my attempt at being humorous it's a poor attempt but it's an attempt nonetheless i tried to be funny i'm not that good at it excuse me food has been a luxury today i have not had enough time to eat um but this is a basic it's very close to file new project on asp.net core at this point there's not a lot in here there is a tiny uh unit test project there is a a tiny health check that i added um as you can see this health check is uh not overly helpful um but it does register up up a health check so that there is something there um to be able to go through and look at um it initially was going to be set up with identity stuff but that was going to require a lot more azure stuff and i just wanted to get this thing rolling and out the door initially there's a very simplistic docker file not a lot here um yeah and that's that's pretty much it but right now the the d here let's just go to the controller the default file new project has this weather controller that is just being backed by an in-memory array which is fine that works um i would like to make this slightly more advanced add another controller here that's actually related to the the tuna waffle theme that i'm clearly embracing and maybe store a list of toppings in a database table just something to show that we can query the database pull stuff out we may get fancy with like maybe a redis cache or something cool maybe get fancy with an admin screen on a web ui if i get really creative i want to try and keep it small but have it try to cover as many cases as what people are likely to run into so things like securing it the like the admin pages with auth and that kind of thing so that there's there's enough s uh enough there so somebody can see a way to do it because in some of these cases there are a lot of options um and then i still have i'm fine i will just you know what this dependent bot alert has been sitting here for a while uh oh and this is currently failing due to my other build stuff isn't it yeah how old is this build failed five days ago we could probably just bump this thing up too yeah uh part of why this is failing is i have a github action here whose sole purpose in life is to tear down my infrastructure um because i don't have unlimited azure credits and as much fun as footing a large azure bill for the sake of making a tuna waffle joke is awesome i don't feel like doing that so i've got this pipeline here i can it's got a workflow dispatch on it i kick it off it uses terraform pulls down all my infrastructure and then i'm not wasting money and then before i run i just stand it back up i'm i'm thinking i might combine for the sake of simplicity i might combine these two together so if you check out the previous video you'll see the there was a bit of discussion i had in there around service principles and accounts and in this type of setup where you've got your your actions managing both infrastructure and app deployment you probably is so i've only got a dev in a prod environment obviously a lot of people have a lot more than that but in this situation i think you need or i think my recommendation is six service principles um simply because you've got at minimum three different cases i i could see an argument for four service principles but you there is likely a separate one for your infrastructure versus your app because the service principal permissions for say your terraform is going to need to be um significantly more permissive than what an app would need to deploy um and then obviously so one for app one for infrastructure and then those two at least multiplied over for the prod environment as well and the third one is you can technically get away with a a service principle that has even fewer permissions but effectively read access to a lot of resources for doing the terraform plan because on these for example we'll just pick on this one here if i look at like one of the last successful ones you'll see that i've got two steps set up here one that does the terraform plan where terraform shows what it's going to do um and then i've got a a terraform apply with an approval gate on it because with terraform you really want to be reviewing those plans before they go out the door to make sure that the that terraform is actually going to do what you think it's going to do so what it's worth i think probably you don't necessarily need the second service principle for your plan versus your apply but if you are working with a very um we'll say tight security type setup because this one is on a um cd trigger um and this one's not this one's behind an approval gate this one the one on the apply actually needs like permission to create stuff whereas the one on the plan really just needs read access in most cases so you can get a much smaller permission set setup [Music] okay so where are we going next where are we going next so for my actions i've got so i've got a bunch of stuff in here and then i've also got these actions here and i wanted to separate them out a little bit but i apparently broke a bunch of stuff so get acr info this is a known bug that i need to fix so part of the issue is when i stand up a container registry the container registry has um let's see if i can clean that up a little bit the container registry has all right it's just bothering me that little bit of my green screen that is i usually fix that up with lighting but apparently not today whatever i'm gonna ignore it um the container registry has to have a name that's globally unique which means if somebody were to fork this repo and run it whatever name comes out from their uh their terraform for the acr would immediately conflict with mine so in an effort to make this project forkable and easily runnable without people needing to go and change a bunch of names what i did is i automatically generated a a name for the acr so it just uses the terraform random pet so basically just picks an animal name and throws it out there i learned the other day that a dog bird is a real thing i was very confused why i got two names for a little bit until i figured that one out so i learned something new for reference dog bird this is a thing i know the opposite of catfish that is no anyway so um so that's the only reason for this thing here but because this thing is trying to look up the acr it's going to fail now admittedly we're going to clean this up in a couple ways so we'll start with the easiest okay so pro tip the period key uh when you are on a code screen in github it drops you into vs code in a browser with the repo open which is awesome now this isn't full vs code you don't get like your your compute bound resources behind the scenes so you don't get like a lot of the extensions and whatnot that would require running so it's not quite the same as if you were to pull it down but i have been working and doing a bunch of github actions this week and nearly all of the work has been done using vs code in the browser this feature is awesome like the fact that you don't need to to install anything you just need a web browser to get going is great um if you want the full like extensions compute bound stuff github workspaces is your solution there but for now i don't have that set up so we're going here um i also don't know if i have any credits remaining for that so yeah let's look at do it okay so first of all we're going to come in here and so the one that was breaking is the get acr info but i thought there was a limitation on this did i not filter this at all that was dumb of me yeah so there's some cleanup that needs to happen in here so we're just going to clean this up so inside of this what it ends up being is if and actually let me go and look up github nope nope uh cortana hands off uh let's see github actions contacts boom so this guy here and i'm looking for the github context so in github actions your context is uh you can almost think of it like a variable source it's a place where you can get information about stuff um but in this case what i am interested in is the event uh i believe it's either event or event name uh oh event is an object event name yeah i think we just want to go with event name is probably the simplest here and then we will do yeah let's do that so github event name boom so over here uh if github event name equals or i guess not equals yeah we're gonna let it go hey i appreciate the follow uh linnameric i i apologize i probably just butchered that absolutely completely welcome to the stream we are having fun with github actions at least that's that's the goal tonight um and right now we're cleaning up some stupid actions that i wrote so we're going to say if this thing is not a pull request so on a pull request we don't actually want to go out and log into the acr and if we don't get the acr we don't want to do that um we don't want to build the image i normally it would be helpful to go through and build the docker image to make sure there's no errors there but in my case the docker image is so it's really thin the number of things that are gonna go wrong on a docker build on this are pretty small really small so i'm not going to stress about that i mean because one solution is you'd separate these out from uh from combining both the build and the push into a single uh step and make them uh making each their own individual steps kind of like the helm helm stuff uh we're gonna do this guy here and then you know what we're gonna move i'm gonna change this up a little bit let's move this home stuff up here because we can package this guy up still uh and actually let's rather than save let's call that package helm chart and then once again if it's a pull request we aren't going to do it's a pull request we won't do any of that stuff and then i think down here we're going to do the same thing oh i had the if in that so if a pull request we won't deploy out to dev and at some point what we're going to do is we are going to have deploy prod but and that's where i wanted to get to tonight i want to clean this up a little bit because there's a there's some reuse that can happen here um and i'm trying to decide between so i've got some composite actions that i've set up inside of the repo already and i'm i'm halfway tempted to push them into another repo you can do either just to be clear and i'm not and i'm trying to decide between the the simplicity of keeping everything in this repo um and if it's valuable to show samples of them being in another repo because typically if you if this was if this was not my little demo org here right because tuna waffles and and i know this should they should shock no one this is not a real organization i i need to put a i need to put a readme on here um but this is this is not uh i don't have a pinned repo boom now i got a pen repo there we go um this is not a real one this is this is legitimately just me having fun and putting together samples because one it's good learning experience two people like samples they like to see examples of how to do things and so i try to keep the examples small but there's a balance there between too small where it doesn't show enough things that are close to real world versus being too big and where it's hard to derive what you actually need to do that's the trade-off we're doing right now um yeah but i think in our case i kind of want to separate some of these out so that should fix that issue there so and also if it's not clear i have zero branch policies set up on this thing we're going straight into maine prs the the pr the the pr trigger thing is only a problem um i wonder if we should i wonder let's look i wonder if we should clean this up even further such that if it doesn't find an acr it terminates i actually don't mind it airing but i'm wondering if maybe i should make it air with something friendlier i mean i guess that's pretty reasonable is it saying it couldn't find the resource group that's pretty explicit that's because my infrastructure isn't stood up i'd have to run the uh the other ones yeah we'll leave it as this okay so um uh up running steps on pr so we'll just commit that sync and so that is that's gonna fail again so i expect this thing to push to run again as you can see my code ql kicked off again and there is and let's take a look at that so on my code ql you will note there's some building stuff down here for compiling it and i think i want to i think i want to break some of this up i think i think i want to break some of this up so uh [Music] but i'm also interested in why these ones failed here so let's let's start with this because um cannot find okay so this is worth pointing out so because this was a change i made earlier so which one did i open everprod uh prod let's look at the prod one and then this is specifically does it have well it's not giving me a line number but it is the terraform plan and i can kind of guess based on what i changed last what the problem is so here's my two jobs terraform plan terraform apply you'll note they're pretty thin there's not a lot here and that's because it is using a composite action composite action is stored in a local file here you'll note that it is set up to look underneath github actions and then it you point it at the directory and inside the directory it needs to find an action.yaml file and then this guy here contains all of the steps to do the work so this this does the az login the hashicorp action for setting up terraform does the init does the plan everyone's happy at least i'm happy it's my workflow i have to be happy okay so let's come in here and it is failing to find that path there so i kind of want to just put you actually you know what leave it there because the place we want to compare it to is going to be up here uh because oh interesting hang on we got to set up job and straight into terraform plan what happened to the what happened to the checkout uh oh that's the problem so in order for it to find this it needs to be able to check out the repository but i put the checkout inside of it so i've got a chicken in the egg problem um and i don't think having a tf plan to a checkout really makes that much sense so i think the solution here is going to be just to extract that and move it outside because as nice as it is to have an individual item doing the checkout for me i don't think it's intuitive if i were to reuse this composite action elsewhere there's a very real chance that the checkout would already be um done and so i don't want that there okay so let's do and i'm going to assume that i did the exact same thing in the apply yeah i did i did so let's clean that up boom boom and so we go here we're gonna slap that in there slip that in there we'll do the same thing on the dev side you'll note one of the other ones i'm i'm heavily led before gene is this get acr i just wrapped up the the steps for doing the az login and then doing the queries to pull out um things like the login server the name and the id of my acr just so that i don't have to think and again the only reason for having this much complexity in here is because i am generating a random name every time my terraform puts out a new um version of the container registry [Music] boom so we'll do that you already are there on dev prod check out check out okay so that should be fine and then we strip both of those out excellent excellent just check the diff real quick and again i'll just i'd just like to point out i haven't left my browser yet this is pretty powerful it's it's pretty cool um the the the cool thing to note is the only thing that's happening when you press that period key on a github repo and you do need to be on the code view it doesn't work from every view but like the main view of the repo and the code in different places the what's actually happening is it's just switching from github.com to github.dev so for example i was on my phone and i wanted to make some changes here and i realized oh it's really hard to hit the period key and trigger the key press action on the mobile browser but i can open up the url and just change.com.dev and poof i got into the the mobile version of it and not a great editing experience a phone screen is really tiny it's hard to see a lot but it works it works uh check out repo uh before attempting to access um composite actions boom okay so i would expect uh let's see boom so that should kick off and run um okay perfect so one this didn't immediately blow up and fail like it did last time so that's a plus in the right direction now the question though is should i be moving these into a different repository that really is the question should it be in a different repository okay and then the question is are we going to stand stand some of this stuff up um yeah you know maybe we'll do a little ef core stuff i put it in the title of the stream might as well um oh terraform plan here is looking for the acr and is not finding it who would have thunk this is part of why i want to i'm disturbed by your repo name thank you kenny so just to be clear for for anybody who missed the the starting here so the repo name i don't think is what's disturbing you kate's demo should not be disturbing the org name however there is small cause for concern there small cause for concern because yes i did acquire the the org name of tuna waffle and yes i did go to the effort here do i have zoom it i did go to the effort of getting of getting a logo my wife was very happily willing to oblige me when i told her that i had acquired the the org of tuna waffle and i was gonna go looking for a picture she's like i'll just draw one i'm like go for it i was like you will come up with something better than anything else so i've even got a logo it's pretty good it's pretty good not great but it's pretty good [Music] but at least this thing here is uh it's it's it didn't working okay so let's actually let's actually do some stuff with this so these two ones dying makes sense because i'm guessing they both fail on acr check or uh scanning for the acr yeah get acr info fails your commitment to the joke is what sells it i know it's it's great it really is great which is why the toppings api darn it needs a database for reasons that i'm sure will make sense by the time i'm done building it or i will make something up okay so let's actually stand up some infrastructure here so we're going to go here we can look at the terraform plan real quick if we care to review it so for people unfamiliar with the terraform plan i start at the bottom because this little summary is what i find to be most useful for resources add zero change to destroy in other words nothing's there yet i'm gonna make a bunch of stuff so it's gonna do a role assignment um and this is just here because i wanted to have a group that could get in and manage this stuff for any of my colleagues that that are crazy enough to help me on this emphasis on crazy enough but for my prod environment it really just contains the container registry now every org is different they might have more than one container registry maybe one servicing their like dev environment and something else um supplying their helm charts and images to to upper environments you know qa stage prod etc um and so that's all this is doing and this here this little random pet this is the reason for all this crazy acr look up stuff i i've been questioning if this is the best way to handle this problem because otherwise i risk failure if somebody forks this repo and runs it while i have my my infrastructure torn down and then i have to change my name just doesn't seem fair so random it will be okay so that looks good uh oh no we go here review deployments approve so prod info will go out and then as soon as this thing gets the acr stood up we will go through and um get that built because i i would like to get what i would like to do is uh get the database up and get the managed identity with access to it such that um i can then connect from my cluster in and ef core all the things at least that's the that's the game plan so let's take a look at doing that and at some point i really need to to finish out doing this and setting it up but so for example for my i'll just call this out real quick so for my terraform stuff i've got an environments folder or an emv folder and then dev and prod okay uh in each of these uh has the uh important stuff the key thing if we look at dev most of the code uh wait are you gonna install the terraform oh oh buddy oh i'm about to be very very happy huh so i'm gonna get terraform support oh yeah oh yeah oh yeah now i'm really happy okay so when i said you couldn't get extensions what i meant was you can only get some extensions anything that requires compute boundary sources is not going to work but apparently this thing does not need any compute boundary sources so anyway for the dev environment it basically just has one module the providers don't do anything other than specify versions and connection info to get back to the the state file and then just a couple incoming variables for example the id of the container registry i know what a shock but then all of the interesting stuff happens in the environments module so if we come down here this is where we now have more stuff so i've got aks related stuff in here it sets up uh role assignments for the the cluster if we come down here i've got a sql server this guy is going to spin up um oh this is just a reference to the ad group it's going to spin me up a sql server connect up some managed identities and i don't recall where i ended on this this may just straight up fail so as you can see my sql admin password is random my admin username i am i really have commitments to random pets for tuna waffles because reasons um and so those ones are being passed in here so the prod environment though you'll note i've got the environment module commented out because right now i'm just wanting to work with a single um aks instance so that's where i'm deploying what i really should do is turn off dev and only powwow and prod but no i'm doing this the hard way okay so that guy stood up a bunch of infrastructure huzzah now i need the dev side of things workflow dispatch fire in the hole okay so that will so the prod one should have given me my container registry is feeding your pets tuna waffles some form of cruelty or treat i think it depends on what the random animal it generates is because i don't doubt that tuna waffles are probably toxic to some mammals or animals in general and there are some particular colleagues that would argue that they would be toxic to them so spoiler alert where i work they occasionally do waffle days and i always volunteer to bring the tuna no one else ever does it's very weird and only i i think about so two of us like it and maybe three if i can con somebody else in to help me eat it there there's been a few people i've conned into at least trying it and most of the time they go yeah not for me which i think is a polite way of saying i'm pretty sure what that that's what that means so uh a smart person would review their terraform plans i'm just gonna run it or at least i'm gonna start it and then i'm gonna go back and look at the plan um okay so let's go to the bottom again tend to add so now we'll we'll take a quick look at what all it's going to add well tuna is the chicken of the sea i love it i love it rollers uh [Music] crank down my mic just a little bit i'm seeing it cap out on me okay so first resource uh oh let's see this is just a data source doesn't count data sources are inputs things that terraform is going to read not necessarily resources that terraform is going to manage there's my kubernetes cluster i've stood this up about a bazillion times so i'm really really confident it's that all of these settings are correct or at least for what i'm doing they're correct um the sql database is the one i'm not 100 sure on if i've even tested this or not so we'll find out in a minute um so a lot of these values you'll see are marked as known after apply known after apply known after apply it means these things would be available downstream of this uh resource so if something else wanted to say read the server id they could get that off of this database now no this is the database not the server itself that's coming that's coming so here's sql server um you'll note it terraform is reasonably good about sensitive values um github actions if it knows something sensitive as well like a secret or whatnot it'll start out but just to be aware the starring out on the terminal is not it's it's the level of security that i need while streaming to make sure i don't accidentally expose my my admin login password it is not the level of security that's going to stop a malicious person so just to be clear what what level we're talking about here this is the stop the streamer from accidentally exposing his key not these stop the the bad man from taking your password okay so it's going to spin me up there uh let's see resource group for app and then some role assignments so the acr poll is what's needed by kubernetes on the container registry to be able to pull that back for my azure infrastructure i'm spending everything in one subscription but everything's variabled out where they could be in multiple subscriptions all that matters is that the service principle that you use to run your terraform has access to everything as long as it does you're good to go so but my service principal right now has i think owner rights over the subscription so i didn't spend a lot of time let's say zeroing in on exactly the permission set it was you want to do what sure and it probably could have gotten away with a lot less um let's see so network contributor which is needed for standing up um i believe uh the ingress controllers and being able to register the public ips for the load balancers if i remember correctly the monitoring metrics publisher i probably actually don't need um because i'm not actually publishing those metrics but i probably should so we're going to leave it uh random password because reasons random username and then one more role assignment this demo contributors is all about me just adding in the users who are who are helping me because i have i have gotten a couple colleagues to to volunteer to help which is very nice of them they are very nice people and we like them very much because it is it is nice being able to do this i will say one of the awesome things about terraform because it one of the first things it does with its plan is build up um a dependency tree with the relationship between your resources uh is this some kind of guilt trip kenny would you like to i can add you to tuna waffles your membership can be remain private if you're worried about people associating your name with the word tuna waffle you don't have to make it public unless i accidentally open up the team and show all the members in it on stream and then it's kind of public in case you care i do also have a because this wouldn't be one of my streams if i didn't show off get cracking at least once so i've been managing my work on a get cracking board and naturally it's called tuna waffles because reasons so backlog in progress done which is great um i did discover something on get cracking boards that i didn't notice the other day so it's got some cool things about when a person moves a card onto it you can automatically make that person an assignment but the one that i discovered the other day is there's actually a trigger for when a card is created so in this case if you try to create a card so how about get kenny to to help right so if i create this card you'll note it jumps it right back over to the backlog that that's pretty cool i i i like this these automations are so stupid easy to to do so you just pick your column pick your pick your action and then what you want it to do and the fact that there's a celebrate is even better so for example my done has the celebrate set on it ready yay i get way too much enjoyment out of that and then i've also got a thing that just after three days archive it once it's been done for three days who cares it's just so easy it's so easy i like i like mine i like my get crackings the card is not legally binding are you sure kenny are you sure are you sure uh on the note of terraform two because it has that dependency graph it will build anything that it can do in parallel it will so oftentimes terraform can get itself um sometimes wrapped around the axle because it tries to go too fast which it's kind of like when the interviewer asks you what's your biggest weakness it's like i'm too much of a perfectionist that's what terraform is doing here what's your biggest weakness i work too quickly uh boohoo go home terraform um but it looks like we are down to sql server creating and i don't actually know how long this is expected to take and then we probably need to figure out the connection from point a to point b so let's get a i'm going to actually do this on a second monitor real quick just just so i can be confident of what is about to what is about to be shown because i have a bad habit of accidentally showing off my secrets on stream and i'm getting really really really good at cycling them i just prefer not to my goal is to not burn up all my azure credits for mistakes that would be bad but the nice part is it's pretty if i needed to recreate this in a different subscription it wouldn't be that hard okay so there's that uh on the note of needing to do that so here i'll show that off too so under uh the settings for your repo you can go through and create environments and again you can see i've set up the six environments like what i was talking about with the service principles earlier where the the deploy ones are the ones that ideally would be scoped with read level access as a service principle behind them whereas the infra ones would have significantly more permissions because they have to actually like create a bunch of azure stuff or i'm sorry the the read only one is the staging deploy is for the app and infra is for the the terraform to create and so for the infra ones there's a approval gate on it and actually i think we can change this to contributors now so let's just take oops that was the wrong one we'll fix this real quick because i i created a team rather than rather than just this um one thing to note too if you're inside of an org um and your teams don't show up oh please um if so there we go if you're inside of an org and your teams aren't showing up as a list one of the things that you may need to do is add the team from your org to the repo so just because you have a team at the org level that doesn't necessarily mean it's been granted access to the repo so you may need to to do that as well okay so we were coming down here i'm just going to clean these up while i wait for that thing to go uh contributors um but the the thing to note is then down here these are the variables for the service principle being used so these could point to any service principle that has the appropriate access to to run say my prod infra so don't care about subscription don't care about anything so if i needed to swap this out for a uh a different azure account i would just need to um either give this service principal access to um said subscription or i would uh and then change the subscription id or or do something slightly different in here right or you know brand new service brand new service principle uh pride deploy one protection rule i wonder why prod deploy this should have this should not be lacking in a protection rule that is i think just a mistake on my part uh let's see prod staging this one is fine because this is expected to be the read-only access so the staging one should not maybe i should rename it but the staging one should not have any blockers on it and then dev deploy is set up to be my cd environment so everything in the main goes straight out to dev and then you can scope your secrets with terraform actions at your environment level you can also scope secrets at a higher scope too so you can do right so repository level secrets would be above the environment secrets and then you can do organizational level secrets above that so you can imagine if your org say uses a third party payment processor that needs an api key to auth with and you need to have that same secret shared from everybody well org level secret right if something that would be if i had something that i was intending to share across all of these pipelines repo level secret in this case all of these are environment specific um but i because i think every single environment has the exact same values in here i could have done every single one of these as repo secrets that would have been a possibility poof look at that okay so hey where my azure window go boom you there and refresh and i should have sequel booyah we got sql okay so we let's go and actually write let's write some code real quick because oh i was doing docs uh stash come back to those later i was actually i was i was trying to be a good developer and like actually like write documentation for how all this is set up um because this is meant to be an example repo which is going to need some docs uh but in our case we are fire official studio come on come on you can do it i believe in your visual studio i believe i believe um oh i probably haven't opened this since i started doing a bunch of the other stuff uh close all taps okay so let's start with this oh side note i've got really good code coverage really good code coverage which sounds impressive right until you realize i wrote exactly two tests i have covered like 96 of my code base with two tests uh when i was done laughing i was like good enough we're stopping there i wrote these tests as a example not as a hard and fast rule but in terms of testing your your health check i don't know if you can get it much simpler than that i tried to make it easy uh worth noting this is leveraging the asp.net core web application factory for doing integration uh style tests so this is going to spin up an in-memory instance of um kestrel that you can use to uh just execute http client request with so when you do this create client what the factory ends up doing is it stands up that in-memory instance sets up an http client with its base url already pre-configured to point at it and then hands that back to you so it does all of the hard work of stand the thing up execute your startup execute you know whatever things are in your di um get that running get the http client point it at the right thing hand it back to you and then for your test all you have to do is invoke your endpoints can't make much easier okay so we are going to go here so add controller that sounds great uh uh using entity framework huh i don't actually want that huh didn't realize that one uh was there uh api controller with actions for entity framework i we're gonna do this but we're probably gonna delete a bunch of it sure uh let's see model class see you assume i have a data context and you see that's the problem is you assume i have all these things add there uh we will go with copyings controller or how about ingredients wow um don't laugh i can't spell but fortunately for me bing can did i get it right no why didn't it show me the did you mean thing there we go okay uh let's see controller boom and yeah that's not even close to what we want um so we want this to be yeah we're gonna we're gonna throw away a lot of this this isn't an api controller did i did i messed something up i feel like i i feel like i did not this is not this is not an api controller api control looks like this okay whatever um okay in this case we are going to some people like to break their stuff into multiple projects i think in the interest of simplicity we're gonna put this here and let's go and do manage packages because i don't have ef core yet do i no let's see here we'll fix yeah this will fix the this will fix the depend about warning too uh yeah sure update all the things update all the things make it be so pseudo make me an environment okay uh let's see entity frame uh uh let's see i believe we want sql server so i intentionally skip over the base one because the i believe the derived one takes a dependency on it anyway so let's go that route uh i accept okay and then so we're gonna just like throw away all of this because this is this is not even no no no no no no because you'll note on on these guys here it doesn't like it doesn't need to be complicated but we do need something to return so and normally i don't advocate for returning back your your model from the database but in this case we're keeping things simple we're gonna start there we will we will make it secure and appropriately structured later uh hang on uh let's see and then we will go class for context boom okay so you become a db context yeah include include thank you uh let's see public db set uh i really should have picked a type that i knew how to spell that would have been smarter uh let's see and then this becomes set of ingredients so for people who may be familiar with like old df or similar there is a reason i am writing it this way uh what is it there is a base method called where's the get set call fine where are you i thought i could yeah set what are static types cannot be used oh it's because i uh okay uh no the reason for declaring the db set this way is because i have notable reference types turned on there are various ways of addressing it but if you pull the ef core docs you will see something like this as the preferred recommended approach for for handling it because under the hood this guy really should never be null in fact let me phrase this guy can't be null um and i believe this guy will either give it back or throw ensuring that this guy will never be known okay so we've got that we've got that um we've got this so how about uh public int id public okay this is creepy i typed public space and it did exactly what i wanted which is pretty cool creepy but pretty cool um and i don't want to deal with noble reference types here so i'm just going to make a notable done see problem solved okay and so next we need to register our database uh or actually we'll we'll work backwards we'll go here so this guy here so this guy is going to need a toppings context [Applause] uh let's see create and assign context that sounds great and we will add an old check sounds great i'll put you about the constructor please that's better and then this will be yes would you like to write all of the code for me public can you guess the rest no um you were correct about async um i wonder if this thing supports async enumerable let's not fight with it at the moment i don't one thing at a time one thing at a time we're already going to have coding stuff to clean up uh let's see task of i enumerable of gradient it's kind of creepy the the the intellicode is very amazing and at times i question if it's psychic uh so contact con tech see it can't protect me from myself it's the problem though uh and so we want ingredients uh let's see a wait to list a sink find the extension method boom okay so a couple comments this is normally a horrible idea this will take every record in your database table and return it back this is almost never what you want with that said we'll move on uh not all code pads return a value that's because i didn't use the return keyword okay so that gets me to there startup and now we want to go services add db context uh let's see toppings context and then do to do and then uh we're gonna need to register a connection string it's gonna be the interesting part so [Music] let's check access control did it was i smart enough okay so my here's the thing my demo contributors group which i i believe i'm a member of i should confirm that really quick yes i am in here multiple times excellent okay so that thing has contributor level access so if we go to the database i should be able to hard code a connection string here and get away with this um manage okay so let's let's take a look here um i should be able to just log in login failed i would like to not have to use is that what it generated for my my username that's funny um how does one set this up for aad integration and i wonder if this is what i ran into before uh let's see do do because i would like to be able to sign in you with that account so ideally here continue log in um sequel how does one enable this yes yes i get it permissions um sql managed instance oh is oh i wonder if this may be what i want to use instead because i am not using a managed i am not using well configure azure id off uh [Music] that's your ad admin uh ssql managed instances active directory admin that is that is for a managed instance though [Music] now i'm curious what all is in stream here replicas this is the database hang on uh server oh an azure ad admin must be set set admin okay can i do um demo contributors save does that get me in now is that really how hard it is if that's how hard it is that's probably just need to uh do a little terraform work then uh this is wait i want to be at the database i want to go query editor while establishing a connection was denied deny public network access set to yes uh right so about that um this is not something i am interested in hiding i would like to be specifically the one that i'm going to have in dev because i am going to spin a separate instance for dev versus prod because these are in the environment folders um i would like to just turn off that networking bit where is the networking stuff firewalls and virtual networks deny public access um so so this is just on um okay and there's no way for me to turn it off save if i add my ip to it refresh let me in i just went into my own database that much to ask um to connect to this server use the private endpoint from inside your virtual network i really would prefer to just connect straight and not stress about this um uh are you really gonna make me configure private endpoint just to get access to my database huh set up a private endpoint yes but then you have to add yourself in i don't want to do that whatever happens you just spin database gain access um there has got to be a alternative option in here somewhere uh no i don't want all security center where is the let the stuff in button wait add client ip save refresh show me that ip still in the list i'm sorry what so it's not even letting create new v-net let's i'm wondering if the fact that this thing's not actually let's see if i can just put this on the same v-net as my um it's my cluster so i'm pretty sure i've already got one i wonder if that's part of the problem because it's not on a network there for uh [Music] yeah aksb net aks subnet sure sure sure um the v-net's missing the service endpoint for it sure sure uh new v-net one sequel this is if this is what i need i'm going to go and do this in terraform here and we'll go from there what i'd really like to do is set up my my local development stuff to be able to point at my dev environment uh public access for the server is disabled of course public access has been disabled i would you really really really are pushing the private endpoint okay instance details uh uh sql private endpoint research type um sql servers that one yeah virtual network current subscription yeah let's just go back let's go back i am not interested in fighting with you uh this is going to be a little obnoxious okay so rather than dealing with this let's i am curious if in my terraform uh so close that let's take a look at the registry so for terraform stuff um the terraform registry keeps track of all of the docks and it is absolutely amazing one thing to keep in mind check what version you're on here as you can see updates for these providers roll out we'll say often so six hours ago so i'm going to be conscious of the fact that i'm not looking at the version that i'm using but i kind of want to know if it's if i see an error about a missing item i will check it public network access enabled defaults to true did i turn it off okay well there's problem one so we're gonna uh enable public access go so that that's that's gonna be part of the problem we're gonna let terraform do this and i'm just going to cancel the prod run because it's a waste of resources and then dev1 so the other beautiful part about terraform is it it um so it keeps track of a state file which the state file is expected to represent the current state of your infrastructure that terraform is managing so it doesn't necessarily go out and look at azure to know the state of say my sql server what it does is it looks at its state file and then compares that against my current terraform scripts and says okay what's the deltas that's what i need to do so mucking with things in portal is a great way to confuse terraform because even though it's going to call out it doesn't necessarily know okay so what does it show as the deltas so what are you gonna try to do here so it's going to [Music] hopefully just modify so two to change so database is showing up with a license type on there i might just tack that on it's already there interesting so i might have to figure out why wait why did that show delta there is no change there okay whatever um but it knows that that property is going from false to true excellent uh i need to approve uh summary review deployments proof and that should go out the door and then hopefully that thing's gonna flip because we would like public access again i'm not overly concerned about this thing oh it actually created v-net so let's surprised it did that we're gonna turn that off it's probably gonna annoy terraform that something else is mucking with the database oh it hasn't even gotten there yet it's still querying we can outrun this i think perfect perfect perfect perfect perfect okay terraform you should be fast on here terraform successfully initialized sudo update my database doom and then it runs again so currently when i'm running these act or yeah these tearful hey terraform these github actions i'm just running a terraform plan on the first step and then a terraform apply on second the problem is i'm not persisting the plan from the first one so there is a small but tiny chance that the plan that i see here is not necessarily the plan that's going to be executed here on the terraform reply i've got a backlog item to address it basically on the terraform plan you output the results of that to a file and then terraform apply would ingest that file the problem is is i have to take that file i have to put it somewhere and then pull it back down and that was more work than just writing terraform apply so i went with the i just wrote terraform apply it's much simpler come on come on come on go green how long could it possibly take what will almost assuredly make it go faster is if i repeatedly refresh my browser oh look at that look at that deny public access okay so thank you thank you thank you hurry up and finish um but now i'm wondering what happens if i do this let me in i suppose i should wait for it to actually finish and then this thing oh it cleared out my admin i will okay so you're done terraform's happy great we will set admin uh demo your lack of dark mode hurts my redness hang on kenny hang on i think it's where where's the button for it is it where is where's the dark mode button there's a dark mode button uh dark where's the button i'm just amazed you don't notice it yourself i do notice it going between this and this it's pretty obvious i just had it set to light mode because i was doing some presenting and if you present on a projector you should use light mode if you're doing something with a lighted screen dark mode you can get away with client with ip is not allowed okay so that is but we should now be able to add that in okay so add client ip save okay so refresh oh go okay okay okay okay so we had to we had to muck with some stuff i had to add some stuff so i need so the key parts are ip address being white listed and then um and then setting the server admin i will not be able to do the white listing of the ip address from the github action that will not be possible because i won't know what the ip address is more important leave my my local one will not handle it but to do set so i'm gonna have to pull that in uh or actually maybe we just do it right right now uh where is what's up there too many things so it was on the server not the database and it is on okay my blind or stupid as your active directory as your active directory admin okay so azure ad admin that looks like the property i want so boom so i want that login username and i think what i'm gonna want is that guy up there so data azure ad group uh demo contributors i don't know if i want display name or or if there's a name property let's look this guy up too so if i do this just go terraform resource boom uh that one take me to the registry and then more importantly take me to the group so let's see here oh boy i really gotta turn my notifications off because now i'm getting spammed from my github uh okay so display name is there there isn't uh is there an id uh object id one of display name or object id must be specified well if i knew both i wouldn't need it yeah there's object id okay so what we do here is we'll go with display name and then this guy here will be object id boom um adding azure ad admin so commit push and then that'll that'll go do its thing and once again i'm just gonna kill the prod run because i know that there's nothing in the prod run that's going to matter um github just can't know that okay so that'll that'll fix that um oh right we were wanting to connect to this thing so um azure or i'm sorry entity frame framework connection string so because my currently running user is there i think i can get away with uh that is not even close to what i want dotnet framework how okay any framework for there we go that looks more like what we want to do yeah this is more akin to what we're going for um yeah so we're gonna do pretty much exactly this with some mild renames so boom boom you sql server that guy and then we are going to go with uh how about uh apps database because i think i actually called the database app or similar probably not a great name for it but it'll do um yeah appdb [Music] so we'll call it apps database that seems fine it's fine it's fine it's fine um okay so that guy there that guy there that guy there and then the thing is we need to put this somewhere now it's tempting very tempting to just dump your database connection string here let me assure you that's a bad idea what you do is you right click over here and you hit manage user secrets look at this a json file oh that's not it uh undo undo i copied the wrong thing and you grab this and you you put it here i will never understand why some people think this is harder than putting it here my opinion they look remarkably similar um and what ends up making this work and we're gonna check and see if it's even turned on where is it uh oh oh it's picked up by default isn't it oh i bet you it is buried inside of that used defaults yeah there's a lot of magic that happens here um so you'll note in the the documentation here it lists all of the things that it's going to pull configuration from right so it'll look at the environment variable with net underscore blah blah blah blah blah call your name down to about halfway down when it says from user secrets but you'll note it'll only use user secrets if ihosting environment environment name is development this is perfect this is what you want local development should not check in their connection strings they should use a secret yes it means every developer is going to have to go and get that connection string set up for themselves that's fine it's okay they'll live document it write a script i don't care have at it and then you put it here and it just works uh but we're gonna change the name of that because we called it apps database and more importantly this is not the right connection stream so back to portal did you finish see a smart person would review the terraform plan we've already established that isn't me okay um i really just want to see this thing work really quick i'm just going to refresh and make sure that that terraform update didn't reset my sql admins perfect that apparently worked or it's running now we're gonna make sure that stays there um but what i want is my connection strings where did that section go okay close you i know i saw a connection stream yeah yeah yeah i'm sure there's somebody sitting there screaming at their computer saying you're a it's right there in front of you like that okay so uh sql auth is not what we want uh let's see user id and password not what i want integrate it off this looks like what i want yeah i think this looks more like what i want so i take this connection string i take it and i go paste then i look at why it shows so many extra oh my word see this is one of the dumb things escape escape okay that goes like that that goes like that i assume boom that goes like that uh let's not launch in docker let's just go with kestrel go do i have to occur running i do have docker running and apparently there's an update to docker that i need to install huzzah let's just we're gonna just do a quick check in here so that apply went out and i want to confirm that my admin is still appropriately set huzzah admin is still set to my aed group perfect okay uh idb context was called uh only declares a parameter-less constructor that's because i was a uh this means that will never be passed then that d um see it it even tells you exactly the code you need to write of hey you you didn't give me a constructor to invoke and then you told me to pass configuration um come on copy paste nope okay db context options topping context oh you had it oh there thank you uh try try again now i still expect this thing to fail as soon as i go to try to use the context because tables don't exist i haven't called like created or any of that we haven't gen migrations none of this is gonna work weather forecast no it launched let's just just catch it right about there oh wait that's not the controller let's catch it right about there because right about there is where i expect to boom execute and then yeah so exception because uh does not support well that doesn't that just bite so it acknowledged the account and then didn't work um see the hard part is i'm running under one account locally and it's not the account that's going to run in the cluster and so i'm halfway tempted to just deploy into my cluster um though the id of my cluster is not being set with access into the sql database anyway currently so that's that's going to be the next one um you can acquire tokens in a desktop application integrated windows application how much effort do i want to put into this i'm almost tempted because this isn't going to be the last time i run into this i've got other tasks on here for key vault and similar and so i'd like to find a reasonable dev flow i don't know what the most reasonable dev flow is like i could have it what would be the best option here because i could i've already got a prerequisite on like the azure cli i'm just wondering if like using that to jenna token and rather than having the full connection string in the settings expecting the developers to just gen a token with the azure cli and dump that in maybe because i'm gonna end up doing a bunch of work for this around around making this work and i don't know if that's reasonable probably is probably is i'm gonna have to think on that i'm gonna have to think on that well let's get this committed because uh let's start with poll get me up on latest uh stage on stage uh yeah yeah yeah hush you're fine you're fine you pulled we're on we're equal so let's see toppings api into program cleaned up startup you'll note that that secrets json doesn't show up anywhere so yeah so starting work adding in ef core uh let's see need to solve best way for devs to get access to sql in dev environment because we could have terraform go through and and gen something up that would be an option i guess like i could have terraformed spin it it's been a dev account that doesn't really scale out nicely to all the developers but maybe that's okay maybe that's fine enough i don't know if i really like that idea i mean we could require something in portal since the developer's already going to need to log into the portal to be able to add their particular ip address to the firewall but this is one of those things that i i kind of have this thought of writing a script using the azure cli that does this just be done with it seems reasonable seems easy i'll put that on my put that on my my backlog of items uh let's see we'll add to the bottom uh create script to allow developer to add their ip address to firewall on sql server uh let's see dev only so set something like that up to automate this process effectively do it do exactly what this button is going to do seems easy enough doesn't seem terribly difficult to do but the question still remains the appropriate way to log into this server and how how should that dev experience look i don't know the right answer i mean in an actual org that's already aed integrated the problem solved the only reason i'm having problems is because my local account is a microsoft personal account in msa and it's not and at least according to that air not supported so i'm gonna think on that it's tired all right it's tired i'm tired i've been going oh wow uh let's see doing math in my head uh what is that 17 hours 18 hours long bloody time i'm a little tired more importantly the mountainous empty so yes it is sleep time kenny you should definitely be asleep definitely if you have thoughts on how the off should work or how the developer experience should be i am all ears um and you can sleep on it too and you can let me know tomorrow um and with that i think i'm going to tell everybody thank you for joining happy coding we'll see y'all next week you

Channel: Kevin Bost

Views: 22

Keywords: programming, terraform, azure, devops, github, tunawaffle, github actions

Id: fQd1IVlov5k

Length: 100min 37sec (6037 seconds)

Published: Tue Nov 02 2021