Deploy .NET Core to AWS Elastic Beanstalk with RDS Database

hey everyone wes here with productivedev i made a video a few months ago demonstrating how to deploy a.net core application to aws using elastic beanstalk but what i didn't cover in that video was how to go about attaching a database instance to your web application and actually deploy that with your app as well to elastic beanstalk so in this video we're going to look at how you can go about using amazon rds to attach a database instance to your web application so i apologize for the delay on this video i know a lot of you have been asking questions about this topic in particular and so i hope this video clears some of those questions up for you and we can take a close look at how we can attach a database instance to our web application if you enjoyed this video i would really appreciate it if you liked it and subscribe to the channel for more videos about web development devops and machine learning thanks for watching and now let's go ahead and dive in and look at how we can set up our application and deploy it to aws so before we get started you'll want to make sure that you have the net core sdk installed and so for this video i'll be using the 3.1 sdk and it is cross platform so just make sure that you download the correct version for your operating system installing docker isn't necessary but i will be using docker to make local development against a local postgres database a bit easier and so we can look at how we can configure our application to run against postgres locally or an rds database in the cloud when we deploy it to our staging environment so what we need to do now is create a project directory for our new.net application and so to do this i'm just going to make a new directory and we'll call this beanstalknet and then i will cd into that directory and so right now we just have an empty directory if you're using windows or you're otherwise not using a terminal just create a new folder in your projects directory for the.net project and now we'll be using the.net cli to scaffold out our new project so we can simply type net new mvc and in our beanstalk net directory this is now creating a new.net core mvc project so if we take a look at what's in the directory now we can see our models views and controllers directories and then we have an appsettings.development.json cspraj here and a www root so let's go ahead and open this up in an ide okay so our project has opened and if we take a look in our controllers directory we can see we have a single home controller here and then of course we have directories for models and views but what i'd like to do is to wire up entity framework and then hook up a postgres database so that our application can connect to postgres and we'll create a new controller action to retrieve some notes from that database so this is going to be quite simplified in terms of project architecture typically you would have a data assembly or a data access project rather in a separate class library but for the purposes of this demo we're going to just create a new name space here so we're going to create a folder and i'm going to call this data access and then within data access here we're going to add a new class and this is going to be called appdbcontext and then this will be our dbcontext for our efcor database connection so this needs to extend db context and for us to use this we need to install entity framework core using nuget so we'll manage nuget packages and here we're going to install microsoft entity framework core and for our purposes we're going to use version 3.1 and we're also going to install microsoft entity framework core.design also version 3.1 and now we're going to install our postgres dependencies including npg sql and i'm installing version 4.1.5 and then finally we'll install mpgsql.entityframeworkcore.postgress also here version 3.1.4 we can now import the missing reference entity framework core so that we can extend appdb context with db or from db context and what i'm going to do here is just create a constructor we won't go into much detail about any framework core in this video but there are i have plenty of other videos that go more in depth about that we really just want to focus on the deployment aspect in this video i'm going to create a single virtual db set we're just going to have one table in this database for recording notes into our database so in our data access directory i'm going to create another directory called models again you would typically have this in a separate class library but i'm going to create a class here called note and this is our poco we'll have an id and our notes will have a date time created on and an updated on and we'll import system and then we'll just have a simple string content for our note okay so now if we head back to app db context we can import this from our data access namespace and then just for the purposes of our demo i'd like to seed our data when it starts or seed our database rather with some data and so we can override on model creating here which takes a model builder and we're going to have some notes which is a new list of note data models for this we bring in collections and so let's just make a handful of notes here our content will be finish youtube video and then are created on an updated on will just be some time that we create now and so for that we'll come up here and i'll save our now is equal to datetime.utc now and let's create two more notes to put some additional data in our table so we'll bump the ids here go to the gym go to the grocery market okay so now we have three notes which will actually get created when we start the database it'll get seated and we can do that like this we can use this model builder that we have as part of this override and we can say model builder entity of type note has data notes okay and we can get rid of these extra parens so with that complete let's head into our startup.cs and we're going to take a look at our configure services method so right now it's very minimal we just have add controllers with views but here we're going to wire up our db contacts so we can say services.adv context of type appdbcontext which was the class that we just created and here we can pass it callback opts ops.use npg sql get configuration.getconnect string and now we're going to have a connection string to our database that will be called notes underscore db and so what we need to do here is to actually create that connection string in our app settings and so we'll go to appsettings.development.json to begin with here and i'm simply going to come down after logging and have a section called connection strings and then this is where that notes db key will live and we need this to be a postgres connection string and for our case that's going to look like this host is equal to localhost port will be port 5432 username notes dev password notes dev and database notes dev okay so in order to well connect to our database locally we'll need to have a database running that we can connect to at this connection string so in order to make that a little bit easier you could run well first of all you could run postgres just on your machine if you're on mac os you could install it using brew if you're on windows you could just use the standard installer in our case we're going to actually fire it up in a docker container because that makes things really easy for development and we can spin it up and take it down very quickly and here just in any editor we can create a docker compose file and we're going to keep things pretty simple here we're going to use postgres13 running on alpine and we'll specify some environment variables here we'll simply map port 5432 from the container to our host and we'll have a docker volume here named volume postgres notes data which will map to data slash data db in the container and then we'll restart on failure and then we can declare our volumes which in this case is just our postgres node's data volume okay so with that complete by the way i'll put a gist for the docker composed in the description of the video but we should be able to run docker compose up and so postgres should be starting here my terminal font is just extra large for the recording but but the logs will look better if we were to zoom out a bit and you can see that postgres is just running and waiting for a connection so i'm going to open up a new terminal window here and in fact i'm going to stop this and run this in tmux this way we can do it in two panes so we'll spin the postgres container back up but then here we'll also run the command dot net run so we haven't actually applied the migrations to our database yet although our application does have the ability to connect to it so before we get much further along let's just kind of smoke test and make sure that we can hit our mvc app at localhost port 5000. so here we are at port 5000 it's a simple mvc app that's been scaffolded out for us and really we're not doing anything except being able to hit a few different routes here our home route and our privacy route so let's go ahead and create a new route where we can pull those notes that we would like to see into our database into a razer view so i'm going to close this and we'll come back and we'll stop our app server and now we'll come back to our code and then in the controllers directory i'm going to go ahead and create a new class and we're going to call it notes controller and if we take a look at our home controller we can see that the controllers that we need to build really just extend controller so i'm going to remove some unused directives here so in our notes controller it's going to look very similar we're going to extend controller and then you can also see the structure in our home controller we have dependency injection setting up our logger and then we have a few different action results to resolve views so i'm going to copy that for the time being we come back into here we'll bring in ilogger and we'll update our constructor here to be a node's controller and the logger to be of type nodes controller in both cases and in fact we'll change our i action result to be a public async task and then we'll return a view now for the view to actually work here we need to head into the views directory and then i'm going to create a subdirectory called notes and then just as the home here has an index.cshtml we're going to add our own view here and we're going to call this index dot cs html as well okay so the model that we're going to use here will be a view model and so for this what we'll do is we'll take a look in the models directory and we're going to add a new class here and we're going to call it notes view model now maybe we'll also have a public class note view model which will look a lot like in fact precisely like the data model that we have in our data access layer so what i'm going to do is just actually copy the various auto properties on that and then we'll paste it here i do like keeping those view models separate from the entity models and then our notes view model we'll simply have a list of note view model and we'll call this notes okay so now we can use our view model and the way that we're going to do this is we'll head back into the index that cshtml for our notes view and this is going to be a notes view model and i'm going to simplify all this we don't we have a layout here and so we don't need to redefine the layout and so let's just keep it pretty simple here we'll have an h2 which will say my notes and then we'll have an unordered list and then we'll just use some razor syntax here to say 4h var note in model.notes we'll create a separate list item that we can access with spiral note and we'll just grab the content off of that note just to get something wired up and working here if we want to set the layout explicitly and the title on the page we can do that like this we can say view bag dot title is equal to my notes for instance or whatever you'd like and then the layout here we're just going to use underscore layout which is our layout.cshtml file and you can find this as well in the views shared subdirectory so here we're just explicitly saying that the default or the layout for this is our default shared layout okay so with that wired up now we just need to head into our nodes controller and we need to pass the correct view model to our view here and so we'll say our notes is equal to our db context which we need to inject so let's go ahead and do that we have a logger let's also inject our appdb context so import the missing reference here we'll go ahead and create the backing field and i'll name this dbcontext dot notes and we're going to keep things incredibly simple here and just get all the notes back from the database again just for demonstration purposes we're going to select that into a new notes view model and actually we should select each note into a note view model and here we're just mapping from the data model to the view model with the id created on the updated on and the content and then to actually query our database we'll say to list and we can actually make this asynchronous so here we'll await notes and we'll use the async version of to list and then we'll simply pass that result to our view oops and we actually need to make this into a viewmodel so we have view models for each of our notes and now we can just say var notes view model is equal to a new node's viewmodel with the only property on it being notes and we set that to our notes okay so now we can actually do that properly we're not using our logger so we might as well remove it okay so if this were a real application there were there would be a number of other things that we'd want to consider here first of all we would probably want to paginate results that we get back from the database rather than just simply calling dbcontext.notes and this is going to query all of them effectively doing a select star on the notes table and so typically for a web application you'd want to paginate that to something that made sense for the view in this case we're just doing this to get things wired up quickly so that we can see this thing get deployed next what we need to do is we need to actually make sure that our database migrations will be applied to our database locally to begin with so what i'm going to do is i'm going to come back here and in our directory i'm going to run the command.net ef migrations add initial and so this is going to create a migration file for us so ef core is just looking through our db context class and it's finding our notes model that we created and it's finding the seed that we created for those three notes and so it has now generated a migrations directory for us with this initial migration and so we can see that here contains an up and a down method so this will look familiar if you're familiar with database migrations in general but here we can see that it's scaffolded out some code here which is going to be used to actually create those that notes table with a primary key on the id property and it's also going to insert data for us and you can see that it's created that for us as well so it's a nice kind of interpreter that's used to generate code or compiler in a sense that's used to generate this intermediate code which will then get run against the database that we're connecting to so what we can do now is run.net ef database update okay and so it says that it's done so let's go ahead and try to connect to our database so what we'll do is we'll say p sql h localhost the user here needs to match the user from our connection string which you can see is notes underscore dev and that's what the password is as well so now we're connected and we can run dt to see the tables that are created and we can see that we have a notes table so let's go ahead and select star from notes and we can see our three notes in our postgres database here running locally in our docker container if we stop docker and then we try to connect again we notice that we can't actually connect to our database but since the data itself is stored in a named volume though the container stops the data is persisted and so without needing to run the seed again we can just connect back to that database and run our query again and things are looking good okay so that's a quick example of how to get our database set up locally in the dot net core project but now let's go ahead and try to deploy this app to amazon or aws and what we'll do is we'll use amazon elastic beanstalk for this now beanstalk is something that you would want to consider using for i would say relatively simple app deployments it doesn't give you the sort of fine grain control over the app deployment process that you might want for more complex setups but it is a really great way to deploy something incredibly quickly and in fact it's still built for production it's just in general built for i would say more simple projects and to that end it's provide it provides a lot of abstraction over some of the details that you may not want to worry about again for simpler projects so let's go ahead and head over to the aws console so once you've signed into the aws management console head over to elastic beanstalk and you'll see that i had an app here previously and what we're going to do here is we're going to create a new environment and so here we'll select web server environment we'll give ourself an application name here net core demo and you can even look for a custom domain prefix here and if it's available it will let you know for the platform we're going to deploy on.net core on linux and at the time of this recording we just have this single platform branch option available and the platform version is 2.0.4 and in fact we're going to leave the sample application code selected for the time being and we'll just select create environment now this is going to actually automatically provision everything that beanstalk needs to manage the deployment that we will be creating and so in the background it's provisioning ec2 instance and some of the other things that are needed to basically deploy our application and so this will just take a few minutes while that's happening let's head back to our code so note that we have an app settings.development.json which is great for local development this is how we're connecting to our database the app itself doesn't know anything about docker but on our laptops here or our local in development environments we're running postgres in a container and it's just mapped to our local port 5432 from the postgres instance in the container and so what we'd like to do instead is be able to connect alternatively to say a staging environment or a production-like environment in the cloud and for this we don't want to use our development settings but we'll create another settings here and so rather than app settings.development.json i'm going to call this one appsettings.production.json or you could call it staging or whatever you like here and so we'll no longer be running on localhost and maybe our username will be just like prodb or something password should be much more secure than this but for now we'll just have a simple password be the same and here we'll be running a database on notes underscore prod i would encourage you to well for one thing don't check this appsettings.product into version control and also create a more secure username and password for the for the application agent that will be connecting to the the postgres database in the cloud so if you want to see how to for sure not check that into version control we can create a git ignore file and we'll exclude bin the obj directory for the artifacts that are created there and then we also will ignore dot appsettings.production.json here and so now if we were to get in it and get status here you'll see that our app settings.development.json will be committed but our app settings.production.js will not be committed in fact we should also add one for the ide the ide metadata okay so that's looking a bit better okay so while that's been happening our environment should be about ready we can always keep an eye on it over here and when it is ready you'll see a health badge here saying that it's okay and so in fact we do see that it's okay for the app that we just created and if we head to the url now with that sample application this is just what aws provides as a sample app for our deployment but of course what we would like to do is to deploy our code to our beanstalk environment and so let's look at how we can do that now we know for sure that we would like to connect a database and this actually takes a bit of time to provision so while we prepare to package our code we should start setting up our database in the background and so to do this we'll head over to configuration all the way down to database here which is currently at the bottom and it's going to fetch some rds configuration options rds is one of the several aws services that allows you to host a database in the cloud and so we're going to use postgres and for now we're going to put it on one of the smallest sizes that we can which is in this case a dbt2 micro and we'll just leave it with five gigs of storage so this is clearly not really provision for database of any of any size but just for demonstration purposes now the username and password here should be what we want or what we provide in the connection string here so we'll create a username prodb and a password prod db again you want to be much more secure than this and for now we'll leave it at low availability which just means that our database will exist in one availability zone as opposed to multiple again for a true production environment where high availability would be a requirement then you would want to have your database hosted across availability zones with multi-a's either so we're going to go ahead and create this oops and we need a more secure password which i suppose is a good thing so we'll go ahead and update the password here and i'll update it here as well so this is going to now or i should say elastic beanstalk is now going to provision that postgres database for us and set everything up with those initially with that initial user we click on view events you can kind of keep an eye on it here notice that it's creating a security group for it which is going to allow the rds instance that we create to communicate with the app server that's running as part of elastic beanstalk and so it's going to wire those things up for us automatically and this is just going to take some time so while that's going on let's go ahead and package our net application and then we can actually upload it to our environment so if we come back here the net cli has a command for this we can run.net publish and then we're going to use a release configuration so dash c release dash o site and so this is going to create a site directory first it will build and then as mentioned it will create this site directory so if we check out what's in here we can see that this is everything that's needed to actually deploy our application so now what we need to do is we need to actually zip this these this site directory essentially this deployment artifact into a zip archive and so you can do that on whatever operating system you're on in my case i can run the command zip dash r and then up a directory and i'm going to create a deploy bundle.zip and then i'm just going to zip everything in this current directory up okay so if we go up a directory i now have a deploy bundle zip file and so this is the file that elastic beanstalk will accept and it can actually just read through the file determine what it needs to do to deploy the application to our app server so let's go ahead and do that this is actually still going to be creating the rds instance likely this should take between anywhere between 5 and 10 minutes i would say so come back to environments and here we're just going to need to wait until the rds instance created at which point we'll be able to upload and deploy once this is complete you'll notice that the health status of the environment goes back to being okay hopefully and you'll see from the recent events here that the environment update completed successfully and in fact if we show all of them here we should see also that it created an rds database with this name and so if we head for one over into our configuration again then we scroll down to where the database is we can see that we have some additional information here for instance we have our endpoint so if we click on this link we actually get taken over to the rds console where we'll see this and any other rds instances that we have running here in this ac and so we've got this single identifier for the database that was just provisioned automatically for us by elastic bean stock so we can see the vpc security groups here as well and here we see in the inbound rules we have the availability to connect to postgres uh we have a source here of sg0ca et cetera if we take a look at this string and then we head actually back over into elastic beanstalk and then we go to configuration head down to instances here and then select edit we'll see our ec2 security groups and these will contain the same string that we have here for the security group so if we just ctrl f and paste that here we can see our security group here for netcore demo env okay so this will allow the ec2 instance that is being managed by elastic beanstalk here that's hosting our web app to connect to the database that we have just provisioned so now with that what we can actually do is we'll head back into our environment and we'll go ahead and upload and deploy that zipped archive in my case it's called deploybundle.zip and then we can give it a version label here and then we'll just select deploy so this is going to actually upload that zip file you can also upload from an s3 bucket directly and so this will happen much more quickly here it's just going to deploy our app if there are any problems then we can always take a look at the logs and then we can request the top 100 logs which you'll find in here and then sort of inspect what happened but hopefully this works out for us okay now that's complete we can see that the environment update was completed successfully so if we head over to our url now and you can shift refresh here to make sure that your cache is cleared for this purpose and we can actually see our application now one thing that i notice here is that i haven't actually created a notes a notes link we can do that as well in our code and then redeploy if we hit it directly one thing we'll see here is we should see an error at this point and this is because we haven't run our migrations against our production database and so i wanted to demonstrate this because we can actually take a look at our logs here and then if we request the last 100 lines this is going to give us a file that we can take a look at and if i just search for error here we can see that we have an error occurred using the connection to the database notes prod on server and we haven't actually defined our server name and our connection string so we need to do that and then we need to run a migration on this database so to do this what we'll do is we'll head over into our beanstalk our elastic beanstalk dashboard we'll head back to our environments and then our app and then we'll go to configuration come down to our rds endpoint we'll click this and we're just going to actually grab that endpoint so that we can use it for our connection string so we'll click on our database identifier and here's the endpoint so we're going to take this and we're going to head back into our code and the host will now be here so we now have our host make sure that we also have the port the username and the updated password specified as well as this database notes underscore prod so once that's done we'll package again so what i'm actually going to do here is i'm just going to remove our deploy bundle and i'm going to remove our site directory and then i'm going to run the net publish command again to create that site directory all over again if you do need to create backups of old deployments then you won't want to delete those but you'll want to keep them around and manage them somehow but our site directory is back so now i'm going to go into the site directory and i'm going to run the zip command to recursively zip everything into our bundle again and then we're going to upload this once again so we'll come here and now we'll upload you'll notice that the version label gets incremented automatically here for us which is kind of nice and so this will upload and then just take another one or two minutes to deploy once again while that's deploying i should note that we could have also changed our shared views if we come into shared and then layout here we can add a list item for our notes controller so feel free to add this as well this is the index action on the notes controller and so if you want to repackage and deploy again then we'll actually have this in the nav bar but we'll still be able to hit it from the url as well okay so now that that's complete we can now go ahead and click on our url again and if we head to notes we're going to get an error this time but it should be different so we head to notes we get an error now we can go check out the logs we can once again request request the last 100 lines and then note the order here the most recent log is actually at the bottom and now we get a different error we say an error occurred using the connection to database notes broad on server and it's trying to connect to the correct server this time so here it looks like we're actually getting a password error password authentication failed for prod db so this could be an error in the connection string again it could be that i mistyped it when we created the default user and so if we want to sort of investigate this ourselves we need to actually modify our security group so that we can connect to that database from our development machine as well so to do that what we can do is we head over to our configuration again head back down to the database link and in fact i have this open in another tab anyway but we'll come down into the database and we're going to look at this security group that we have set up so we're going to come down actually to the inbound rules and i'm going to edit the inbound rules here and we're going to allow us to also connect to postgres which is port 5432 from and you can select your own ip here so i'm going to select my ip and then save the rules now that that's done i can actually run the psql command and specify the host of the rds instance and the user prod db so now we should be able to connect to it and i'll try to connect uh it looks like the password authentication the password that i provided was incorrect so let me try again here i think i know what the correct password was okay now we're getting a different error which is that we couldn't connect to the server fatal database pro db does not exist so we should have a postgres database available on it though so what i'm going to do is i'm going to add this flag d postgres make sure i type in the correct password this time and now we're connected to the postgres database so one thing we can do here is we can list the users so we can see that or list the databases rather and so with backslash l i can see we have a postgres database but we don't have the database that we need to connect to and so let's go ahead and run our database migrations so here what we need to do is i'm going to export asp.net core environment to production and now i'm going to run the command.net ef database update this is going to run our migrations against the production database which we should now be able to connect to you from our development environment after updating the security group okay and now we see that password authentication failed so i'm going to go ahead and update the password and it was just a typo so there we go let's try again and so this time it actually succeeded so now if i list the databases again with backslash l we can see we actually get notes prod so let's change to notes prod database and now we're going to list the tables that we have here with backslash dt and we can see in fact we have our notes table in fact if we select star from notes we have our three notes in our cloud rds instance okay so with this i think we're ready to actually redeploy so we need to redeploy because we've updated the password so we're going to come back and then we'll head back into our environments go to our environment where application is deployed we're going to need to create a new deployment artifact so i'm going to disconnect from our database go up a directory i'm going to remove the site directory i'm going to remove the deploy bundle and now i'm going to run.net publish c release output site once again and go into the site directory and then rerun the zip to recursively zip everything in this directory come back and for the third time upload the next deployment so we'll upload our deploy bundle and we'll deploy once again okay so that happened rather quickly we can click on our link once again and now we can see we actually have our notes and if we click here we can now see that we're actually retrieving those notes from our database and showing them in the razer view so this was just a quick demonstration of how you can get a postgres instance wired up to your asp.net core mvc application and deploy everything using elastic beanstalk so if you enjoyed this video i'd really appreciate it if you liked and subscribe to the channel and be sure to leave comments or questions below thanks for watching

Channel: Wes Doyle

Views: 6,114

Keywords:

Id: 5Eyjc1l_geY

Length: 40min 46sec (2446 seconds)

Published: Sun Nov 15 2020