Migrating Databases with AWS Database Migration Service (DMS) - Demo

hi in a couple of previous videos we have seen demos of application discovery service for planning application migrations and cloud and year for migrating workloads to aws in this third demo we will look at a demo of aws database migration service or dms let's get started i am raghav vadhari and this is hands-on aws in this demo we will go through a few steps we will first talk about the architecture that we are trying to use in this lab it is assumed that you would have used the cloud and cloud formation template available here in this location to set up the lab environment and go through the first two demos which is using the application discovery service to uncover application data as well as using cloud endure to do migration of those servers from the simulated on-prem environment to the target aws environment which is us best 2 in this case this demo on dms depends on those two demos being complete now when it comes to the reference architecture we started with assuming two different applications running inside a private subnet on an on-prem data center and those two applications are wordpress and off bus erp and these application servers and database servers four in all have been migrated to the target environment on aws using cloud endure and we saw that the migration was successful we shut down the source environments we updated the dns to point to the new environments in aws target vpc and target subnet and then validated using the fully qualified domain names that those applications remain accessible and we considered the migration of those servers successful now one thing to note is the migration that we did using cloud endure where we are replicating each server like to like on the target environment is what we call a lift and shift migration it's also called a rehost strategy for migrating to the cloud so here you do not change anything within the server itself you just change the host from an on-prem probably a physical or virtual server to a cloud-based virtual server that's what we did now another strategy to migrate to the cloud involves re-platform and a good example of that is converting a self-managed database like mariadb or mysql or oracle to manage service like rds which supports multiple flavors of databases including open source as well as commercially available software and that is done using a service called database migration service which allows you to migrate a source database to a target database when either of them are on earlier platform now here in this reference architecture as we said there are two different applications the wordpress application itself uses um mariadb database on the backend and the off-base application uses three postgresql databases so for the purposes of this demo we will target to move both of them to aurora the first one mariadb will be migrated to around mysql and the three post postgresql databases will be migrated to aurora post sql managed database instances it will be three databases on a single rds instance now with that in mind let's start the hands-on demo so at a very high level these are the steps we will go through we will first create those two arm databases rds instances and then set up the databases on those instances create users to access wordpress as well as off base applications and then we will go and configure dms on the dms console we will create go through a few steps of configuring the dms and then we will execute the replication tasks which will migrate those databases and then execute a cut over update dns and validate let's get started so when we head over to our airbus console this is the cloud formation template we use to bring up the lab environment and we have gone through a few steps on the migration hub we have discovered using application discovery service four servers which were running two applications we had to manually group the servers into these applications and then we explored easy to instance recommendations and we also created a cloud editor project and then created blueprints and set up the blueprints and replication settings for the cloud india project and kickstarted a migration that data replication of those on-prem servers onto the replication server and then we executed a cut over on cloud endure which created these servers on the targeted aws environment then we switched shut down the old servers updated the dns and made sure that the new servers are working fine so with that these application lift and shift migrations are complete now we look at what are the tools that are available for migration other types of migration so dms being one more service so just to quickly check we have a few instances running and out of which these servers whose names and an on-prem sim dot env are the target servers that are created by cloud endure after the cutover has been executed and each of these are running in the target vpc and the target private subnet and those are what we did in the previous demo all right so now we move to the first step of the demo which is creation of the target rds instances so let us go to the rds console and start creating the two rds instances the first one being the aurora mysql for wordpress database so we go to the rds console and click on create database we choose the details we'll do standard create we'll do amazon aurora with my my sql compatibility for the first one which is wordpress and then we will keep the latest version [Music] we'll do dev test for the purposes of this demo we'll give it a name that is easily recognizable and then for credentials we will keep the same password that we have been using all right and then we'll choose a smaller burstable class we will go with a lighter weight instance to avoid high cost for bpc we will select target publicly accessible now upc security group we will choose an existing one and we'll select default and add the one that contains target security group and we'll pick the first one then we will move on password authentication this is wordpress db uncheck enable encryption enhanced monitoring and then click on create database let's give it some time all right the database is now creating we will go ahead and create the second rds instance as well this time around for postgresql for the off-base application we'll go through similar steps this time we'll select postgres sql compatibility and rest is very similar dev test password awsome id20 instance class burstable dvt3 medium don't create replica target select target security group in addition to default the first availability zone password authentication is enabled by default for postgresql so you don't have to do anything more here for database options you can leave this one blank because we'll be creating them through command line manually we will need to create more than one database for the rds instance supporting the office application no need for encryption no need for performance insights and no enhanced monitoring required so with that we will create this one as well oops all right i got this one wrong this needs to go in here this is admin sorry this office click on create database give some time and there you go this is creating so the wordpress instance is already created so you can see connection details here this is the username and password that you gave and this endpoint you can copy this endpoint and keep it handy let's copy this over to all right i think you get rid of this extra text all right now we have created both databases we will need both database instances so within these databases we will need to create the databases one more piece of information that you would need to note down is for each of the writer instances of the database clusters that we have created you go into the details and okay while this is still creating let's look at the wordpress instance that has already created and it is available so let's go to the end point of the writer instance and note that one down similarly for the office instance which is now available you have an end point that you can copy from here all right so these ones are the writer instances all right now let's go back to the wordpress writer instance remember we have a bastion host running in the private vpc that we can use to connect to rest of our instances you use putty to ssh into each of the database instances let's start with the wordpress db remember the wordpress db has already been migrated so the server we are connecting to with this domain name is the one running on the target aws environment so here the command from command to use to connect to mysql on the aurora instance from here would be this one all right we are connected to the rmi sql and we will use all right that's it we created the wordpress user on arrow mysql database now we'll move on to do the something similar for the post sql database that we created on our rds instance remember this was created as a cluster with no database name so we'll have to manually create that so let's open putty again and go to obviously obvious db server might 20 so we will establish a session from this bastion host this is the dns value for the writer instance into this go to your bastion connect it to opensdb server execute this adw70 again that's the one we are using it consistently to avoid confusion so we will do that so here we will execute a series of sql commands to create database three databases to be sure so this one let's do this we're creating three databases and we are done with that all right just to recap for the wordpress db when creating the rds instance we chose to create an initial database called wordpressdb here we created a user by name wordpress with this password and granted them all the required privileges in the case of postgresql rds instance we did not create any initial database here we are creating three different databases that are required by the off biz application so with this done we are ready to jump into our management console and start setting up the dms service so let us go to dms which is the database migration service as you said the high level steps to configure dma service are creating a replication network launching replication instances and then creating the endpoints for source and target the first thing we will do is to create subnet group for wordpress mysql replication so we will go to subnet groups and we will create a subnet group give it a name description is optional so i'll keep it the same we'll create it in the target vpc click on create subnet group you need to select target aurora and target private and click submit group similarly create another one this time for office epc would be target again and then target aurora target private and create subnet group all right now the next one would be to create a replication instance so go to replication instances and create a replication instance and provide these details this one is just a unique name here in his optional description this is plain text you can select a small instance this is fine epc should be target publicly accessible not required network configuration you select the wordpress subnet group that you just created and select the availability zone that we used when creating the rds instance and then under vpc security groups select default as well as target security group default target rest is fine so similarly we'll create a replication instance for postgresql we'll give it a unique name description instance class is fine epc should be target and not publicly accessible on security office is good value zone should be the same one security groups target security group as well as default and then click on create all right now now is the time to create a bunch of source and target endpoints so as we know from the earlier walkthrough of the architecture we are dealing with four different databases one mariadb and one postgres three post sql databases the first one is maria db2 mysql and the rest of the three are porsche sql toposque sql so for all these migrations that we have planned we need to create an endpoint for each of the source and each of the targets so those are eight different endpoints in all i know it's tedious but it's worth creating by hand for a small demo like this for large scale migrations you would be doing scripting to do creation of these endpoints in mass let's start so we are at the third step of configuring dms all right you go to endpoints and click on create endpoint so a couple of things we'll need here one is the private ip for each of the ec2s that is hosting the source databases so let's go to the easy to ins console and go to wordpress tv and capture the private ips similarly for office db all right let's save that going back to the ds dms console we are at create endpoint screen where we can start creating the source endpoint for for wordpress we will supply the following details and point identifier on this one source engine is going to be mario db herein is optional provide information manually server name would be the ip address that you have just copied for mariadb which is wordpress db currently what will be 3306 for paradib right rest is optional so one good thing to do is to test the endpoint test connection so this is wordpress see the text test is successful it's all good we'll create more endpoints this time we'll create a target endpoint this one is rds select rdsdb instance and we're creating it for wordpress instance one hmm provide information manually server name password awsome id20 it's a good idea to run this test choose target select wp and run a test all right the target connection was also successful we'll similarly go ahead and create the remaining six source and target endpoints similarly we are creating a source endpoint for the second office database which is of bezel app so we supply similar details and we are running a test successful so we create an endpoint i want to create a target endpoint for the same particular amount will be rds of this the lap all right the text is successful so we have one more pair of source and targeted points to create so this one is for office tenant so let's do that source engine is going to be pushed sql manually get the source ip address for the server name port 5432 username is going to be of this and database name is going to be of this tenant password is your standard aws mit funny let's test the connection all right so now we create the last endpoint which is the target endpoint for of distance give it a useful name manually password database name is of this tenant run a test all right this looks good so we now have eight endpoints created for source and for target that should be good for us to proceed to next step all right with configuration of tms complete we are now ready to move on to the last step which is the migration and cut over the migration executed by creating a set of dms replication tasks for each pair of source and replica target endpoints let's jump into the dms console and we will create a total of four migration tasks one for each pair of source and target so there would be one mysql replication task and three post sql replication tasks for each database that needs to be migrated so for mysql let's go to left side navigation has database migration tasks click on create task and we'll have to provide some details for each task task identifier descriptive name is optional replication instance for wordpress is wp source database for wp target database for wp and you migrate existing data you can use default values for most of these things for table mappings you go to wizard add new selection rule enter a schema click on create task you can have automatic creation of task upon create so it will launch the replication task as soon as this task gets created so that will save some time so let's continue to create tasks and in the back end any tasks that we create all right i think we will need to use a new window to create a new task that's fine so let's move on to the next porsche sql task this is off this replication instance would be off base source of base target of this migrate existing data rest of it can stay the same add a new selection rule enter a schema wildcards you can automatically click on create start migration task all right as you can see one has already created and it is starting let's continue to create other tasks this one is for of this tenant of bazel app application instance is going to be off base open the lab source of the lab target migrate existing data add new selection rule enter the schema leave it as default automatically create the first one is already running you create last migration task this one is from this tenant replication instance is going to be off base sources of distance target endpoint of piston end table mappings add new selection rule enter schema leave wildcards automatically start the migration task upon create and click on create task so now you have created all four and all of them are in different states and if you give it enough time all of them will be running and reach the load complete stage at which point all your source databases using the source and target endpoint migration endpoint configurations have been replicated to the target instance by the dms service and they are ready for your cut over okay after waiting for a few minutes we see that all four databases have fully replicated now it's time to shut down those source servers do a dns update which is the actual production cutover and then validate whether the applications are still running fine once the dns update has been completed using the target instances that we have just replicated so in order to update the dns and shut down the source database we will log into bastion and go to the wordpress database initially and we will look at the ns lookup records and we will load the aurora wordpress endpoint that we have noted on earlier into a local variable and then use it to update the ns lookup database records dns records and then we will shut down the server and after which we can validate that the wordpress site is still working and we will repeat the same steps for the offbeat server as well let's go on to the bastion host let's first bring the end point of of the server sorry um the writer instance of aurora copy it here we'll copy this all into a bastion host notepad all right let's go to wordpress db let's look up so it's pointing to 10. 0.1.121 which is an ec2 instance so let's execute this then when you do and let's look up for this domain name again after this update you should see the rds instance endpoint with this you're now ready to shut down your old server so now the wordpress server wordpress db server is down so if your cutover is was fine your wordpress instance should still work so you can see that the cutover seems to have been successful without any downtime so you're still able to access your application while the database has been migrated from your ec2 server to an rds instance we'll do similar steps for the office server as well so the only difference is this is of bizdb and then for this address you will bring the writer instance of of this rush is all the same so let's copy this to your best in host notepad let's try this look up first it points to your east instance let's load this up all right let's do this again okay we did a small mistake this host name needs to be office db i think we wrote the wordpress db so let me fix that all right let us fix this luckily we saved this and do this this time with the correct hostname all right now if we do ns lookup on of this db it should point to of this db's rds instance similarly if you and let's look up wordpress tv it should give you wordpress instance so now you are ready to shut down your hdb instance as well okay now let's test both applications now let's first test wordpress still seems to be working fine let's see if it has some of our test comments that we put in looks good let's see if you can run the office application to admin office all right seems to be working fine let's see if it has some of the entries that we made new store was something we created all right looks good so that brings us to the last step which is validation which we just did so let me correct this all right so we went through a long process and we detailed it out as event so we started with the databases running on ec2 instances that have been migrated using lift and shift strategy using cloud endure and once we had applications validated and running in that mode we then used database migration service to set up replication endpoints replication instances and source and target endpoints for each of your database that you wanted migrate from wanted to migrate from an ec2 instance where you are self hosting and managing the database to a managed service like rds and once you have set up all those replication instances and the source and target endpoints correctly all you need to do is create replication tasks and execute them and once the replication tasks show the status as load completed you are ready to do a cut over by shutting down the old server and redirecting your dns entries for the database endpoint to point to the rds endpoint so hope that was a useful demo of dms service catch you at some other time thank you

Channel: Hands-on AWS

Views: 835

Keywords: aws, cloud, migration, database, rds

Id: JTSLF42bv9Y

Length: 52min 22sec (3142 seconds)

Published: Sun Sep 05 2021