AWS DynamoDB For The .NET Developer: How To Easily Get Started | AWS LAMBDA SERIES

amazon dynamodb is a fully managed nosql database which provides fast and predictable performance with seamless scalability with dynamodb you can create database tables that can store and retrieve any amounts of data and serve any level of request traffic dynamodb supports on demand backup point in time recovery of data automatic expiry of items and many other features dynamodb is a fully managed database which means you don't have to spin up any servers install softwares and maintain the upgrades of them this is all done for us by amazon web services also called as aws in this video let's learn more about dynamodb the core components and how it can work from a dotnet based application we will learn how to make create read update and delete operations from a.net application how to connect to a dynamodb and also use the sdk that comes by default hello everyone my name is rahul and welcome back to this youtube channel if you're new here i make videos around cloud devops and dotnet i'm sure one of this interests you so make sure to hit the subscribe button to help me grow this channel without much delay let's go to the command line and create a new api application that we will use for this video so i'll use the dotnet cli and specify the new command and the web api template this will create a new asp.net web api application in this current directory i'll open this up in rider ide to do that i'll specify writer 64 and specify dot to open the current folder i have the api application opened up so we have the program.cs the startup.cs as in any asp.net web api application and we have a default controller which is the weather forecast controller now this currently creates some dummy data inside this and returns on the get call let's run this application to make sure it is working fine the application is running so it launches the swagger endpoint which is set up by default for this api application so we can go to the weather forecast endpoint use try it out and click the execute button now this returns back the hard coded data that we saw in the controller endpoint now let's move this data into a dynamodb database and see how we can access it from there we will also see how to add items to it and then update and delete them as well let's switch over to the aws management console which i have already opened and we can go to dynamodb since i had recently visited this service it comes up under here if not you can search for dynamodb in this top search bar and this opens up dynamodb in aws dynamodb is not an instance based application you cannot create an instance for each of your application this dynamodb service is unique for this particular region and your account so inside here i can start creating tables now i have the region selected as sydney which is ap south east 2 which we will be using in a short while the core components in dynamodb are tables items and attributes a table is a collection of items and each item is a collection of attributes let's explore each of these terms in detail to start creating a table you can either click the create table button here or expand the menu on this left and go to tables and create a table from here as well so let's click create table to create a table we need the table name switching back to our application in rider we have this api returning a list of weather forecast so let's navigate together forecast and we can see different properties inside here which is the json data that gets returned for this api so we'll create a table that matches exactly the name of this type this is purely for conventional purposes and i'll show you how you can override this name if you want to so let's copy this navigate back to aws and specify the table name as weather forecast next it asks for a primary key which is a partition key so before we provide that let's understand a bit more about primary keys in dynamodb dynamodb supports two kinds of primary keys one which is simply a partition key in which case it is composed of just one attribute in case when a table has only one attribute primary key these values needs to be unique which means no two items can have the same partition key value it also supports partition key and sort key in this case this is referred to as composite primary key so this has two attributes in these cases dynamodb uses the partition key to create a hash value the hash value determines the partition that the item needs to be stored within the partition these values are sorted based on the sort key now the combination of these two attributes must be unique for any item so you can have items that have the same partition key or the same range key but both of them together must be unique if we head back to our api application we can see that the weather forecast has a date and it specifies the temperature and a summary normally the weather forecast is for a place or a city which is missing in this particular data object so let's add in a new property which is a string and specify the name as city so this will represent the city for which this weather data is applicable now if we navigate back to the weather forecast controller we can get in a city because you always want to get the weather forecast for a particular city so let's take in the city name as part of the controller to make it working by default let's keep a default city name i'll keep it brisbane for now let's hard code the city when we return back the data so let's specify city the same as the parameter that comes into the web api so this works as expected and this is going to return a list of weather data like before now in this case a weather data is created for a particular date and a particular city so the combination of the city and the date is always going to be unique now in this particular case i need the data based on city names so we can keep the city as to be the partition key so the data will be partitioned across the city names now within each city name for a date this is going to be unique so we can keep the sort key as the date so a combination of these two will be unique so let's head back to the console and continue creating this table so here for the partition key let's keep it city note if this is case sensitive so since my properties start with a capital letter i am naming it accordingly now i also need a sort key so let's click this check box now if you were to specify just city then that's the case where it only has the partition key so let's specify that sort key and give it date which is the property that is there inside the weather forecast as date by default it uses the default settings which you can override by clicking here you can select the read write capacity mode which is basically determining the pricing of this table if you click the link here it takes you to the pricing and shows the difference between the two modes now for the on demand capacity this is usually used with unknown workloads and the provision capacity mode is usually used when there is a high predictable set of traffic now navigating back you can see i am running on a free tier so i am only eligible for provision now i don't want to get charged extra for this so i will leave it like that now with provision you can also specify the read and write capacity which you can learn more about in the capacity calculator so let's skip this advanced settings for now scroll up and use the default settings let's click create and this is going to create a new table for us inside amazon dynamodb under this sydney region so you can see here we have a weather forecast table created if i switch over to a different region let's say if i go to singapore this table does not exist there i'll have to create a completely new one now switching back to my original region which is sydney for now clicking on the table name navigates onto the table details and you can start creating items inside of this you can see the high level details of this table right over here so this specifies the primary key sort key and the other details associated with this table to create items you can navigate off to the items tab and start creating items here to create one you can click the create item and this pops up a nice little dialog to add so this by default has the city and the date defined this is because this is our primary key so any item added to this particular table needs these two properties it can have any other attributes but mandatory have these two to enter the data in a json format you can click the tree and select text now this is pure json format but this still needs these two properties let me paste in a new item so i have the city the date the summary and the temperature which are the same properties that was there in our weather forecast class in c sharp so this adds a new item to this database so let's click save if you want to add more items you can again click create item and also specify the values inside this editor now i prefer this text mode so let's select that and replace it again with new values now in this case i have the city name as the same which is brisbane however the date values are changing so i have 2021 4 or 5 which is april 5th that's today and also i have the data for sixth now this time component is also making it unique so i could have values which is having a different time but for the same date but let's stick to one value for a date in this particular scenario now here we have tables which has items and which has attributes inside them two of these attributes constitute the primary key which is a composite key and there are other attributes now if we try to create a new item let's select test again and create a value for the 7th of april and if i remove the city name and try to save it it says it cannot process this request because that is mandatory so let's add that value back in and click save so this is going to create a value for the 7th of april as well now we have values in here for three different dates we've now seen the core components of dynamodb and how all of them fit together now this is schema-less which means you can have any different type of properties on each item dynamodb does not mandate all items should have the same properties but on a bare minimum it should have the primary key properties now when you get started it might be a bit tricky to start choosing partition keys and sort keys now dynamodb has a well-documented page which i'll put in the link below which walks you through different considerations that you can make when choosing a partition key so you can go through this and understand more about this in detail but as you can see in each aws account for a region it can have multiple number of tables each table has a partition key based on which the values are partitioned and stored physically on different partition ranges now each of these table items which is a row inside here has a primary key which can be a composite of partition and sort key and it has other properties now that we have a good understanding of dynamodb and the core components let's switch back to our application and make it start using this particular dynamodb table that we have added so switching back to rider let's modify this weather forecast controller to start using the dynamodb to do that we will need to add a new get package so let's go over to the project right click and say manage nuget packages let's search for dynamodb and this pops up the aws sdk to dynamodb v2 package this is the one that we need so let's install that into this current project and click install the package is installed successfully so let's close these and start using it inside our code the nuget package which we just added provides an i dynamodb context so we can inject that inside here into this constructor now we could directly create the context inside the controller but it's generally preferable to inject in these dependencies so let's create the dynamodb context create a backing property for this let's save this onto a class variable so we can create a read-only field which is underscore dynamodb context and that's getting saved inside the constructor now to start using this let's modify this kept method let's move this existing dummy data method into a different method i will use this in a short while so let's say generate dummy weather forecast and click next so this refactors out that existing method i'll use that later so let's remove this and start using the context inside here so we have the dynamodb context we can use the query async method to start querying the dynamodb table so specifying that it prompts us to specify the type so in this case this is going to be weather forecast this has multiple variations one which takes just a hash key value and one which takes a hash key value and an operator so let's specify the hash key value for now now the hash key value is the same as the partition key which is used to create the hash hence the hash key value the sort key that we created is also referred to as the range key this is because it specifies the range of values within the partition now these two names are often interchangeably used so don't get confused so partition is hash because it creates the hash key and the sort is the range because it provides the range key so let's specify the city name in this particular case because that's what our partition is now by default it has the value brisbane so let's none call the get remaining sync which gets all the items inside this particular query so let's make that call now since this is an async call let's add in the await and also specify the sync and a task return type so this is returning now a task of i enumerable of weather forecast let's also make sure to return this data from the dynamodb once its fetches back with that setup this controller is now using the dynamodb context to query a dynamo database now we'll need to set up the i dynamodb context so that the application can resolve it when it asks for this controller to do that let's head over to the startup.cs which is where we specify the dependency injection now if you're new to dependency injection in asp.net i have an entire video that walks through it it will be linked here or in the description below now coming to the startup class under the configure services method this is where we need to specify the dependency injection services so let's collab these and start registering the services so let's specify services and add in the i dynamodb context now this is a thread safe class so we can add it as a singleton instance so let's specify singleton and create the i dynamodb context now this is implemented by the class dynamodb context from the same nuget package so let's register that if we navigate to the dynamodb context class using f12 you can see that it has a constructor which takes in i dynamodb which is a dynamodb client instance now this is not yet registered so let's go back and register that so coming back to startup let's start creating a dynamodb client to register the dynamodb client let's create a new property so let's say var client is equal to new amazon dynamodb client which is the class that implements that client now this has lots of different constructor overloads i'll use the one that takes in the credentials and the client config so let's pass in credentials which we will just create now and also the config now let's create these two variables so let's go up again and specify credentials now for demo purposes i'll hard code the credentials inside the code but in a real application you would want to keep this out of your code so you can use it in a configuration file and also specify a secrets manager that can be used to keep secrets out of source control if you're new to it check out the video linked here also if this application is deployed inside the aws infrastructure itself you can configure the iam to so that it has access appropriately i will be covering those in a different video but for now to keep things simple let's hard code the credentials right here so let's create a new basic aws credentials which takes in an access id and a secret i'll create those in a moment so let's leave that with blank strings let's also specify the config which is going to be a new amazon dynamodb config which takes in lots of different values in this particular case i just want to specify the region endpoint and in my case i use the southeast 2 which was the sydney region so i can specify the region endpoint and say ap southeast 2. now depending on the region of your application you can choose whichever region that you created it in note these values can also be in the configuration so that it can be different for different application instances let's make sure to end the semicolon and now this code compiles correctly now that we have the client created let's register it into the services collection so let's say add singleton again and specify the interface which is i amazon dynamodb and pass in this created instance so we can specify client to specify the basic aws credentials let's navigate to the aws console go to iam dashboard you can search that from here navigate to it under users you can create a new user i have one already created which is called dynamodb but you can do a new one from here now under this you can specify the required permissions for this user in this case i have granted it amazon dynamodb full access you can restrict the permissions based on your application's requirement once that's set you can also go to the security credentials to create a new access key so this will give you the access key id and also a secret associated with it so if you want to create a new one you can click create access key now that's going to create one automatically so let's copy and paste this access key that i have created into our code so let's specify the key id and also the secret value now this is now able to talk to the amazon dynamodb instance with all that set up let's run this application to make sure this is working as expected so let's make sure to put a breakpoint inside the weather forecast controller it launches the swagger ui as before so let's navigate to the weather forecast click try it out and select execute this by default has brisbane city name so let's select that now this brings in the city name as brisbane and this is making a call to the dynamodb context let's resume the execution using f5 and this returns back the data from the dynamodb database so you can see all the details that we had added which is three items for this particular city so the same three items exist inside the dynamodb console as well just like that we added now we have successfully retrieved the data from a dynamodb database from this api now currently this call is returning all the values under the same partition now if you have the weather data for a long time then this method is going to return all of that to filter the values let's use an overloaded method of the query async so let's pass in a query operator which takes in different values like between equal greater than etc so let's specify the between operator and pass in a list of values now this requires a i enumerable of values now based on the operator that you have selected you might have to pass in one or more now since between needs a from and a to date let's specify two values so let's create a new object array and pass in the date so let's specify datetime.utc now let's just specify the date and also the next value as datetime.utcnow.date and add two days to it so let's specify the add days and specify two now this is going to get the values between these two ranges so let's run this application again and make sure this is working as expected let's click the execute button and it makes a successful call and returns back the data as expected now there is only one value for that rate range because of which it only returns one value now based on your application needs you can use different operators inside this query and specify the attributes appropriately now that we have the get method working fine let's start adding items to the dynamodb so let's create a post method specify public sync and specify just task because we don't want to return any values and give this a name post let's also take in the city name as before so here let's create some dummy data so let's say data is equal to generate dummy weather forecast which we extracted before and pass in the city name from there now this is going to create us a list of data based on this dummy data generation logic so we can iterate through each of these items inside here so let's specify the data property that we just created and use this as item to save these items into the dynamodb we can use the dynamodb context and call the save sync method since the type is automatically reducible we can simply pass in the item to this method let's also make sure to add in the await call so here we are generating some dummy data and saving this one by one to dynamodb so let's put a breakpoint here and run this application to make sure this is working as expected the swagger ui is shown it shows the new method the one that we just added so let's click try it out and specify a new city name so let's specify sydney now clicking execute it's going to hit the breakpoint it's generating this data for us automatically and looping through them and calling the await dynamodb context let's resume the execution and navigate back to the dynamodb console if we refresh this we can see all the values that we just added in here now this has five values because that was generating five values one thing to note here is the date time values even though i specified my current local time the aws sdk has automatically converted it into utc date time now this is happening by default because of the sdk converter that is available inside here which is the daytime converter one so if you see when it tries to convert it it automatically converts it into universal time so if you want to overwrite this behavior you can add in a custom converter for this type which i might cover in a later video so if we come back to the dynamodb console we can see these values have the time component as well associated that's because i had used just the date time dot now to create the values so if you don't want that to happen you can remove it by specifying the date property as well now but this is still going to convert it in the utc time so let's run this again let's choose a different city for now so let's give in london and say execute so this is again going to generate the data but this time without the time component so if we switch back to the dynamodb console and refresh it we can see all the values for london that's created however note it has automatically converted into utc time which is 1400 because the time component was 0 0 it automatically assumed it was local date time and it converted it so be aware of this behavior and use custom converters if you want to now if i run the same post call again it's going to create similar data but with different values so if i run this again let's say it's going to again call this it's going to call the generate dummy data but it's going to be for the same date range now if i run that and come back to dynamodb console these values are going to change so if you keep a note for london this was 54. if i refresh now this has become 42 and 35. now this is because the save item is automatically updating or inserting the item inside here if it finds an exact match with the partition key and the sort key it updates that item if it does not exist then it creates a new item now if you want to explicitly update an item let's say our specific item we can use the load method so specifying the await on dynamodb context use the load async method and we can load a weather forecast item i can specify the partition key which is the city let's also specify the date time for which this needs to be loaded so let's specify datetime dot now dot just the date part and let's add one day so this is going to take tomorrow's date which is the fifth of april now once we have the specific item that we have loaded we can update the properties independently so let's say summary and give it a value of test once updated we'll need to make sure to save this item back so let's call the dynamodb context and the save async and pass in the specific item again so let's run this and make sure this is working now note i have commented this code so that this does not automatically create the data anymore navigating back to swagger let's click the execute so it calls for london it picks up the specific item so it uses the datetime.add and it takes the next day's item so we have the value populated back up and we have the summary getting overridden now let's save this back and navigate back to the console to see if that's working as expected if we refresh this we can see the value test being updated for tomorrow's weather summary similarly to delete an item let's add in an http delete endpoint so let's specify public async and given task and say delete now this is also going to take the city name so let's specify the city we can either load a specific item and delete it or also delete it by the range key to delete we can pass in an item explicitly as before so let's add in the specific item and then call the dynamodb context dot delete async and pass that specific item to this method so let's make sure to await as well alternatively you can also pass just the city name and the date if you have that with you so let's run this to make sure this is working as expected we have the new delete endpoint so let's call this for the city london and see if that is getting deleted so this returns a success so let's go back to the console and this zero four zero five is going to go away so let's do the refresh and the one with the test is now no longer there so we have successfully deleted it now if you have the explicit partition key and the range key you can also use that to delete it directly without having to load the item initially the dynamodb sdk provides three levels of apis so there is a low level interface a document interface and an high level interface now these are part of different name spaces inside the same nuget package what we used is part of the high level interface this falls inside the amazondynamodb.data namespace so if we navigate back to the code we can see under the usings this is using the amazondynamodb.data model this also uses the document model just for this query operator so if we remove this we can see this is the only one that throws an error everything else works as expected now some functionality is not available as part of the high level interface like if you want to create a table dynamically you cannot do that using the high level interface i'll walk you through that in a separate video where we set up testing for this application i hope this helps you to understand and get started with dynamodb and how to interact this from a dotnet application to keep things simple i manually created the table using the console in an upcoming video i will show you how to automate this so that you can create these table dynamically onto any environment that you're deploying this application too we will also see how to set up local environment using docker and run dynamodb inside that if you like this video please make sure to hit the like button if you want to be notified of future such videos please make sure to hit subscribe it really helps me to grow this youtube channel thank you and see you soon

Channel: Rahul Nath

Views: 3,842

Keywords: aws dynamodb, aws dynamodb .net, .net dynamodb, .net core dynamodb, .net core dynamodb tutorial, asp.net core dynamodb, .net and aws dynamodb, dynamodbcontext, dynamodb .net core

Id: BbUmLRaxZG8

Length: 29min 2sec (1742 seconds)

Published: Thu Apr 08 2021