Advanced Data Access Patterns with GraphQL, Amazon DynamoDB, and AWS Amplify

okay so hello everyone and welcome my name is Nader dabit I'm going to be talking about data modeling with amplify graph QL and dynamodb essentially we're going to be looking at how to create and work with different data access patterns on a DynamoDB database using graph QL in the amplifi library so this is going to be broken up into four parts we're going to be doing an introduction to amplify really quickly we're gonna then be looking at the graph QL transform library and the directives that are available there we're gonna then look at how to use that library to model different data access patterns and then we're going to jump into a demo so amplify is made up of three main pieces we have the CLI that creates updates and deletes the cloud services via the command line we have client SDKs that then allow you to interact with those cloud services from your web or mobile or native project and then we also offer a hosting service that is there for you to be able to deploy your web applications the CLI workflow looks something like this you initialize a project you add features and then you push deployments and then you make updates and then you run additional pushes to push those updates to your back-end resources the CLI also does a few other things though and one of the things we're gonna be focusing on is this graph you'll transform library but we also do stuff like graph QL code generation lambda code generation for popular use cases of lambda functions we do local mocking and testing for not only graph QL api's but also of cerberus functions and then you can also create and switch between local environments serverless environments locally so the graph tool transform library is we're going to be diving deep into this it's essentially a bunch of different directives that you can add directly to your graph QL schema that add additional features using the command line and running essentially a built at the deployment timer at the mocking time so a few the things that you can do you can model relationships between different types so we're talking about one-to-many many-to-one and many to many relationships you can add authorization rules you can add single or multiple authorization rules so you can do things like at profit and public access you can do stuff like set up group access and things like that you can create dynamodb data sources directly on your schema we also have this at model directive that does a bunch of stuff like adding additional graph QL schema around a base type for operations like mutations queries and subscriptions and stuff like that you can also model different data access patterns against IMM ODB and under the hood we're kind of using GSIS and sort Keys global secondary indexes and that's kind of what we're going to be diving into a lot today you can also set up a lambda function or HTTP resolver but those are two separate types of resolvers that kind of allow you to use app sync as a graph QL API gateway and then you get poured in other micro services we also have a supports for elastic search but the four directives we're going to be focusing on today are going to be these four on the left so at model at all at connection and at key so we're going to walk through each of those really quickly so ad model that's kind of the core directive that does quite a bit so using ad model you can say I want to have a to-do app but all I want to do is provide it to do type we're going to use that model to make a few different assumptions to kind of get you started first of all we're going to create all of the different operations that you're going to need to create read update delete - duze so we're going to create that schema for you we're also going to create subscription definitions for own create own update and on delete so we'll have all of that schema created as well as well as input types we're gonna then create a dynamo DB table for the to do so we're gonna create like a to do table and then finally we need to map those operations to the dynamo DD table so we're going to create all of the resolvers that map between all of those operations to your table so just running this one model running the command that CLI command we can deploy all of this infrastructure or we can test it out locally and then you can take those resolvers that we create and add your own business logic to kind of define additional functionality that you want or you can create custom resolvers for additional operations that you like to do we also have the app connection and this is what allows you to define relationships between types so a common example would be a one-to-many relationship so for instance we have a user that has many to Do's and it to do that belongs to a user to set this up all you would need to do is add the @ connection to the array that you would like to create the relationship between and then we would automatically create a new field for to do that would be something like to do user ID and then when you create it to do you can assign that to due to a user and then when you run a query on the users you can then fetch that child data which will be that - duze array for the user so there's a bunch of stuff you can do with that connection we're going to dive a little deeper into that later at allthe is what allows you to define these authorization rules so say for instance we wanted to have a to do that is only accessible by the user that created it and only that user can not only read it but update delete it and modify it to do that we would just add a new role for owner and this would assign some owner user ID under the hood to the table and then when we run a query we would only query for the user users to do's and they would only have access to those two dews but something more common is actually kind of a combination of two different roles and we're going to look at do that so something like a blog where you have a post type and the post type has both public and private access so anyone can read it but only the owner that created or updated it can only the owner that created it can update it so we're gonna look at that in just a moment as well but the main thing we're going to be really diving into today is actually around the at key directive and that key is something that allows you to utilize DynamoDB s really powerful global secondary index patent under the hood and provide a bunch of different data access patterns on all of the different types that are made up of your graph QL schema are made up in your draft cool schema so we're gonna walk through this in pretty in pretty in-depth manner and we're gonna kind of also then show how to model a bunch of different data access patterns against a product type but let's start off by looking at this kind of at key directive and go through each field so the key directive takes three fields or three different main fields it takes a name it takes a fields array and it takes a query field so the name is just going to be the index name that dynamodb is going to create under the hood now this is not too important in this example here because we're not going to ever be referencing that index name but you're gonna see later that if you would like to model relationships between two types you can actually reference this index name and a parent to kind of define the index that you would like to query on but not worrying about that right now you would not really need to worry about this this is just kind of giving a unique name that's going to be built under the hood they're really interesting are the really important part here is actually going to be the fields array in the first item that you pass into this fields array this is gonna allow us to add a new partition key which is essentially a new column that we can query about for this type so by default when we use app model on a type of to do we're gonna get to query access patterns we're gonna be able to get it to do by ID and then we're gonna be able to list to do is that's just gonna scan the table and give us all the two dues but at that point we probably are going to be okay for certain things and in under the hood that list operation again is performing a scan and you can then run a filter on that scan but it's not a performing operation because what ends up happening is you run a scan with the filter what's actually happening under the hood is that we're scanning that entire table bringing back all of those items and then filtering those out after they've already come back what we would like to do is to perform more efficient operations for these types of queries is actually a dynamo DB query and we can set up really powerful queries using a combination of the app key directive with this partition key and then in a moment to sort key so what we're gonna do here is we're going to say we want to now query on the category property so we're now we're creating Association when we say we want to have an app key with a field of category as the first item in this array we're able to now query on this new field and then we can say we want to get all of the - dues of a category of this type or that type and then we want to define the actual graph QL operation that's going to be generated in our graph QL schema that's going to be this query field so using this we're gonna now have a new graph QL query so we're gonna have three queries now in this to do type we're gonna have a get item by ID we're gonna have a list items and they're gonna have introduced by category which we can then use something like this well we're going to say we want to look at this table and we want to be able to get the items that have a certain category and we want to perform that query so now we're gonna have this new index that's created or this new query operation that we can call against the category row so we can now call a query using something like this we can say we want to get to dues by category and as you see that's the same graph to a query name that we provided in our key directive we want to get to dues by category and we want the category to equal shopping this is going to perform a very efficient query operation and we're only going to select the items that we want to get from our table and we're not gonna have to then filter because all of those items are going to already be coming back to us directly in that operation so we're gonna say we only want the category of shopping and we're gonna get that selection set now this is pop this is powerful but what's actually more powerful is when we're able to use a combination of this partition key along with the sort key so let's take a look at that this is the same to do type and we've added one new field we've added the priority field we've also added two fields to that fields array in the at key so when we see that we have the @ key of name category index but we now have two fields we have the category field as the first item and then the priority field as the second item and as you see those actually still map to two different fields in our actual graph QL type but what we're doing here now is we're saying again like in the last example the first field that we're passing into this array that's going to be the partition key that's going to be the column that we want to query on but we're now having a second option of a a sort key and this sort key provides us with a bunch of different operations that we can now run against this this new column we're going to now be able to do stuff like equal I'm sorry equal to less than greater than between and begins with so we can do these comparison operators on the data that's coming back so let's take another look at this this table we have a partition key of category so we're now saying we're going to be able to now query to dues by category so we can now just say okay we want to get all of the to do is that equal shopping that that operation still works just like the last one that we looked at but we now have the option of passing in also some fields to this sort key so we can now say we want to get all of the to dues with a category of shopping and we want to specify the priority maybe of of equal to 3 or maybe between 0 and 4 or whatever or maybe we want to say we want to get all of the to dues with a priority greater than 1 so we can do all those operations now and all of this is done with this one key that we've just added so a query that we might run now would look something like this we want to get to dues that category of shopping and then we only want to get the priority between 1 and 3 this will give us just the first two items out of this database and again what we're doing is we're using these operations so we can use equal to less than or equal to less than greater than or equal to greater than between or begins with and depending on the type that you're working with begins with I believe only works with the string so it begins with is something you might use for something like searching or something like that so with that in mind let's take a look of a more robust example we have this product type here and we have a product that has a name a price and inventory a category and a created at what we want to do is we want to model 11 different data access patterns against this product type so some of those patterns look something like this we want to get a product by ID we want to list all of the products so those two are kind of given to us by default by app model now we want to add non additional access patterns we want to list products in an inventory range we want to list products in a price range we want to search products by name we want to list products by category we want to search by category we want to list products by category in order by name search products by category and name lists products by category within an inventory range and then list products by category within a price range so that's a lot to kind of you know take in but let's look at this you know comparison operators again and we're gonna look at this schema that we're going to work with so what we've done we've done in a bunch of keys and using the query field as you can kind of get a good idea of some of the operations that we're going to be working with we're gonna be doing we're gonna be naming these query fields the things that they sound like so products by category and inventory then if you look at the relationship between the actual fields the first field is the category and then the second is the inventory that the last one you know category and price the first field is the category so we're gonna be able to query about category but we have the option to also query about price so let's now go ahead and take a look at a demo so I'm gonna go ahead and open up my you know app that I have here we're gonna go ahead and look at this schema so this is the the main thing we're looking at so we have all of these different fields here we have the the product field we have the I'm sorry we have the product type we have the different queries that we have set up there and all that stuff so we're kind of ready to go this is already been deployed what we're going to do now is open this up in the amplifi console and then we're going to go to the actual app sync dashboard and test this out so I believe you had an API I'm gonna go to view an app sync and we're gonna go to the schema just take a quick look at this oh there we go I'm trying to zoom out a little bit try to find the product type so we have that product type and then if we go to our mutations we have create product update and delete product and if we go to our queries though we now have all of these different operations so we have get product and list product but we also have all of these different operations that we defined in our schema so products by inventory amount products by price so on and so forth so let's test these out so I'm going to go into the queries editor and we're gonna start using these queries so before we create any queries let's go ahead and create a couple of mutation so I'm gonna go ahead and create a product and we're going to return the ID the name the inventory the creative that the price was try to return everything I guess and then the base type so to create an item we're gonna go ahead and need to provide some of these fields so for the name I'll call these Easy's for the price I'll say 1000 for the inventory we'll give this maybe 100 and then for the base type we we're gonna provide a base type of product now we're gonna be using this idea of a base type across all of the items in our database and this is kind of a really common thing that you see done in DynamoDB this gives you the option to have this base type as a new partition key that you can add additional sort keys on but it also allows you to kind of have some way to query by some base type now this is especially true in single table design but with appsync we're using multi table design but it still rings true because we still can use this base side as a partition key so we have the name the price the inventory the category we don't have the category so let's got the category so the category is gonna be something like shoes okay so we'll go ahead and create that we'll also create another couple of pair of shoes I don't know what the bronze shoes are called but something like that probably and then we'll also create a couple of like shirts so create like a polo shirt and this is going to be in the category of shirts and then I'll create a t-shirt all right so we've created a few different products so if we go ahead and do a query of list products we have a bunch of different fields we'll go ahead and try to return all of these and then maybe the base type so we list products we see that we have all of our products coming back so now let's start doing some selection sets only on these products by using these new queries so I guess I'll go ahead and copy this and let's go ahead and look at the different queries that will try to use let's shuttle first look at maybe getting products by inventory amount I'll call this a you know a custom query because that way we can reuse this name and then we'll call this products by inventory amount and here we're gonna pass in the base type of product and then the inventory and then we can set the operator there and then from here we're going to say we want to get the items so for the inventory we might say something like less than 500 and now we see that we have the inventory of 100 and the inventory of 200 in the inventory of 400 we can also just say we want to get the base type of products which is it really interesting because that's going to pretty much bring back everything but let's say we wanted to let's say we want to kind of go between let's say we want to say between 100 and 400 we can also do that we see that we have inventory of 100 inventory of 200 and matauri of 400 we get to say maybe 200 and let's say maybe like 500 and 1,000 we also get that so yeah there we are we're doing we're doing some more detailed queries now so let's now do products by price and now for the second argument we're going to be asking for the price and we can say equals what's actually it's going to be hard to do equals let's say greater than 500 we're going to only get like these expensive items or something like that and then we can bump this down to 3 300 we see that we have the price of 500 the price of a thousand we might say between 100 and 200 we only have one item there so we're getting now that that selection set of products by price let's do product search by name that's pretty interesting we're going to now say for the name we can now say stuff like begins with and now we can say Y we can say begins with J so you get Jordans back we're gonna say begins with teeth let's see what else we have so we have black t-shirt Jordans LeBron polo so these are all different letters so yeah so that's kind of what's going on with the product search by name products hence about category is also pretty good because now we can say we want to only get the category of shoes now we're getting only shoes but we could also say we want to get categories that's that begin with s and that way we could maybe have like a search on our client that brings back things that start with s and therefore we're getting both shirts and shoes back and if we say SH o then we're going to now only get shoes back and if we do - Shi of course we should only get shirts back um let's go even further and let's do products by category in the inventory so we're gonna say we want the category to be ladies shoes and the inventory to be between 0 and 100 we want to kind of find the things that we don't have a lot of inventory along and then maybe we want to find the things that were like overstock so we might say greater than or equal to yeah let's do greater than or equal to a thousand that's to greater than or equal to 900 you don't have anything there let's go back to 500 you down to 100 so we have inventory of 100 we have inventory of 220 of 400 so we could say greater than equal to 400 just to get that one item and then if we said shirts we should have some inventory of 1000 to query against um so that's yeah that's kind of that's kind of the main idea though is that you're using these different GS is to do all this stuff so let's take a look at our database real quick and it's this product table here and if you look at the items you'll see that we have all the items that we've created and if we look at our indexes the here you'll see all these different GS is global secondary indexes products and they're given the same name that we kind of created so product category index products by category index and so on and so forth I think I kinda fit if you look at my schema you'll see that these are the names that I use here but I think in my slides I kind of made these smaller and I did something like something like that to kind of just make less text on the screen but this is what you end up with under the hood and you have the the name of the index which doesn't really matter but the partition key is going to map directly to the category or I'm sorry to the field that you have in your actual DB so we have the index of partition key category and then we should have the corresponding category row here we have the sort key of name create an inventory price all these different ones these are also going to map over to a row in our database so all those are going to be kind of like mapped there so now let's go back to the slots and continue on looking at some other stuff so what are some other common scenarios that you might run into with with a amplifier that you want to kind of take advantage of or understand I guess the one of the main ones is multiple authorization sites so we want to have a post type something like medium a blog or something like that or even WordPress where we give owner privileges to the person that creates the item but the only person I can read it is someone who is or anyone can read it actually but the only operation we're going to give to a public read or to a public operation is read so owners can create update delete anyone can read and that's kind of what we want so to do that we set up two roles on our post type we have the allow in a role and this is what gives the owner privilege and then we add another rule that we call public and we pass in the array of operations that we want public to be able to do and in this example we just want them to be able to read so that's a common scenario another one is is when we're when we're using identity we might want to specify the identity so by default we're going to set the when we say allow owner we're gonna set the owner field as owner but let's say we wanted to specify that odor field is something like user name we could do that by mapping the owner field straight to a field that's already set on your post type and then by default this needs to be set as nullable because when you create the item you're not passing in this user name this user name is being populated for you by the graph QL resolver and we're kind of going to be setting that in the mapping template when we're creating that item and then you can query query it back against against your date your API and this this owner field this user name this identity is being read off of the JWT in the resolver and we do that by reading this property called the context that identity that's all that's automatically person another thing that you might be interested in is combining at connection and at key and this allows for some really powerful stuff because then you're creating not only relationships between types but you're defining how those relationships work so let's take in a car that common basic example of one-to-many relationship so we have a post type with many comments and a comment belongs to a post what we're going to do without by default if you don't define any specifics is we're going to take this and we're going to assume that we want to have a new field and we're just going to come up with the field name so the field name is going to look something like this it's going to be the parent field with the child field with them the identifier so it's going to be something like post comments ID so we're gonna create a field on the comment call it post comments ID and then we're gonna put that on the comment so that way you create a post then you use that post ID to create a comment and then you use that that ID as the post comments ID and therefore you have the relationship but that might not be that might not make a lot of sense if you want to specify what this looks like so to do that we can actually create that field ourselves and to do that we would do something like this we would use the @ key directive but the only thing we would not do is we would not add in the query field because we're not gonna be querying this directly from graph QL this is going to be queried from the actual query operation for the parent so what we'll do is we'll create a keen a key name of something like post idea next this is going to create that GSI on the comment table we're gonna then in the comments array we're going to add the @ connection and we're gonna specify the key name that we want to use on that child field or on that relationship field so we're gonna say the type of post has many comments and for that connection we want to use the key name of post ID index we're gonna then select or specify the fields that we'd like this to be queried on so for the query of getting comments by post we want to use the post ID that is stored in that comment table so this is going to be mapped directly to the comment I'm sorry to the post ID in the comment and then the ID for the post is just going to be coming in from the post itself and we need to specify which field that is so we're just going to say well it's the post ID so it's the post ID and the parent and it's the post ID in the in the relationship and the child so a couple of pointers I would say use local mocking and testing when playing around with these different data access patterns and that way you can kind of make sure you get things right and I'm going to do a quick demo of local mocking when changing the index structures via the amplify CLI when you kind of if you if you change a GSI structure you're gonna get some failure when you try to push it up as of today and we're working on that because essentially what you need to do is remove the index completely and then add it back and by default the way the CLI works today is that it will just scale but what we're working on is we want to be able to detect that failure then we want to delete it and then add it back for you so just right now just letting you know if you want to change an index structure just remove it and then add it back and everything should be fine you can create as many GS is global secondary indexes or at keys as you would like to at one time so you can create like five or ten or whatever but when you start to update those you can only update one at a time you can also remove as many as you would like at a time I would I would look deeper if you really want to be become really good at this stuff I would maybe take a look at dynamodb and kind of look at how look at a talk or two on how to model data with GSIS and sort keys I would read the data access patterns section of the amplified documentation and then I would also say that I would use some consistent type I'm sorry with some consistent field across all items in your DB I said I use some consistent type here in my slide but I meant use some consistent field across all items in your DB just like in our product type we had that base type of product that allows you to have that base type as a partition key so local mocking let's kind of take a quick look at that in the amplify folder we have a couple of different things we have our back-end our cloud back current cloud back in and we have our team provider but what you're going to see is that we can actually do local mocking and in this amplify folder we're going to now store some data about your mocking environment so we're gonna say amplify mock and this is going to go ahead and allow us to mock our API locally and this is going to first introspect our graph QL schema make sure it's valid it's then gonna go ahead and create all of the databases and all the global secondary indexes and all that stuff that we need locally using the CLI so we're gonna see that we now have a post table a comment table and a products table we're going to then run graph QL code generation to generate the graphic UL code locally and then we're going to run a local mocking server at localhost so here we have this local graphical editor running against that those that deed those databases that we created and we can now actually run locally so we might be able to go ahead and go into let me go into amplify and just copy some of these operations that I've already done I can now just paste these here I can go ahead and create a product oops I need to kind of I think I need to comment some of these out for now so we created a product we can list products and we can also use this custom query so we're using all of those features but we're just doing it locally you can do all the operations so you can even do stuff around authorization so if you wanted to switch also authorization types and set a JWT and you can even add a group and I'll add someone to a group and generate the token and then the token will you pass them in the request so you can do all that stuff locally it's pretty cool stuff that's it so yeah thank you for checking this out I'm hoping to do more of these in debt in depth talks and we're going to be trying to add a lot of the stuff I just talked about into our documentation and a little bit clearer manner but if you have any questions check me out on Twitter at dab at 3:00 and feel free to ask me any questions there thank you

Channel: Nader Dabit

Views: 18,009

Rating: 4.9736409 out of 5

Keywords: graphql, serverless, aws-amplify, amazon dynamodb, aws appsync

Id: VG2XWoD-rS0

Length: 34min 53sec (2093 seconds)

Published: Fri May 15 2020