How I Built a Full Stack App on AWS - Building GuardianForge pt 1 Planning & Discovery

hey youtube it's been a minute but i'm back with some more developer content for you so if you've been to my channel before you know i usually do dev related tutorials but this video is going to be completely different so over the last few months i took a break from creating content and i focused most of my efforts into a project i created called guardian forge guardian forge is a third party companion app for players of the game destiny 2 which is one of my favorite video games of all time i play way too much if you ask anybody who knows me anyway this video is really going to focus on my discovery and planning process behind how i built guardian forge and how i took it from just an idea that was in my head all the way into something that's hosted and used by hundreds of users every single day so before we get started one thing i want to let you know is there's going to be very little code in this video this is all conceptual we're going to be talking high level but at the end of it you should have a really good understanding of how to put together a full stack application using aws which is what i used for guardian forge additionally if you're looking for help with developer projects come join my discord fullstack.chat it's the link to get into that um or you can reach out to me on twitter i'm at brian mmdev and uh yeah that's that's pretty much all i got so we'll jump right into it now he said before we even get into talking about the infrastructure and how i put this thing together i at least want to give a brief demo of the app uh this is guardian forge accessible at guardianforge.net the main gist of it is you can go up to this find players tab type in any user of the destiny 2 game and then here's a list of all the characters they have in there i'm a big fan of using the titan character so this is a list of all the equipment that my titan uses at any given point in time including any of the perks on all my weapons the different mods which affect the way that the different weapons and armor all kind of synergize together and you can create a build you can add some notes in here some notes and then click save build when you do the build will show up on this homepage under these latest builds here where you can select any one of these guys here and it's going to show you all the details of that user's character at that specific point in time where this becomes very beneficial is for a different create for clans for different creators out there anyone who's looking to share specific loadouts because they really do affect the way the game plays inside of destiny there's all these different stats here that affect the way that your character plays and how different abilities regenerate different weapons also behave differently depending on the perks that they have armor itself has different perks which can make your character for instance heal faster or regenerate their melee attack faster and then you can add additional mods on top of that to further make your guardians stronger in certain activities or parts of the game so depending on how your loadout is inside of destiny it really does affect how hard or easy the game plays and different loadouts can be used for different activities within the game so for example what you're seeing on screen is a video of a content creator i follow called castle he's demonstrating how you can use a very specific build for a titan in order to optimize playing against other players in the pvp mode called crucible so i often review videos like this in order to find out what the uh optimal equipment to use for specific activities especially the higher end activities are or certain activities which you go into with uh different teammates they they can affect the way that the team plays together um and one day i got the idea while i'm watching one of these videos it's like it would just be nice to have um a link inside of the description of a video like outlining the entire uh build that this video this content creator is using because um if you ever watch some of these which you can check the link in the description below i left that link to castle's video they can get really complex and when i'm watching these videos i'm usually doing other things so it's really hard for me to keep up with exactly what the the creator's talking about um and my memory is pretty terrible so that's another reason what kind of sparked this idea in my head so before i even start writing code one of the things i have to do is i have to make sure that my idea is even possible to pull off so there's a number of different destiny third-party companion app so i know they have an api or something to work with or some way that these third-party apps are receiving data from the game itself so that's the first place i started i started searching for the destiny api and uh let's take a look at what i found okay searching for the destiny api kind of landed me on this page you can see there's a number of different api calls on the left-hand side these are all organized by um you know kind of the general topic in which one of these api endpoints are used for um so this is kind of where i started just poking around inside of here seeing what we're looking for i know one of the one of the things that i discovered first was the get character uh which seems to indicate that it's going to get uh the character in returns information for the supplied character so you can see there's a number of different parameters that are required in order to even get the character we need the membership type the destiny membership id and the character id and from that a lot of it was just me uh sifting through these api docs to try and piece together exactly what a lot of the stuff meant so to be entirely honest a lot of this process really just kind of poking around the api and testing out some endpoints to find out what works and what doesn't uh we could see that there is a set of user calls here so if i see get bungee user net user by id you can see we'll get this response of general user and we have the membership id the display name profile picture and a bunch of other information here that more or less kind of corresponds to what exactly we're looking for to get out of the character you can see here one of the things that is required is this this x api key header which is a unique identifier that identifies your application inside of destiny's ecosystem so uh it says here when you're registered you have to go to uh when you register an application at this specific url you receive an api key so that's really kind of the first spot where we had to start here so inside of bungie's portal here you can see i have this option to create a new app once you create the application i'm not going to create one now but you name it you provide a website and then it provides one of these api keys which is really just a randomly generated uh string of characters and then we can use that api key in order to make calls into destiny's api okay so in my project here itself i'm using an extension called the vs code rest api client which i also have a another video on my channel i'll go ahead and throw a link to that up in the upper right hand corner if you are a developer and interested in that but this is really kind of where i start exploring different apis so um you can see here's a here's an example of a request to get a user by the the bungie net user id uh so if i send this request and provide it with a bunch of information here and this is kind of again my starting point to try and like sift through everything and figure out how it all connects together wires up and then just works in general um this tool is very useful for for trying to to discover all this information because you can quickly using text just pass information in here search through and then execute the the api uh right within vs code which is pretty awesome okay so after i gathered all the information i need uh you'll see here's an example of that get characters call we got the profile there's my profile id character and one of my character ids if i hit send request you can say there's an invalid parameters because we're missing the component's query string this is one of the things that confused me pretty early on back inside of the documentation you can see there is a query string parameter named components um but there's no further information here beyond uh see destiny component type to figure out uh what's being pulled down so if i copy that and search for that you can see on the left hand side under enums there's this list of different component types you can you can return back so i went ahead and put a list of components inside of my call here and i went ahead and executed it and all of a sudden you can see we have all this information available to us so i could totally see that there was an item hash and instance id but a lot of this i wasn't really sure what to do with so i started doing a little more digging around inside of this portal and i came across a call called get destiny entity definition and you can see inside of here we can pass an entity type and a hash identifier so we're getting close but i didn't see anything about an instance id in here to recall one of the things i said is that there were some third-party apps so i decided to go ahead and start inspecting some of these third-party apps to try and figure out how they were doing other information so refreshing this page you can see there's a number of different uh calls that are made into into the destiny api and so it's specifically here here's one that has components and all the information we just recently saw inside of vs code however i also noticed there was this call called the manifest and i've heard about the manifest before but i wasn't entirely certain what exactly it was or meant uh if we go inside here uh take a look just for example json world content pass which is a lot of what i use you can see there is a url here for english here right so i went ahead and copied this and since this is just a forward slash comment we know that where this is going to be is https uh www.bungie.net slash whatever that url is so let's go ahead and open up a new tab and take a look at what that looks like okay so what you're seeing on screen is the result of one of these component one of these manifests being brought in so this doesn't look the greatest because uh my browser is having a really hard time trying to render this just because there's so much information in here but just as a quick glance we can see this there's this gambler's dodge which i know is an ability inside of destiny there's also hashes here which seem to correspond with this this hash information that we've seen inside of our calls so i knew that the man the manifest was something i had to learn how to work with this is a part of the video that i'm not going to show what i actually discovered full blown manifest for destiny is about 138 megabytes which is huge and makes total sense as to why my browser couldn't render it was just too much data to officially be rendered inside of a browser so i started doing some digging for tools to try and start playing around with some of this json data and i settled upon a tool called dabrite which i'm going to demo for you right now all right so what you're seeing right now on screen is that tool i was just talking about called dead right json viewer recently become a huge fan of it i've never heard of it before this project because i never really had to work with the json data set quite as large until i built this project so just picking out one of those hashes that we saw in the the previous api call if i paste one of those in you can see that it actually searches pretty quickly through the different nodes here to try and find some information so we can see like there's a source here if we click to spray display properties you can see liar's handshake which i know is a piece of armor inside of destiny you can see we have the icon here and one of the other things that was nice is even though we found uh this specific hash inside on the bottom here you can see the full path inside of which uh node component it was where the hash was and then the different uh the different pieces of information you have highlighted it changes to kind of help you identify that path which is really helpful for building the application so at this point of the process i pretty much was confident that i could build this application and really the next part of it was just planning out how i wanted to structure things um i knew i wanted to use aws and i wanted to try and pick one new service i hadn't used so far because usually when i create new projects and do these kinds of things i like to pick something to push my boundaries so let's go ahead and take a look at the infrastructure that i designed and how things work um keep in mind that this is not how things work today but this is just the original planning process behind guardian forge so whenever i approach a new application i generally like to try and split it up into three separate categories you have the front end which is the client that the user is actually going to interact with you have the back end which is what the the front end is going to interact with that's where your api is stored and uh some people lump this last thing into the back end but i like to think of it a little differently because realistically it is a different layer and that's the storage layer so where will the data be stored i also like to break out things that i want the user to do so for instance i want him to be able to search for destiny users and any and select any of their characters create a build and add notes and other meta information as i demonstrated earlier in this video and then be able to view and share builds which is ultimately the core of what guardian forge is designed to do so uh let's take a look at some of the pieces of infrastructure i chose and why i made this decision so of the three big front-end frameworks view is always my preferred choice just because it seems to resonate with me the most i don't have anything against the others but it's just my personal preference i've hosted view apps inside of aws before in fact i've actually created video tutorials on how to do that i'll go ahead and throw a link in the upper corner of the video as well so go ahead and take a look there for the view application for that front end piece uh we're gonna we're gonna use two aws services the first is s3 which is used for storing unstructured data uh it's kind of like a file system but but a little more advanced than what you would expect to find on your computer this is where ultimately our built view application is going to end up and we'll be able to access the raw files for that view app um and then i'm going to put an instance of cloud front in front of it and what that cloudfront does is it is a content distribution network it essentially replicates your front-end files all over the world so this way wherever users are accessing your application it's going to optimize the speed because it's geogra the files are geographically closer to them than they would be if they were stored in a single data center in one spot of the world cloudfront also has the capabilities to create some pretty advanced routing rules so you can um you can redirect traffic depending on the url and while i didn't do anything with that at this particular point in the project i did eventually use the routing rules for open graph information which i'll i'll describe in a later video okay so for the back end there were three main services i chose um i i really kind of started off with just using one which was an application called light sail and this is what i had never used before i've used all the other services i described in here i've used before but lightsail was pretty new to me uh realistically what lightsail is is just a it's a way for you to easily spin up linux vms or windows vms uh virtual machines inside of aws you kind of get your own dedicated server very similar to like a uh like a a droplet by digitalocean so i used lightsail to create a linux vm that i wanted to host my apple my api that i just that i wrote in go because go was one of those other things i wanted i wasn't super familiar with i wanted to create an actual functioning api and go so the next service i decided to use for the back end or the api is actually api gateway um i could have called lightsail directly but i decided to to put an api gateway instance in front of it because if i ever wanted to change my infrastructure which spoiler alert i did end up doing at some point in this process i could simply add some different redirection rules for the api and start forwarding that traffic to a different uh different destination api gateway accidents is like an entry point for apis into the various aws services now for storage there's actually two services here the first one is dynamodb it's a nosql solution aws i use guardingport as an opportunity to practice modeling a single table schema design for dynamodb and the main idea really is to use dynamodb as more of an index instead of a full database um it seems to operate the best that way if you're not storing huge amounts of data inside of dynamo but small snippets that are required for various points of your application and then the last thing i used for the storage was again s3 so instead of storing the actual application files from the front end we're using s3 here to store the core build definitions so just as a quick demo inside of guardian forge the difference of this if i look at these different entries inside of the latest builds these all come from dynamo which are just these little snippets so it doesn't contain all the information however and clicking on one of these when the apple when the page loads all the data you see inside of here is actually pulling back a raw json file directly from s3 bypassing uh my having to store all this information inside of dynamo and really what that translates to is uh cost savings because it is cheaper store data inside of s3 than it is inside of dynamo provided it's not accessed too frequently if we take the various components of aws that we're using for this application and then we restructure them uh to take a look at exactly how everything talks and communicates to each other this is is pretty much what it looks like uh so on the top top lane you've got the front end that's being called uh the client will actually talk to cloudfront first if the website file that it's looking for is not inside of the cache what it'll do is it'll it'll request that file from s3 so once that file has been loaded up in a cloud front it never has to really request it again because it'll be distributed to that endpoint whenever the application requests any kind of data from the system it'll it'll first go through api gateway which then forwards that traffic into light sail or rather my virtual machine that's hosted inside of light sale and then depending on what kind of data it's requesting or what kind of operations being done it will it'll contact and communicate with dynamodb to either pull records from it or store records from store records to it and then the last piece of this is those build pages themselves if when the build page gets loaded instead of accessing api gateway it'll actually go directly to s3 to pull that information in so the last thing i really want to talk about for this video is actually how i store my code and how i kind of manage my work process and then my deployments um the obvious platform to use is github however i'm a real big fan of azure devops and um aside from the name it actually has very little to do with azure it used to be called team foundation online or visual studio visual studio online i can't specifically remember but at some point along the lines microsoft decided they needed to throw the name azure in it just to make it sound cooler i guess i don't know uh anyway so let's take a look at how i'm using that okay so we're inside of azure devops and you can see on the left-hand side i have a number of modules that i have here boards is one of my favorite places to work if i go inside you can see i can i can store all of my different tasks and things that i want to do and then i can also create sprints out of those tasks if you're familiar with agile you can then create tasks on top of the product backlog items and then manage them kind of through the whole workflow process additionally the code files itself the code files themselves can be stored within uh azure devops as well if i go into commits and this is one of the really neat things you can see we have tags that relate to the different releases uh if you go inside of pull requests which i have not used in quite some time depending on the the commit message you put actually links the uh the pull requests and the commits all that stuff up directly with the uh with the product backlog items so you can kind of have some visibility and traceability on what work what pull requests what commit how they all link together which is really handy and then the last bit that i really use in here inside is the pipelines which this lets me define custom release processes that um i can use to control the releases of uh the the new builds of guardian forward so instead of having to manually build this every single time you see i can come in here and i have different environments that are set up and inside of those environments i have a bunch of tasks that are set up in order to to build and publish the different components of the overall uh application so that was kind of my discovery and planning process for building guardian forge mind you this was done quite a while ago uh so a lot of things have changed in the application since this was first done but um we're gonna go ahead and cover a lot of the thought of those things in some future video releases um i hope this was helpful i hope someone got something out of it if you like uh videos like this go ahead and leave me a comment below telling me you want to see more content like this or just tell me six tutorials that's fine too um if you're interested in touch with me feel free to reach out to me on twitter brianmdev or on my discord fullstack.chat and uh yeah i'll see you later

Channel: Brian Morrison

Views: 76

Rating: 5 out of 5

Keywords: aws, cloud, developer, destiny 2, destiny the game, api planning, api discovery, api exploration, vuejs, golang, go, infrastructure planning, cloud infrastructure, api gateway, aws lightsail, aws s3, aws dynamodb, dynamodb, application planning, web app planning, web app infrastructure, aws infrastructure, aws cloud, cloud services

Id: Qva-s4UwOWg

Length: 18min 10sec (1090 seconds)

Published: Tue Sep 14 2021