A Firestore Data Modeling Session with Jeff Delaney

we are alive everybody welcome my name is David East I work at firebase like firebase all things firebase and I'm joined today by one of my most liked knowledgeable firebase friends which are you on this you on this side right here oh no I know this up there you go the just joined by I'm not sure can you hear me yeah I can hear you okay no se you just just enjoy yourself cool yeah I'm Jeff I run the fire ship YouTube channel as that's where most people know me from and I think we're gonna do some some data modeling today so should be pretty fun yeah man we are I'm excited so just to kind of set the mindset for this Jeff and I have been talking about this idea for a while now and Jeff's idea where we want to make a JavaScript game that's a JavaScript jeopardy game where you have like a board full of like jeopardy tiles and then people can log into the app and then we sort of act like the Trebek and we select the questions and then you all would answer it and then we would award points so it's kind of like a mixture between like a quiz app in jeopardy but it would be like you know we'd run it as a live stream one day if we get this app going and everyone joins we just see who wins you have a leaderboard and all that stuff and so it's like a really fun thing to do and we're just gonna get it kicked off by talking about the data model so if you all have any questions about what we're doing just you know obviously leave it in the comments and the way we're gonna break this whole livestream up is that we're gonna be like either focusing totally on the code or the UI or the data structure and then when it's time to break and go to your questions we'll scroll up and we'll get to good to them so if we're not answering your questions right away it's because I'm really bad at multitasking so that's why sounds good to me yeah so thank you all for joining but so let's see here so job of Jeopardy uh Jeff tell me your full vision for it like I kind of gave it but like what yeah you you you're the one who has I'm kind of imagining a game board I think maybe we'll do like nine questions for the first round of it which will give us a nice 3x3 you're in for the UI and so what kind of create that Jeopardy grid and then when we select a question it'll pop up and activate that question and that everybody at that point can answer it in real time and will award more points to whoever answers it the fastest CVO points for answering correctly but you'll also get points for answering at the fastest and we'll kind of do a server-side timestamp to validate who submitted the quickest answer yeah this isn't terms like timestamps sounds like it's gonna be the hardest part yeah I should actually do well I don't think it'll be too bad but we'll find a way to make it work and yeah at that point we'll continue doing that for nine questions and then we'll aggregate all the scores in the background with cloud functions and whoever has the highest total score wins the game and we'll all throw in some fire ship swag and I don't know firebase can throw in some swag as well but some kind of prize for people for sure man I'll see what I can do I want swag person i oh I forgot that I do have this though I wanted to wear this for the stream purely purely vanity at this point so that ice this one right here so yeah that looks cool it's got one yellow part of a shirt so I need some more firebase stuff myself yeah man of course you do I I know I said I wasn't gonna read the comments I saw this one pop-up how to build a trivia app in 100 seconds by Matt Carson and let's just put this on 10x it's deal yeah after we get this done we should just chop everything up and do a 100 second hundred second trivia app and firebase I think that can work out pretty well oh we do have this I I saw this - what is jeopardy for non-americans ammonius that's a great question thank you for bringing that up so jeopardy is this like evening time show that comes on in America probably a lot of places in the world now but for I don't even know when it started many many many years ago and like 60s 70s I think and basically it's like a board of categories each one of us sending us our category questions with a sending dollar amounts 100 200 300 so on so forth and then you you say I'll take this category for this much and then they actually give you the answer and then you have to say what the question would be and so if it was like like if we did a JavaScript jeopardy one it would be like this operator or this reserved keyword in JavaScript does not have block scoping you would be like what is far they buy correct and then you would just move on so you should check it out it's like the one of the greatest shows of all time it's seemingly very boring but somehow like once you start watching it you just can't stop yeah it's been on forever - and Alex Trebek is because the host that shows kind of an iconic deities do you have that patriotic and write some kind of cancer right now yes yes yeah see which he has he's over one year out and he yes Alex Trebek is the man yeah if you haven't gotten into jeopardy this is like you should just go YouTube it you're gonna like this is gonna send you through a amazing rabbit hole all of YouTube content yeah I think they have it on Netflix now maybe Hulu yeah yeah I did see it on Netflix oh it's like oh man as soon as I start watching this I can't stop yeah anyway so we want to just kind of since Jeopardy's amazing we want to recreate that magic and we're gonna kind of go about mocking this up into two different ways right now at least is that I have hidden behind the scenes a sketch window open and we're just gonna start mocking up a UI and then we're going to start mentally modeling the the data based on that UI so i guess 'sir probably should flip into that so here's sketch for those who don't know who sketch what sketches just this nice you know like UI mocking app various it's kind of like Photoshop except really specific for apps and it's a I'm gonna make it you'll see things and so this sketchy of these artboards and I like to start with like a desktop HD even though a lot of times we're gonna come in on mobile I find that either like to start directly on mobile or directly on HD because that way it gives you like a good idea of constraints but this is like this might really be throw away yeah I just throw it out there I think we should maybe do this as a mobile first design to speak out okay I think a lot of people are going to be watching on one screen and then to answer the quickest you thought I'm gonna have your phone out like Tapani that's really quick so you're not switching between tabs that's true all right we'll go with mobile first and I like to put the big desktop next to it regardless of which one I go first on just to always remind myself of the other screen that I'm gonna be responsible for yeah yeah for sure though ideally I'd like to get a lot better at framer do you ever use framer X which is really cool in advance they just haven't taken the full time to do the dive so yeah yeah framer and web flow and tools like that are becoming very popular but I guess no code movement if you want to call it that a framer is like you get the code and the no code it's because you can write all the react components in it right but then also you don't have to it's kind of like this weird honest said weird this really cool in between no no not with we're gonna design with grey because that's easy and that way we won't be taken up by a color just got that one you're like I know it fonts even gonna pop up right now ooh what is that that is Monaco why is Monaco up Joe Monaco is like the last font I would want up nonetheless lot probably Bradley hander something like that um let's go you with my favorite fonts it's called upgrade mmm do you have Adobe fonts on here we're talkin to you I do that both you can never have too many fonts yeah that's for sure let's go way smaller right I'm like messing with fonts dirty channel alright well this is not great alright anyways so it's if you came here on a phone what I probably want to do I guess we would how would I'm trying to think about how we would have them see the board the board cuz I don't want like a micro board like something where you'd fit all the tiles onto here hmm I'm definitely thinking we use CSS grid for oh yeah designing the forty your because I envision is a simple three by three three by three grid on most devices but maybe on smaller devices we could break that down just to either to two by four or whatever um so we could make it responsive in that way the grid very easily well I mostly mean like how much am I going to show above the fold because what I want to do is ask actually I don't want to be an RGB I want to be in oh no not that I want to be in and I can almost see it being just like a hundred percent viewport width and height and it just sort of responds with different sized boxes they wouldn't be perfectly square but right I got tiles like that so I guess you're thinking I might as well just do it we would have something like this where you would have this would be like the category or already are we gonna do categories that's yeah yeah that's actually a good data modeling question because now at this point we're starting to think about the metadata that goes into a game exactly I think we would have a game document in fire store that might have like an array of categories problem in this case I think three categories will be good so if you have these categories this P category be like this keywords and then we would probably have keywords and then it would be like yeah to a big dollar amount we could you know we can weight the different questions based on like how challenging the are make them you know get incremental II more challenging after each one I mean you know sign of Haider sports the more challenging ones right I guess who would need to make these so big because I think there's like two modes to this screen whatever it's not perfectly centered yeah it's really off there's two modes I think that we would care about like there there's the board mode and then there's also like the what question is selected mode because you would have like all right you'd have because you have to kind of hide the questions and so we'd be like all right the next one is oh and then you'd even have the issue of I probably get into the weeds here of like what happens when the question has been answered or the answer has been questioned do you have like some grayed out state like that you know looks like I lost Jeff and I didn't know it all right so I'm reading the questions all right so I'm going to take this one over I'll look at my messages to see where Jeff went this is a first for me but so what we are gonna do now is we are going to take a look at this I'm like talking to myself and where's Jeff all right so I just saw this one Steve that's funny alright so put in your questions where's Jeff is a is a good question too but so what I'm gonna think about is is that I'm mostly confounded right now I'm like really torn about what I want to do with this you I like cuz the Jeopardy board is is big so like the problem I have is that you tend to have these these boards that are like these are all out of whack the the width that is I think it's but it's think it's six categories just go for though Jeopardy's has way more categories than this and then you would have these type of things right here where would be like oh I didn't do this right that's like I think it's gonna break alright so what we would have is that paragraph what is it doing what we would have so I'm trying to get this text lined up and it is just wants to fight me and I don't know why oh here it is okay that's why so instead of 30 to make this a queen bee no 26 no meat or yeah there we go I don't really like that but we'll just go with that for now I'm group that's easier to copy so we would have like these type of boards is your traditional jeopardy board and so you have these different categories so Mikey words the other one would be like it's a good JavaScript jeopardy scope I don't know put this in the middle that's a good I know right where is Jeff but uh what's dropping your good categories here what do you think is a good JavaScript Jeopardy category I'm gonna message to Jeff and be like where'd you go maybe he lost the Internet that would be me here so keywords scope it's another keyword scope functions you know functions and scope are pretty similar and then another one Aybar should be in all caps then key words would be you know no operators so these would all be like these categories and so you would tend to have these on the grid right here and this is like a perfect candidate for CSS grid so then you'd have 200 and let's make the opacity all the way up for that one and why are you behind and you just kind of have this for all of these so you'd have 200 across the board and so the question would be kind of hidden behind all these and then down the screen you would have it like that so we just kind of keep going on and so forth oh here it's Jeff he's back welcome back hey thanks yeah my internet pulling went out we've been having some weird internet issues to the waywe so yeah sorry about that everybody I think they waited for the livestream to start and then they were like and now right what I was doing is I've shown the difference between the board and from like a big view versus the board of uh of like the smaller view of like the challenge of displaying it right so yeah yeah yeah that's basically what I had in mind there's a bigger grid on desktop and then yeah I think breaking down and just a single row would be good on mobile but how do you know and I have a couple ideas but I'd be interested to get your concept here what would you see for like when we shortened it would you see like an overflow X Crowell so we have sort of like this whole like hidden area over here of like scroll content or would it be like each one of these categories becomes single file on like the one vertical scroll I would just do a vertical scroll I usually like there's some use cases but I think horizontal scroll on mobile it usually doesn't work very well yeah usually did you end up tapping the content yeah yeah yeah that's definitely an issue see I think you just scroll scroll all the way down or you just go super small with the individual items and make them a grid as well all right let's go with scroll a scroll vertical for now and see maybe what you could do - oh I have an idea oh no that doesn't work or actually knew this would work and another thing another thing we'll have to finish the UI somewhere maybe this could come out from a drawer or something like that on mobile is just the high scores and like the actual user management to be able to log in and log out mm-hmm yeah that would definitely come from there but you could have and really you could have all the categories no no I feel about that okay now you can make them like an accordion style I guess but yeah that's actually it's a really good idea so you could have it be like an accordion style and so they're all that's actually probably the best now that I think about it yeah that's a really good idea because then you tap on it and then that would show up and then that way they would be single you would kind of get the best of both worlds of having them single file and compact yeah yeah I think that could work okay I like that yeah cuz it's really quick and then we could even do some some magic on the firebase side of things were for fire store and we could sink the point of which like we could we could say we can make these ones interactable or we can say like alright we're gonna open up the functions one and then on everyone's screen it all opens up right yeah that'd be cool weld it like when the question actually opens with my vision is is like you have a modal window that pops up that like you can't actually close so like until you answer the question so that question pops up takes up the full screen and you can answer it and then what's the answer it's like a lockdown and goes away let's do that right now so I like that so let's do a little bit let's do a it's create a Matt what am i doing let's just create a mask so this I should have really grouped these but I didn't and so I'll have this mask and we'll turn up the black I have some borders on there I did not want yeah these should not have borders it looks weird you know like maybe we should get a super retro with it make it like a 1990s style Jeopardy game I was wondering about that why are these floated above oh I think it's cuz I need to do this that's why cuz yes there we go no I was doing it backwards in my head all right so we'd have like this mask that would roll up maybe and then you would have maybe almost like fullscreen you probably need a mask then actually it's pull this out then and it would be like this operator allows you to flatten or can cat can cat arrays and merge objects together and I'll change the and this actually brings up some interesting data data modeling questions as well because now we have to think about there's going to be one single active question in the database somewhere so you have to think about how to activate it and deactivate it and kind of manage the state of that question on the server right and then we need almost like yeah and we need something that's like it'll snow boarders yes we need something that's gonna be like the timer and so yeah they're gonna yeah the time will be one thing to think about too and like also this question will have multiple choices and then only one of those choices will be the correct answer and we'll have to keep the correct answer in a completely different document because we can't send it down to the client here because someone could technically figure out what that answer is that they know firebase well enough right so they'll say masquerade if they know firebase that well then I'm almost just like you in but I mean it wouldn't even be too hard you could go to the network tab and figure out what you know find the request for that document and you know then the time frame I think you can find the right answer and for every single one yeah no totally so I'm getting really carried away with this and so this would be like the timer that's like you know starting to time around it's a weird thing but so we had have like the timer and then probably in the middle there'd be like like maybe like oh there is 45 seconds left yeah be kind of cool to like count down the score like so you get a thousand points if you answer immediately and then that kind of ticks away over the course of 60 seconds I'm seeing kind of see your score diminishing as time goes by oh yeah that's a great kind of put some pressure on to get an answer in quickly yeah you could also say this operator loads unions together and then you would have like you kind of have like your leaderboard section down here I've like maybe people who have answered yeah somewhere I don't know if you'd be able to do it on this screen per se like there definitely should be a leaderboard that's always visible so the people winning the game kind of get recognition in the app itself right and we need to we need answers because we can't just speak it even though that we get it cool and this is a good day to modeling topic too because we have this question here so like if you think of this question as a single document you have the question text and then you probably have an array or a map at least three different multiple-choice answers ribbon is it yeah best yeah you're right so let's do it like you know like really six standard with my colors that I'm not going to AH so you'd have like a no no that's too much group this say yes so that way this would be would you see this actually here's a good question I guess we start talking about data modeling would you see this as one document or multiple documents I would say this is one document so I would have like individual questions that I would break the questions down into their own collection and then have a game sort of linked to the IDS of the questions that were part of that specific game you could kind of embed everything together but I know that's my initial thought is breaking questions down into their own individual documents because for each question we're going to have a response from every single user which will also be its own document so that'll be I would envision that being nested in a sub collection under each individual question so like there's a few ways you could do it but you could have a game document that has a sub collection of questions and then each question then has a sub collection of responses so you kind of have a hierarchy of three basically yeah I hired you three three levels a game a question in a response um game question all right so it's alright before I get too carried away here the you I guess that's all I want to do is let me let me just do another rectangle here and let's make this like the data structure so you imagine what would what's the so we'd have like well well one thing you can do you want to pull up firestore like if you have an empty project or something we could all right gonna sketch it out of that right that way I should get out of sketch by the way who wants to guess what I'm drinking I'd do that every stream now um so what is in this Cup Knights me share my other screen and kitchen window firebase Council and here's a fresh new project continued jeopardy Jas I'm very excited about that one all right so let's on what's initialized fire source since this is brand new Merle it's like the smell of a new car you project full of potential and start in test mode I like how it fires stored will now disable your database after 30 days if if you keep it a fast mode without updating your rules yeah I get lots of emails about that all the time since I create so many projects there like this client access to your database is expiring yeah yeah Marin ten times it hurts me because I'm like gods fine and then write my go out of the things stop working yeah let's hear while this is going up through some of this questions uh we got a winner Georgie Goffe it is just coffee you went you went with the obvious one Gary your commitment to matcha is admirable I don't have any but but you I like how you always guess it and Paul bender sorry not not milk no milk is good um let's see here I'm going back up reading some of these things when it comes to fire store DB modeling you basically want to model the DB upon the UI to optimize reads instead of actually making sense when you look at it oh well it depends on what making sense means but I'm spending maybe oh there we go I just needed to switch make the screen active so one of the questions was do you want to model the UI the DB after the UI and it depends on your system there's some systems word that doesn't matter I've created a lot of like data pipelines through fire store like where I like pull data from different api's and stuff like that and a lot of the data is actually stored more so in a normalized fashion but then I will like scheduled functions you know intervals throughout the day or just once a day that will take that normalize data and then merge it into like something that's more like a you know a logical view or something like that in the sequel or notes yeah I think the optimal thing you want is like it the optimum way to do it is if you can create a document that has all of your data for a specific screen or page in the application that's the ideal way for things to go but a lot of times people have a relational data that just doesn't quite work like that like you just have exactly queries and things you have to make right you want to you want to you want to optimize for the most simple reads and you can look at your hierarchy and firestore like it's a sequel statement so like if you have select star from users where UID is equal to one would be about the same as saying users slash one and then use it's the same statement there's just no query and then when there's no query there's no logic it's just like knowing exactly where it is in the database so it's just a it's the same concepts that you would have in the sequel world just structured differently okay let's pull this up so what's what's your what's its what collection do you want to start with and let's start with a game since that'll be at the top level collection and so you want this to you're being really fancy hero is anything about being this fancy gonna like it you want it you want to say like a game is like one session and so like if you did this daily you would so this is like multi-tenant in the sense of games it's not just yeah I'm sure I want to make this this app scale so yeah games its own collection that way we can play multiple games if we want to it would be a little more simple to just forget the games document and just like hard code all the David or front-end app so maybe that's a better choice in this case but I think oh you just white realistically if you're going an app you probably want a games collection no you totally do all right so we have games and so what's with games is that sort of like a a placeholder like what's the document level is it just gonna contain like metadata about the game like the name of the game yeah stuff like that but the categories would be one main thing to hire so I think that an array of just an array of string values should be fine it's not good so just be like functions operators potent potables Oh watch this jeopardy so one other thing we should definitely do here is aggregate high scores like the running high score for whoever's winning the game and I think putting that data here on the game document would make the most sense oh it'd be like you know the leaderboard here yeah just embed the leaderboard in here and that is an array yeah probably an array of maps so each map has the username and then the high score for that user you can't do can you do a mat okay you can do write map now for a longest time we didn't have that in there because the it was very complicated but I think they fixed that since then um okay so that would be like with the field be like a user ID and the score I would say I would do the users display name here but you probably want the user ID is well because basically with this you want to just be able to take this data and show it in the UI because you're what we're doing here is duplicating data instead of like making a query for it so we'll have a cloud function figure out the high scores in the background then we'll duplicate that data here so it's read very quickly and in real time for everybody I see you could do you can get really crazy here know if this is a smart idea ream me and reel me in if this isn't good but what if you had a map of maps and you would have UID and then you would have the the you ideas one of the keys and then you would have the display name as one and then the score as the other side would be like I don't report 7y ii work as well but is to map though it will be a little more difficult to sort the map yeah I can see some issues like just kind of wrangling that data on the back end when is basically after every every round we're going to have to recalculate the high scores and wants to do that in the background somewhere in it an array would be probably the simplest way to go but so the raise is good when you're when we'd have to do any sorting or aggregating the object is good when we want to be doing like when the key value has like importance and we're doing like lookup of the user by the UID right and I don't see any lookup boys off the top of my head so let's just keep it as a map right now but yeah sure fine actually yeah maybe in the future and want to make it an array well just keep this I don't think it's that big of a deal because you can always just like object that keys and so long as there's not 10,000 users in it your you'll be fine yeah yeah basically we'll just grab the leaderboard look at that map and I don't know if firestore keeps that like the map key is sorted I believe it would be likely like I don't know if this is real time database err fire story mixed up I believe I might be Lex psychographic lis some kind of alphabetic order know basically like your folder or order of like when you create new folders and like on your desktop right yeah that makes sense so it's not based on when they're inserted it you're gonna see yeah that would make more sense yeah I'm gonna saving this before I lose all that like accidentally refresh okay so we have categories and we have the leaderboard and then the name so that makes sense because that all can be led on one UI there could be some contention though with this document with updating the score how frequent do you see this leaderboard being updated so this will only be updated once every question so there should be a single cloud function that listens for when the questions it comes finalized okay and then at that point it pulls everything calculates the score and then updates the leaderboard okay so that makes sense yeah yeah so I don't see it being a high frequency write or anything like that so this is actually one of the most what we just both discussed here just for the audience is one of the most important things to think about when you're structuring your data really for any type of system but just thinking about for firestore sakes we're asking ourselves how often are the writes going to happen to this document so like how often we gonna update it and the reason why this kind of stuck out for me is because this score right here is something that is that going what is the frequency of this is going to be updated and so Jeff has a really nice system where he's saying at the end of every question we're just gonna have one update it's gonna be like it's that updates this whole thing all the users scores every round and that is a really nice way of doing it because it limits your rights just one and that keeps you from having like race conditions from trying to multiple people like slamming up against this dock and that can create problems at times you can obviously create lots of you can code against that but that's just more work another way of doing it if it was going to be updated frequently was B would be moving leaderboard probably to like a sub collection and then each one of these would be a document of the sub collection if that we were going to update everything so frequently right yeah but happy yeah because there's a limit in firestore for one one right per second but you can burst above that one that so you don't like you don't need to be sharding or anything crazy if you just need to burst above it temporarily right but if you have something that's super high frequency you like you know a Twitter post it's being likes like a thousand times a second right you need to try the database there's a firebase extension that can kind of handle that for you automatically exactly yeah the choose distributed counter you know yeah no I was perfectly said so just one of those since part of lesson here is thinking about data structuring is that a lot of people ask like when should I something be an array in a document and when should something be like a sub collection of that document and this is exactly well this is one of the ways of doing of the ways you decide that as how often are we gonna update this leaderboard and Jeff has the system where at the end of every question one cloud function will run and just do that management and so it's one update and so that's that's it sort of batches that into one write and that will make costs a lot cheaper and it will also make just the update way more efficient so yeah that's that that is why this isn't leaderboards so we have leaderboard name there's probably other metadata that we would eventually need to add for this but let's not worry about that now so with games then there would be a sub collection of questions now that's kind of interesting question because we could make questions a root collection or we could tightly couple them to games and just make them all those questions it's owned by an individual game and the advantage of a root collection I could see it would be if you have multiple games and you want to use the same question in a future game right um I don't think we're going to be doing that so I'm more inclined to think just putting this in a circle action for this particular for this particular app now my question for you is and this is just like a general question because I think either is fine why questions is a sub cool I know why because you already said this earlier you would have questions as a sub collection not as a field because of the security model right not just that but also figuring out like managing all the responses for an individual question because once you have a question then we have to think about how users respond to that question which i think is probably going to be another subcollection like that because then now so there's that's that's a really interesting way of putting it so that's actually let's create that so we can talk to it more so we'd have a sub collection and the reason why we have it as a sub collection is that way it's much it's basically the like writing a it's implicitly writing a query to get to this questions so for this is our ID for our game and we're saying game slash this this is the same and sequel is saying select star from questions where games are game IBM doing all in caps equal this ID and that's the same thing except no query involved so it's just it's the same concept so if you're from the sequel world you're like I just want working or at my select statement this is it your cat your structuring is your select statement I mean if you're a fire store no SQL user you might be wondering why not to make it a root collection mm-hmm some advantages of a sub collection or that the for one like David said the queries simplify it and it also reduces the amount of data that you need to save yeah the document because we'd have to save a game ID on every single question if it were in the root and we also have to most likely create an index if we wanted to query by multiple parameters on those questions yeah I'm so just friends kind of simplicity across the board yes and if you do want to do queries across subcollections you can do that with collection group queries mmm you have a good point that's kind of been a a game changer for data model and buyer store absolutely no it's fortunately for us and this since that's like way overkill because it's unlikely we would need to query across across questions at for too many reasons all right so we have questions and then we have this document ID and so the question would have like let's just say the question title and it would be what did I do for that one the text all right and so that's the title I'm just gonna save it there for now just to operate in the data view let's see all right so this operator I think encounter is merged together right so what else do you see Jeff living in this questions document the next thing we need is some options the question itself options that user can select and so again I'd like I probably make that a map I guess it could be a map okay yeah it doesn't I was just making a map because I already selected that but so what you I would be the key will be the ID for the option so each option should have a unique ID okay that can just be whatever but I wanna fix that I would right normally to make this unique I would actually write a script that could do this that populate this or I'd create like some admin UI and I would do the client generated unique keys so the value would be like what is the spread operator would be like one of them I think we're actually have to make this a map of a map for what hap yeah because we'll need to the question text actually I'm trying to think of this additional data that we're going to need beyond the question text but now that I think about it actually I think we really need anything else okay because yeah that's I was wondering about that myself because we don't want to embed for security purposes we don't want to embed like correct is correct true right yeah that's because even you think we have an option on that it says correct true or false but we're not gonna want that on this particular document because it would have to be delivered client-side and someone could potentially read that from the network yes I'm just gonna put two options and lazy alright so we have these two options right here Oh interesting yeah see there's the sort right there was just be the 4k yeah so if we'd want actual order then we would probably have to use an array which I don't think is all that the only thing we the only thing we need is the key I think to tell us which answer is correct which one so this is mostly what we need for the view but what would we do to keep the security model good so you know no savvy fire base user would know how to sniff out the answer well before we go into that there's one other thing we shot on this document is that like the actual state of the question because remember a quail being activated for 60 seconds that's right we need to have it in an initial state like an active state and then some kind of finalized a complete state okay so question state will just say though there's Oh finalized I see they're saying because there's like inactive like the question has not been answered so basically unanswered unanswered yaddam active answered yeah yeah yeah yeah that looks good let's just do it I answered and there might be like one other state while the scores are being finalized in the background it's probably unnecessary no no that's a really good that's good cuz that that just gives you more information in the UI to be like you know processing I guess would be like processing yeah right one thing that would be really cool in fire stories that you could create like a type like that where you have you know yeah that you can then like somehow use in the front end UI event it being like an enum type of like you would have like strong typing you basically like your own custom enum of like of those things in that way yeah now I see what you're saying that would be very very basically yeah you can't put the question in the wrong state and have your entire app break right yeah you would have lit would basically be like a reference type but but with more would just be way more tailored to your need right it's like something you can do in graph QL because it's the schemas strong type so you can kind of do things like that and graft you all but I'm a really good point okay so we have state options title would we have something like you're saying would it be like I guess start time for like the question I don't know I think that might almost be better on the game document because I was working in document could have a list of it the questions that actually well because we meant made this sub collection so actually that's not really necessary I don't know we might there probably should be some sort of timestamp on this document yeah well let's just put timestamp just for now and time stamp though would exist though if I'm sample would exist when the question was beginning to be answered you could have multiple time stamps here like it created at started at finished at a lot of times if I need to manage like if I window the actual time the state changed then I'll just put multiple time stamps like that you write all right we'll just start with started at four now yeah I think that's good for now we can kind of build on that once we get to the UI and figure out what we actually need exactly that's the great thing about no SQL is you have the flexibility early on to just kind of prototype and iterate until you actually know exactly what schema you need right right exactly and then we can lock down the exact schema we need with rules later right um okay cool so let's what do you want to get into next do you want to get into the server sinking of the timestamp or do you want to talk about the security model of not having the game being the question the answer the question being visible yeah let's take care of the answers I think okay would be the next logical thing sounds they're actually the other thing when you think about is the responses from the user oh yeah that's actually yes actually I agree with that yeah so what what is a good response or would you have that beat would that be like a sub : I would probably go yeah because the response is tied to a question so I would I would do it as a sub collection yes as you're thinking no then we might need to do a collection group query I know it's let's see so collection ideally this would be responses I imagine yeah that sounds good and then a response would you oh this is an interesting question would you have the response be a G unique generated ID or would you have it be the users ID I would have a users ID in this case because they usually have one response so when you have one of something owned by a user I would make it a user ID for the document ID this is another one of those like cool firestore modeling tricks to know my dream called a trick it's like an actual technique is that in any case we're like you want a unique like a distinct key like in sequel you can use some type of ID that's guaranteed that's distinct to a user or distinct to some property in this case when the most common technique is users so by making a document ID or collection ID or something like that as the users ID that way when we write to that we can actually check to see if they're if they're trying to submit multiple responses and then you can write a security rule that says hey if data already exists at this spot then get rid of reject it because we're not gonna allow you to answer twice or maybe we do want you to answer twice because you allow you to change your answer before time locks in or something like that but then that way we only we know exactly where to update and we're not like searching the list of unique IDs for which one's yours right yeah I think that's perfect that'll set us up for a rule that only allows users to create data and not update or delete data after it's been submitted exactly and so what fields do you imagine here like the the answer and would that be like the key of the answer yeah so that should be a key of one of that they'll get that key from the front-end code and that'll be one of those keys from the map and the question document and that seems like it's probably the only thing that we need to record maybe some time stamps but we also want yeah we may be a timestamp but we'll also want the score because we can't be calculating a time-based score here as well oh good call I didn't think about that yeah so you would have answered at I'm stamp yeah that'll be a server timestamp using the firebase SDK and then we can take that timestamp and validate it on the backend or calculate the actual score all on the server and so it's not based on the user's client-side clock or anything is the score the user's current score or is that like the score they got the answering the score would be set on the back end somehow what I thought yeah they'll create the initial document and then we use firebase admin on a cloud function to then calculate the score and update that so they'll be like a pending state while the scores being calculated so we don't need a score here then no like it's big it's that one there because it squirrel be just for like modeling purposes there will be a score set there eventually but it oh by the user okay it's all purposes all I'll put that in there okay all right all right so that right there makes sense and so now we have the response and then we'd have multiple ones per UID and then this is where we would set a clock function to listen to this creation state and then to be able to aggregate to the leaderboard in back all the way back and the root document yeah yeah that's right okay cool um I'm sorry were you saying um I'm not sure like well we'll have to kind of go through that and figure out and that would be for a future video but we'll go through and figure out how to do all that data aggregation yes um let's let me look at the I guess we can look at some of the question what would you wanna do I'm gonna look at the questions or do you want to what's like a good what you think is a good stopping point here well we need let's go ahead and set up the answer I think that's the last yeah I partner function we'll also want a collection for a user but that can be something very simple yeah yeah okay so what's so where where do you imagine answers would be his answer is going to be a sub collection of questions like answer yeah I think so yeah I think I probably make the most sense to just have that side by side and we'll give yellow the responses a collection than a answers collection that have the same matching document ID it's that way when were analyzing a question we can just pull the same document under document ID under the answers collection to get the answer for that in the back end I see so let's say it's this one it's the what is the spread operators the key so you would have it like this and maybe I'm not following you right but so like the collection ID for questions slush this answers this is the correct answer in this case would that contain whether it's true or not yeah we just contain the key that's correct so I would just have like what one field on there that says correct answer and oh okay actually wait would you put that on the question or would you put that on the case would you put that on the game I would put it on that a sub collection under questions so you see how you have your responses no not there um now getting confused it's a data so we got question yeah so I would put it yeah under games we have questions and then answers yes I think that's what I'm thinking too I wanna do it yes right here would do the same yeah I was thinking the same thing too okay yeah cuz then basically what you're saying is is that you could do it that way under questions it would be more complicated but possible but here you're just creating one big map of this is assuming that every question has a unique ID and that way you can have this giant map of ID and true or ID and you can't just have this isn't real some database Union object so he'd be like ID and is correct or something like that yeah I think you'd only need like really only needed the question and then like you mapped the question ID it's the answer ID so they share the same ID oh yeah pull that and that'll happens you have the correct option oh okay okay I see you're saying so oh okay so that way as you're in the back end you'll calculate all your scores you'll pull that answer document so you just keep that you know sitting there in memory and then you can go through each of the users responses and see if they selected the correct answer so I think that's like different names for the same thing because either way what you're oh yeah it's just literally different names for the same thing or it's different techniques that'll achieve the exact same thing and I think they're both it's the same performance it's just interesting to call this out is that my ideas that question answers would just be literally if ever if we can guarantee which you can that every question every answer has unique or every option you know answer option has a unique identifier then we can just map whether it's correct or not and that way you just do a quick lookup but it's the same thing in your case if you go to questions or answers by the question ID and just get this the one back what's the right one like it's the same right yeah so basically the lesson there is when you get in those type of situations where you're like which one's better and I you know in that kind of case it's achieving the exact same thing by a lookup each time right yeah there's a lot of a lot of trade-offs in like a lot of ways you can solve the same problem using multiple techniques and it usually it's kind of boils down to the trade-offs between them right and just funny enough this one has the same exact trade-off like it's just yeah so it's just like it doesn't matter I mean you could eventually migrate to the other data structure if you find there's some big performance gain all right usually it's not gonna end up being you didn't need crazy amounts of data and like a lot of traffic at one location to start seeing games like that right I make sure I copied this come on I know that we have these lock down actually what is it yes it's this one all right I just copy in the bar all right so now start collection a collection ID is going to be answers and so for this question ID we would have a field that would just be like correct answer and it would be the key of the correct answer right right yeah I remember the key is so just right correct key so that way now when we so this right here would only be accessed by the admin SDK and the cloud function right so that way when we run this that it's able to be like oh we answered that question let's go look it up let's go apply that and then it can run through the users answers the responses that is and set of questions so it's going to go through all of these responses look at their answer so whatever the key is and then it's going to be able to look that up and be like okay this was the correct one did you get it correct yes you did congratulations you answered it at this time this start time was this so that gives you a score of here and then we'd write that scoring right yeah exactly that's what I had in mind all right sweet we are on the same exact page so yeah that sounds like that whole system will work really well in a real time like in a very write heavy race condition fearing world because you have distributed the things out that could be contentious and you are aggregating them by one responsible like you know security given client that being the cloud function right yeah I think we've kind of minimized the reads by aggregating the leaderboard and things like that we don't need any math some queries on the client-side and then we're doing all of our you know calculations and writes with just a single cloud function call so looking at Max position what's interesting is actually I really liked how you brought up the client-side query part that is a very easy way of when you're first starting out to kind of prototype your app up without throwing up a cloud function is that you're saying like oh let me just run getting the data on the client that starts to break down as you as the security model grows more complex just like we have for the answer here you wouldn't query what the answer is on the client because then the client would have access to the answer so then we would need some type of server component but since you're already running a server component you might as well enforce all the rights at once with the cloud function and kind of functions are cheap because what you get like two million invocations for free a month and so if you're only running one cloud function per answer and we're running ten questions on this game that's gonna be free and you're not going to be hammering not everyone's be pulling down every doc or writing or fanning out on the clients all these Doc's or stuff so you're really aggregating or you're consolidating your your reads and your rights so you're reducing the amount of reads and writes therefore reducing the amount of cost that you would see from fires for all this though would be within the free tier so we're just projecting lots of traffic but the cloud function would still just be once even if there was a lot of traffic right see I made a video a bike probably bout a year ago about how did not get a thirty thousand dollar buyer because those great great videos yeah that's basically like related to this if we were to instead of managing all the responses on the back end we could just query every single response for every single user and calculate them but then we'd have you know an exponential number of reads on the client side which would obviously cost us a lot more money yes yeah I like this architecture a lot because not only is it performance wise doing the correct things but it's also in line with cost as well so like as traffic scales up the cost will not expend in Chile scale-up as well because it's still one club function that runs per question right yeah I would say like each client is probably going to execute maybe like 20 to 50 document reads throughout the course of the game they something like that and then you know hand flow rights on the back end which you know feel like a big game it's not very much considering how many firestore reads you get it you know it'll localize and all that exactly so this this is really good because you're not you're not leaning heavily on one metric that could push cost you sort of distribute out where you're relying on so and like the club function in a way is basically like a cache in front of your fire store reads and writes right yeah yeah yeah the goal was just to not like end up with an exponential or you're executing exponential needs as your data grows exactly so yeah this is yeah exactly you it's the exponential stopper I don't know there's probably a bit more it's kind of like like big o-notation with algorithms yep yeah about how many how your reads are gonna scale a lot based on how you know then they're looking right now yeah which is exactly like when you if you ever are building an app on fire store or fire base in general and you're all and you're worried at launch what's gonna happen like oh what if I get put on the top of some reddit subreddit and or people retweet it like crazy and everyone goes to it and you're worried about the cost well free tiers really big but on top of that this is the same the way that Jeff and I have talked about it is the way you look at it like where would all of those where all my rights where all my reads how like you know how many times with the cloud function be triggering and you look at all those instances and you look to see where if you're gonna be reading a ton of data or writing a ton of data from the client where could I - that up in a cloud function is usually the most practical place to do that right yes I usually tell people to try and calculate just the number of reads and writes that a single user has throughout the course of the app just gonna average that out and then just multiply it by you know however many users you expect to grow - so yes you know that hopefully that number is just linear like so if you have ten reads and then you have a thousand users you have thousand raids but that if that number goes up to a hundred thousand then you probably have some kind of issue somewhere with the way things are set up right so all right let's see I think that was a really good modelling I think we can probably get into I know that you have that boiler plate we're gonna do this whole thing it's felt by the way which I'm super excited about and so that'll probably be next episode we'll probably start reading this data and start creating some mock UI for it yeah I think that'd be a good next step we'll get get the UI going and all that stuff and then maybe after that finalize everything with the backend data aggregation mm-hmm and just real quick before we go I'm gonna go through some of the questions or comments Gary says he's gonna send me matcha tea Gary you send me some matcha tea I'll drink it let's the air web developer TV says hello do okay they're flipped over today Ben says do you name your collections plural games or game what is your best practice I like I like the plural plural convention that's gonna have most frameworks when they have like collections of data it should be plural it's kind of convention over configuration just so it's predictable agreeing Matt Carson asked can you query the query bog can you query by the ID of a map in an array and you ever wait he's talking about straight to a field I think so like you can do the dot notation if you have oh yes you're right I forgot about that that's been like a forever feature I haven't used that in a long time I was like a dick feature I forgot about that I'm never seeing that beam mind blown I just never had to use it but could I use it quite a bit I use it I have a dad geo was that pretty heavy way too oh yeah that's that's a very good point anything with yeah because latitude longitude all that um although manner let's see when I plan on the normalizing data I do something like user ID X then add items to that map display name oh and maybe he's commenting to that I thought that was gonna be a question yeah that's good yeah Cedric too like the sequel translation I think that helps help me out a lot when I first because I came from a very heavy sequel back and I basically got my degree in sequel databases it's just funny because I'm like I can't remember less than I've really gone created like a big heavy sequel thing but my whole my whole undergrad and I say undergrad like I didn't over grad or you know post-grad I didn't I only did an undergrad um and my college time and my early career was all all sequel the first F sequel database I ever used is there a library equivalent that could convert sequel statements to fire store equivalent there actually is it's called fire sequel it's a JavaScript library it's written by Joseph sale and it is really good it basically has a little client-side ASD that will translate that into fire store statements and it's really cool you should check it out it's the NPM package is named fire sequel Jeff firebase admin does not cache documents in memory right yeah that's right if you have like multiple like cloud function invocations they won't be cached in memory as far as I know right I don't think that the admin does the the same caching as The Client SDK does but even if it did it would just get wiped out once the function ran out so you can't really rely on that yeah like under the hood you can think of your function as a container that gets spun out based on the amount of demand yep you have action that's being hit all the time it should stay spun up and then it'll scale additional containers as you did yeah if you yeah I'm not sure I'm going with that but like in our case we're just reading one single document in the cloud function and then we'll calculate all the scores which is that one single document reading yeah sense I guess yeah yeah we don't we're not we're not relying on the caching in this case Kevin asked any good method tool best practice to document your firestore schema before you start your actual implementation I actually have there's multiple ways of doing this I like to start out by different view I like to start out by really I should you just do it markdown to start out with just to really kind of like get those thoughts out of your brain and onto paper so very quickly and then from there you can sometimes actually come into sketch and I draw the schema like a little doc that I'll have alt / they slash that to those parts of the UI and then really if you really want things like hardcore documented then some of the best types of documentation are the rules and the tests so if you start writing your rules out that will serve a schema so you don't have to worry about read and write validations at first you can just worry about the actual shape validations of the security rules to make sure that it contains all the properties you expect those tend to be just little functions Mike McDonald years ago did a really good video on that for role based systems Jeff I'm sure you have like 16 yes I do that as well yeah one thing I'll add to that is that for most of my projects I'm using type script and for datum only one thing I like to do is just start directly with interfaces and kind of build out those interfaces and that's another way you can potentially model your data we have this tool that'll actually take your interfaces typescript interfaces and turn them into your validation functions but I think it requires like some binary or some runtime that's not friendly for most computers I can't remember what it is so it's like haven't you guys used internally no it's just like I think it's like you need some I can remember what it is now it's been a while since looked at it but I don't know somebody one engineer internally was working on it it was open source it's like out there somewhere but they never got finished but it was a really cool idea to be like yeah just give me your typescript types and then it's only for validations but it will create all your function validations and keep them kind of in sync with your your code yeah just off and then lastly to you can use the emulator Kevin to write unit tests to spin up the emulator and run actual right just like you would write like a jasmine or mocha test and be like here's an object I expect this to pass I expect this to fail and then just expect this that and then that's a really nice way of documenting and making sure all your schema stuff is lights out correct yeah I would say that's the best way to do it in production when your rules really matter how many regressions that lead security leaks you know which is especially important as your app grows yes couldn't agree more David says that he just uses a Google Doc which is the best way to really get started on anything oh yeah that's awesome yeah I use Google blocks for tons of things like notes for all my videos that I make I usually start with a Google Doc and sketch out an outline and I keep a ton of stuff in there yeah nice to see you too thank you for stopping by and Cedric we don't know when the next sessions planned but this was fun so we'll do it sometime so when you got a we try to be organized beforehand tries a very key word there but uh but we want to we're gonna have to figure out how we want to move forward with the UI we're learning each and every time so an on dress thank you thank you for something I always love seeing you and so yeah thanks for stopping by this is we're just kind of exploring this we really want to run this as a live stream eventually we think this would just be super fun to get everyone together and answer trying to find some some esoteric go through JavaScript Y and C and those could be some oh that would be a great category JavaScript watt or Watts being the category in E Plus array equals right yeah be good or like a in jeopardy at popular categories before and after so we could use some like weird JavaScript mutation birthdays would be like Kristi others before you do a trace slice now what happens to it afterwards oh yeah it would be like 1% gets it right I don't know and let's see or also much JavaScript history to doing some going back to light Netscape and that kind of stuff yeah how may I'd like this many days it took to build JavaScript yeah cuz I yeah I made a whole video on that which I think I was like 14 days Brendan Eich netscape mama wants to know if you could share how you make your videos I get that question a lot and I made a livestream about that maybe probably over a year ago at this point but it's on my list I'd like to make a video for that in the future yeah you do your stuff is good thanks and Lavi a asks any special reason for choosing spelt uh uh what Jeff's the one who who was like we should use I didn't I did say no or argue I agreed but you can go forth in time so I think this is a really good use case for spelt because it's a single page application without any routing or anything like that mm-hmm and I also wrote a package called Spotfire that takes basically your flyer store and firebase data and puts it into small components that you can easily just prototype with I mean you lie and it'll be a perfect use case for this particular app yeah we're not carried we don't care about SEO we don't care like none of this is going to be static outside of maybe some app shell stuff we might do so this is very happy app world of web I always like to think of like the archetypes of web development Jason Miller once did a really good blog post and like tweet series about this like archetypes of apps versus like the architecture you should use love that one but yeah in this case this is so happy that we don't have to worry about like it's going to get indexed what static needs to be there and all that right so sweet well thank you all so much for coming out I'm gonna go get my son to a first grade phonics class that's was that knock was I mean so I will see you all when we will throw it up on YouTube and tweet out about it and let you all know so thank you all so much for coming out and we'll see you on the next one thanks everybody see ya

Channel: David East

Views: 5,537

Rating: 4.9661016 out of 5

Keywords:

Id: i-SP9WeZ5SE

Length: 76min 30sec (4590 seconds)

Published: Mon May 04 2020