SPEAKER: So as you recall
from our last video, there are a number
of rules governing how Cloud Firestore works
that all play a part in how you should structure your data. And while it's fine to talk
about these rules in theory, I think sometimes
it's better if we can put these in the
context of some real-world or close to real-world examples. So come on with me. Let's figure out the best
way to structure our data on this episode of Get
to Know Cloud Firestore. [MUSIC PLAYING] So for the purposes
of this video, let's keep pretending
that I've got this great restaurant-review
app that I'm powering through Cloud Firestore. Maybe it's got a
search page where you can search for the
top-20 Japanese restaurants in Boston, stuff like that. And from the Search
Result screen a user could click
on a restaurant to see details about it. This Detail screen
would have more info about the restaurant along
with maybe a few snippets from its most recent reviews. We'd have an option for a
user to see more reviews, and if they click on any
of these review snippets we would show them the
full and complete review, so pretty standard. Let's tackle a few
questions around how we want to store our data. For starters, where
would we put the reviews? When I've used this
example in previous videos, I've generally assumed we've had
this kind of database design. We've got the restaurants
in one collection and all the reviews are in
individual subcollections. But is this actually
the right design? Let's look at a couple
of alternatives. Alternative number one,
let's not make these reviews separate documents. We can stick all the review
data into the main restaurant document as a map
or an array of maps. Once a user clicks
on a restaurant, I could give them these
little review snippets immediately because I've
already got them loaded up. And hey, as a nice
little side effect, I've also saved myself a
bunch of document reads. So this is an alternative,
but maybe not a very good one. One problem with
keeping our reviews in a map field like this
is that when a user decides to search for the top-20
Japanese restaurants in their city, we're not only
grabbing all the information about the restaurant
but we're also grabbing every single review for
every restaurant on that list. And that is a bunch of
reviews and, frankly, a whole lot of data that they
will probably never need. I'm also going to
run into limits, either the 1 meg of data limit
or the 20,000-field limit if these restaurants
get too many reviews. I also can't query these
reviews either, right? I can't filter for just the 10
most recent reviews or anything like that. I only get back my giant
restaurant document with all the reviews
in there, and if I want to sort those reviews
or filter them or something, I would need to do
that on the client. So I think keeping reviews in
their own separate collection is the right way to
go, but let's look at our second alternative which
is what if rather than keeping these reviews as a subcollection
for each restaurant they were just in a completely
separate top-level collection? Well, now this gets interesting. Your first impression might
be like, oh, don't do that. It's going to be a lot
harder to get reviews for a single restaurant. But that's not the case. Remember, queries
in Cloud Firestore are always fast, so grabbing
all reviews for Todd's Tacos from a single collection
of 3 million reviews takes essentially the
same amount of time as just grabbing all the reviews
from a single subcollection. More importantly, by
putting everything into a top-level
collection like this, I can do searches
for reviews that span different restaurants, like
I could search for all reviews by an individual author. That's not something I
could do by keeping them in separate subcollections
like this, at least not until collection-group
queries get supported. And so if I went with the
subcollection strategy and I knew I was
going to have to find all reviews by a
single user, I'd need some kind of
workaround that I would be doing in advance. Probably what I'd is over
in the user collection I'd keep a list
of all the reviews that this user has
written and then use Cloud Functions to keep
all this data up to date. Sure, it's not the
end of the world, but none of that
extra work would be needed if reviews were just
in another top-level collection like this. On the other hand,
if I have everything in a big top-level
collection, some things are a little harder. Specifically doing
something simple like grabbing reviews for
a restaurant sorted by date or sorted by score would
require a composite index, whereas I wouldn't need that
if these reviews were already in their own subcollection. Also, I suspect a
number of security rules will be easier to write when
I'm dealing with subcollections of a specific document instead
of a top-level collection, mostly in cases where you're
going to want to restrict access to a subcollection
based on information that's in their parent document. But really, either one of
these solutions is fine. I've seen them both
used in the wild. But if I'm going to be
totally honest with you, the main reason I went with
reviews as a subcollection is that it makes it easier
for me to emphasize the whole, hey, look, Firestore
supports shallow queries thing. So I'm going to keep using this
one for demonstration purposes, but you feel free to
do what's best for you. Now as long as we're here, let's
talk one further refinement. Remember that in my
Restaurant Details page I've got my restaurant info
and then these snippets from 10 recent review
section at the bottom. And so with my current
database structure, I'm either populating
these reviews snippets from a review subcollection
or my reviews top-level collection. And that's fine, but that
does mean bringing up a single restaurant page is now
11 document reads instead of 1. And yes, I will get
billed for those. And if you consider that
a typical user might click on some restaurant
details, skim the reviews, go back and repeat that
process several times, that can add up
to a lot of reads. And so if I really wanted to
avoid all those extra reads, I might look at creating
a review snippets field in the original
restaurant document. This could contain the
first hundred words of the 10 most recent reviews,
maybe a headline and the author name-- basically enough to create
a presentable review snippet but not so much
that I'm downloading a ton of unnecessary
data for each restaurant. And again with some
Cloud Functions, which I promise I will talk
about in a future video, I can keep those in sync. Now this is nice because
I'm now only reading in 1 document instead of 11,
and it simplifies the loading of this page because I only
need that one document which, most of the time, will
already be in memory. Plus if I ever do need to show
more reviews or the full text of one of those reviews,
then I can read those in from the individual
review documents. But is this optimization
worth the extra code needed to keep
these values in sync along with the extra
data our user is loading in every time they
bring in a restaurant? Honestly, I'm not sure. Viewing restaurant details
is a user-powered action, which means it's not going
to happen several times a second, right? It will more often
happen a few times over the course of a
several-minute session. So this could be a case where
you end up doing a bunch of work to save a million
read a day or $0.60, which may or may
not be worth it. So I think this is one of
those situations where you're going to have to measure what's
actually happening in your app and decide accordingly. I know that's kind of a
wishy-washy recommendation, but that's software
development for you. OK, next example-- how would
we store flags for restaurants? And I'm talking about
little binary values for takes reservations,
is romantic, and so on. Put those away. Two months ago, I would
have said the best way to store these would
have been either to make these as individual
fields inside the restaurant document or group
them into a map field. And to be honest,
these are still perfectly reasonable solutions. But now that we've added
the array_contains field, I could just as easily put
these into an attributes array. Then it would be pretty easy to
search for all restaurants that take reservations. But keep in mind
that right now you can't run two array_contains
queries on the same array. So if I really do want to look
for restaurants that are both romantic and kid friendly--
which is kind of an oxymoron, I know-- I should keep using
separate fields. Should have read
the reviews first. OK, now here's a
fairly common case you're going to come across. What about storing
a list of users who can access a document? Now this is a very
common piece of data you're going to want to
keep track of, particularly in collaborative apps
where users can invite their friends to view
or edit their data, and you're going to need to
keep track of who can edit what. For example, let's
say a restaurant has a list of editors, and
these are the user IDs of people who are allowed to
edit the restaurant document. Now I know I haven't really
talked about security rules yet. That will be in the next video. But one thing you should
know about security rules is that they have the ability
to grab a specific document and read its content. So I could, for
instance, put everybody who's allowed to edit
a particular document into an array, and
then my security rules could say something
like, hey, only let users edit this document
if their user ID appears in this array. But what could work better
would be to use a map here where a user ID is the
key and their value is their role, something like
editor, owner, community manager, et cetera. And so we could
write a security rule that says a user can
edit this document if document.roles.userID equals
editor or something like that. Now this is great, but it
does have one small issue in that when you
retrieve this document, all the contents
of this document are transferred along with it. Remember there's no such
thing as a partial document query on the client. So you're kind of
leaking the list of users and their roles for
this restaurant. Now this may be OK. These user IDs
are pretty opaque. You can't really reverse
engineer them to figure out who the underlying
user really is, and they are unique per app. But still, the
phrase this may be OK isn't super reassuring when
we're talking about security. So this information
might be better stored outside of this document
in one of two ways. Option one, have a distinct
editor subcollection where the document ID might
be the user ID of the user, and maybe we store
this person's role and some other little
information about them inside each document. Or option number two,
just have a subcollection called private data that
contains only one document. And this is where
you can keep your "who has access to what" map
that we talked about earlier along with other information
that you might not want to share with
the general public-- maybe data for your
internal sales team or data that Cloud Functions will
need to perform its duties. Now both of these examples
work fine, but definitely think about this
kind of situation for your app because
you will probably want to create access
control lists like this and you will probably have data
associated with the document that maybe you don't
want everybody to see. OK, last example, but maybe
the most important one. What if my user
wanted to store a list of their favorite restaurants? This is the kind of
many-to-many relationship that often is problematic
in NoSQL databases. So first, let's assume that
our database has some kind of users collection somewhere. Well, one option is we can just
store a list of restaurant IDs as an array in a favorites
field for each of these user documents. Thanks to our new array
union and removal methods, it should be pretty easy
to maintain this list as our user adds or
removes restaurants from their favorites. If I load this array into memory
when our application first starts up, my client
could very easily tag a restaurant as a favorite
when we see it in the app. The problem, though, is
that it's hard for us to do a "hey, here's all
your favorite restaurants" kind of page using
an array like this. I would basically have
to do a separate query for every single restaurant ID
that I get back from this array in order to populate this page,
or maybe slightly better, get a callable Cloud Function
to do this work for me, and I'll talk about
that in a later video. Now this isn't a
great experience, but it's not terrible either. And if I think
this feature isn't going to be used
very often, then this would be a perfectly
fine solution. Don't let this scare you away. On the other hand,
if this is going to be a frequently
used feature, there may be ways of making this
a little easier and more performant by using
some denormalized data. For example, instead
of just storing a list of restaurant IDs,
I can keep a big old map of maps that would contain
enough data to populate a simple My Favorite
Restaurant screen. Maybe I have the restaurant
name, cuisine type, and address stored in this favorites field. That might be enough to
populate the My Favorite screen. And if the user ever wanted more
information about a restaurant, well, then I can query the
full restaurant document and populate a Restaurant
Detail screen just like normal. But the trick with
denormalized data is that we need to make sure
that if Todd's Tofu Hut ever changes their cuisine
from Japanese to burgers, every copy of that
denormalized data has to change as
well, which means we need to be able to
query every user who has this restaurant listed
among their favorites so we can make that change. As it turns out
though, we can do that. I can basically create a
query that says something like select all
users where favorites dot restaurant 4215 is
greater than the empty string. That would give me a list
of all user documents where this restaurant is listed
as a favorite in the map field somewhere, and it would be
fairly easy to have a Cloud Function update all those. Now the disadvantage
is that this is a big chunk of extra data
that I'm loading per user, and we are eating
away at my 1 meg slash 20,000 fields limit here. So I'm going to need to limit
the number of restaurants my user lists as a favorite. Although honestly, that
kind of makes sense. If you list 500 restaurants
as your favorite, hasn't the word favorite
lost all meaning? This got kind of philosophical. Now another option might be to
keep my user's favorite list as an array of IDs and then
keep the restaurant snippets inside a subcollection. I could then load a list of
the user's favorite restaurants in memory without loading
all the extra restaurant details along with
them, but they're also easy to load up if I ever need
to show this favorite screen by querying the subcollection. And I can still
find a list of users who have favorited a particular
restaurant by running an array_contains query
across my users collection. Now once I have that,
it's fairly easy to get at the individual
snippets that I need to change, although you'll notice I can't
do it all in one single query now. I basically have to fetch
these documents one at a time. Also, just a heads up, if you
have an array that you're going to use with an
array_contains query, those array elements
will still count against your 20,000-fields limit. So I'm still going
to need to limit my user's favorite restaurants
to a reasonable number. But you know what option
might work best here-- a completely separate
top-level collection. Imagine that I have a
FavoriteRestaurants collection. Every document in there would
contain a user ID, a restaurant ID, and enough of a snippet
about the restaurant to populate a My Favorite
Restaurants page. Now grabbing all the restaurants
favorited by a specific user is a very simple query, so
creating that My Favorites page is very straightforward,
and it's just as easy to grab all the instances
of a particular restaurant if I ever need to change or
update this denormalized data. So you know what? I'm going to say this is my
favorite option, at least for this particular setup. Whew. All right, well that was a
lot of examples to go through, but I'm hoping by now you're
getting a better sense of when to use documents,
when to use maps, when to use subcollections,
when to use arrays, and when to put things in a
separate top-level collection. Now there's always
more to explore here, and as you've noticed,
there's rarely one simple answer to anything. There will always be
trade-offs no matter what option you choose. So hey, if you have a
setup you like better than any of these situations,
go ahead and share them in the comments below. And I will see you on
a future episode of Get to Know Cloud Firestore. Well, I think that's a wrap. All you all hungry? Let's go to Todd's Tofu Hut. I hear they serve burgers now. No, no, meat burgers. I don't know why they
didn't change the name. Ask them. No, it's a different Todd. Come on, let's just go. [MUSIC PLAYING]
