SPEAKER 1: Money. They say it can't buy happiness,
but it can buy jelly beans, and that's almost
the same thing. So what do you need to know
about how Cloud Firestore works to make sure you're not
spending too much money? How is it different
than Realtime database? And did I just make
that jelly beans remark so I could be eating
jelly beans in the next scene? Let's find out on this episode
of Get to Know Cloud Firestore. [MUSIC PLAYING] Wow, these are good. Do you want one? SPEAKER 2: Unh unh. SPEAKER 1: They're delicious. So I think a lot of folks coming
in from Realtime Database land might be wondering where
the automatic, hey, just migrate my entire database
to Cloud Firestore button is. And the truth is, we
don't have that button, and I don't know
if we ever will. And there's a couple
of reasons for this. One, thanks to a lot of
the things I talked about in the last video--
shallow queries, and the ability to run queries
across multiple fields-- you can structure your data in
Cloud Firestore in a way that probably makes a lot
more sense logically, and some of the workarounds you
had in the Realtime Database world, like combo fields or
unnecessarily shallow trees, aren't really needed anymore. But two, the way
these two databases determine pricing
is very different, and if you take your
entire app that's optimized for one
pricing model and sync it into a completely
different pricing model, you might end up spending
more than you needed, and that means less jelly
beans for everybody, and that would be sad. So let's talk about pricing. And before I do, let me hit
you up with a quick disclaimer to make our lawyers happy. [CLEARING THROAT] (SPEAKING VERY
QUICKLY) Please note that all pricing guidelines
and examples in this video are used only for
demonstration purposes, and may not reflect your
full and accurate Cloud Firestore pricing. Your app's needs are
unique, and might not benefit from recommendations
laid out in this video. Be sure to review the full
documentation on the Cloud Firestore website for a
more complete sense of Cloud Firestore pricing to best
determine which pricing strategies are right for you. Wow, that was exciting. So if you're coming here
from Realtime Database land, you're probably used to
having most your costs be proportional to the amount
of data that you download and upload to the database. Cloud Firestore is different. When it comes to
pricing, it cares less about the amount of
data you download, and more about the number
of operations you perform. More specifically,
Firestore primarily charges based on the number of
reads, writes, and deletes you perform in the database. But what is a read or a write? It seems obvious, but let's
go into this a little further. So writes are charged
whenever you create or update a document, and this holds
true no matter how much of that document you are changing. Whether you're changing a
single field from true to false, or swapping out 30 different
fields all at once, it still just
counts as one write. A read occurs any time a client
gets data from a document. So if I say, hey,
Cloud Firestore, give me the top 20 Japanese
restaurants in my area, and Firestore gives
me back 20 documents, that will count as 20 reads. And the nice thing here
is only the documents that are retrieved are counted. If you remember
from our last video, Firestore does all
of its searching through indexes, not the
documents themselves. So even if I have 30 million
restaurants to search through, asking for the top 20
Japanese restaurants in my city still only results in 20 reads. On the other hand,
if I were to say, hey, show me all the
restaurants in my city, that could result in a
boatload of reads, and honestly probably
more than I need. That's one big reason why
pagination is important. And we will talk about
that in a future video. Also keep in mind
that these reeds apply to real time updates as well. Every new update my client
receives also counts as a read, but only for the
document that's changed. For example, let's say I've
got a real time listener set up for these top 20
Japanese restaurants. Now if one of these restaurants
suddenly changes their name, that new document would
be sent to my client, and that will count
as a read, but it only counts as one document read. And this is true even though
in my listener method, I'll still get all 20 documents. Firestore is smart
enough to merge that one new document
alongside the cached data that hasn't changed. (BABY TALK) Ooh, who's
a smart little database? Yes, you are. So that's all
generally good news, but let's look at a
situation where things might get a little pricier. Suppose instead of
my restaurant app, I have a stock
ticker app where I'm updating the price of every
stock on the backend every 15 seconds. And imagine somebody has my
app open in the foreground, and they're following
one single stock symbol. Well if they keep their
app open for three minutes, that app will receive
about 12 updates, which counts as 12 reads. And if that person is following
10 different stock symbols, well, we're now up to 120 reads
for that user's three minute session. And my gosh, if I have, like,
1,000 different users all using my app at that time,
well that's 120,000 reads right there just
for those three minutes. I mean, if I were to
keep up that pace, that averages out to like 57
and 1/2 million reads a day, and-- wait, wait,
are you freaking out? Stop. Stop. Calm down. It's OK. Breathe into a paper
bag or something. You see, I get it. We're engineers here. We're all about calculating
the big O of things, and seeing exactly how
big stuff can scale. And if we see that the
number of reads scales proportionally to, like, stocks
times users, times updates, that does scare
us, because we can see exactly how big this could
get in a worst case scenario. But things aren't always as
bad as they first appear. Let's go back to our stock app. If we've got 1,000
different users all using our app for
a three minute session, well that works
out to roughly half a million daily active users. And those 57
million daily reads, that's going to cost
us about $34 a day. And honestly, if
you're paying $34 a day for an auto-syncing
multi-region database with half a million
DAU, that's not so bad. And there are clearly
ways I could limit this if I needed to, right. I could reduce how often I
update these stock prices, make it a pull-to-refresh
situation, or extreme cases-- maybe not this one --set
up a caching layer. So I don't want you
to automatically panic at the worst case
scenario, but I do want you to be aware of
the cases in which Cloud Firestore could make you panic. As a general rule, I like to
say that the more real-timey and the more crowded
an app feels, the more it will probably
cost in Cloud Firestore land. So let's go back to our
restaurant review app. This is really about
delivering large chunks of data fairly infrequently. As a user, I might
run a search, and that could bring in 20
documents, and maybe I drill down to see details
on one of these restaurants, and I might fetch in the
10 most recent reviews. But these documents, they're not
going to be updated very often, and these reads are going to be
pretty infrequent because, you know, even skimming
reviews takes time. Writes are also going
to be pretty infrequent because it takes a pretty long
time to compose a restaurant review. Even if my app auto saved
a draft every 30 seconds, that's going to be
pretty small potatoes in the grand scheme of things. So I think my restaurant
app is probably set up pretty nicely
to take advantage of Firestore's pricing model. The same would go for anything
like a blog, a wiki, or maybe a news app. Cloud Firestore
might also work well with a turn-based
multi-player game. If you think about it, a
typical turn in a board game usually takes a
couple of minutes. So using Cloud
Firestore to power a multi-player chess
game or a word game would probably be
pretty reasonable. Cromulent? SPEAKER 2: Mm-hmm. SPEAKER 1: I don't know
if that's a real word. Now on the other hand, imagine
you've got a group whiteboard app where lots of people
all get to scribble on the same canvas
at the same time. If you're sending tiny updates
at the rate of several times a second for that real
time drawing feel, you're going to
generate a lot of writes and a lot of reads
per second, and this might be one case where the
real time database would be better suited
to the situation, at least from a
pricing standpoint. OK, now what about a chat app? Well this gets
interesting because this is something that
could vary wildly because the number of
reads is very much affected by the number of people
in the same chat room. Let's think about
a one-on-one chat. If we assume that every
person sends a chat message every 10 seconds, two people
chatting for five minutes will generate about 60
writes and 120 reads. So if I take 100
people and group them into 50 different
one-on-one chats, I'm looking at 3,000
writes, 6,000 reads, and that's pretty reasonable. So if you wanted to use
Cloud Firestore to power a person-to-person
chat in a dating app, or maybe let users chat
with customer service representatives,
that would probably work out pretty nicely. On the other hand, what if
we took those same 100 people all still chatting
away at the same rate, and put them all in the
same room where everybody could hear each other? Well now all 100 people
get an update whenever anybody writes a message. So I still have those
same 3,000 writes, but now instead
of 6,000 reads, I have 300,000 reads, which
is a pretty big jump. But even that estimate
is probably misleading, because it turns
out that when you get into a chat
room that crowded, people tend not to
chat at the same-- would you be quiet? I am working on a
video for YouTube. As I was saying, people tend
to not chat at the same rate as before. It just gets too noisy. And if the main lesson you're
getting out of all of this is that it's hard to provide
heuristics or guidelines that work in every situation,
hey, then congratulations, you're getting it. You really are going to have
to look at your app's behavior and figure it out from there. Now while we're here let's
talk about a few things that might affect reads or writes
that you might not expect. First up is Cloud Functions. As you know, these things
can also read or write to the database, and when
they do, that gets billed. Suppose I've got a
Cloud Function that updates my average score
on my restaurant app whenever somebody creates or
updates a restaurant review. Seems reasonable,
but what if I decided I was going to calculate the
average rating by gathering up all the reviews-- the
reviews sub-collection, reading in every review
score, and then writing the average back to the
original restaurant document. I'm hoping this is
generating some alarm bells in the back of your
head, right, because this is going to generate a
whole bunch of extra reads that you might not want to
perform every single time a user changes a review. I think it would be better
to use some clever math. Like if you store an average
review and a number of reviews value on that
restaurant document, then you could change the
average for a new review without having to go back
through all of the other review documents. Or if you really like the idea
of writing a function that looks at every review
for a restaurant, at least make it a Cron job that
runs only once every few days. Another thing to look out
for is security rules. They could generate extra
reads in your database, too. I know I haven't really talked
about security rules yet. That will be covered
in a separate video. But when it comes to
evaluating whether or not a certain action is
permitted, sometimes you'll need to check an arbitrary
document in the database. These are checks you'll
usually be making with get or exists calls, and
these two calls typically count as an extra read. That said, Cloud
Firestore is pretty smart about caching
these values when it can, and in practice, security
rules are probably going to be much less of a
concern than Cloud Functions when it comes to pricing. Sure an operation might
trigger an extra reader or two with security rules, but
it won't scale dramatically the way a badly written
Cloud Function might. This is definitely an area
where you're better off optimizing for security
instead of getting overly clever in some quest
to minimize database reads. In fact, as a general rule,
I'd be careful about overly optimizing for pricing. While you definitely
want to avoid any boneheaded maneuvers
that will drastically drive up costs, I've also seen some pretty
convoluted schemes out there where developers will spend
hours creating overly complex code that ends up
saving them $0.30 a day. Remember, your development
and debugging time costs money, too. Don't be penny wise
and pound foolish. Sorry. Rant over. So let's look at
a few other places where Cloud Firestore
can cost you money. I've generally ignored
bandwidth costs in this video, and that's because while
there can be a cost associated with outbound
network usage or what we like to call network
egress, which it turns out, is not a type of bird, it's
generally a very minor factor in the grand scheme of things. Like, I think you get the first
30 gigabytes of data for free, and for most larger
customers, this ends up being like 4% of their bill. So I tend not to worry about
network usage too much. Similarly there's a cost for
storing data in the database, and this includes not
just your data, but also the indexes and metadata
that goes along with it. But again, this is pretty
cheap, roughly 41 times cheaper than what you'd be paying
in the real time database. So in general, it
tends to be not as much of a factor in your pricing. And this cheaper
storage in fact means you can look at
doing things that might have been too expensive
to be practical in real time database land. For instance, every
time a user decides to update the review
for a restaurant, I could totally
save that revision as a completely new document,
and keep the old ones around. Then my user could go back and
revisit every single revision they've ever made. Sure maybe that's a little
overkill for a restaurant review site, but it would
be really useful in, like, a blogging site. Also, if I wanted to store
icon or small profile picture for every
user, I could totally do that right there
in the database without having to look
at a separate service. Remember, Cloud
Firestore supports storage of binary
objects as values, so I could keep a small
JPEG or PNG file in there. Keep in mind you're limited
to one meg per document, but you know what,
for a small icon, I think you'd be fine here. So now that you know what
you're going to be charged for, let's talk about measuring
and estimating costs, because I think all of
us worry, whether you're on Cloud Firestore or
any other cloud service, or just running
your own backend, there's going to be
some error in our code, or a badly thought
out architecture that's going to end up flooding
our service without our realizing it. So while it's
certainly important whether a service is
cheap or expensive, sometimes what matters
more is that it's measurable and predictable. And I will say the nice
thing about looking at reads and writes as your main pricing
factor is that estimating those is a heck of a lot easier than
estimating download sizes, especially because you can
limit the number of items that get returned from a query. So I can easily set up my
app to return only the top 20 results when I perform
a search, and then I know exactly how much that
search is going to cost. It would be way
weirder to ask it to limit my results to the top
2 meg worth of restaurant data. I don't even know what
that would look like. But outside of
estimating costs, let's take a look at how
you can get details on your actual real world
usage, and get warnings if things seem out of whack. And I'll be honest here,
this is an area where I find things are a little
rough around the edges, and I wouldn't be surprised if
you saw some nice improvements over the next several months. But let's say your app
has been out for a while, and you're curious if your
reads and writes match up with what you're expecting. How do you find this out? Well to get this
data, you're going to need to get added
to Firebase console and jump into the
Google Cloud console. You see, every Cloud Firestore
database that you create is attached to an app
in your project, which means that if you want to
see your database usage, you'll want to go to the Google
Cloud console at this URL here, and select App
Engine, and then head over to the Quotas page. You'll be presented with a
bunch of different usage stats. The three you are
most interested in are these three here. Firestore Read Operations,
Firestore Entity Writes, and Firestore Entity Deletes. And then for storage
costs, you're going to want to look at
Firestore Stored Data. And this can give you
a pretty good idea of what your usage looks like. If you click on
History here, you'll see a list of your usage
costs over the last 90 days, and if you click on
any of these numbers, you'll get the exact
breakdown of your usage. And then maybe more
importantly over here in App Engine Settings,
you can set up a daily limit on spending. By setting this daily limit,
if Firestore costs ever get over this
threshold, then the API will start refusing requests. Now admittedly,
this isn't a thing you want to have happen
on a regular basis, and so my recommendation
here is set this to a point where
you're like, yeah, if I ever reach this
kind of level something's definitely wrong. Put the brakes on
until I can fix it. Then make it a habit of
coming back to this page every once in awhile. Look at higher costs
are doing historically, and then adjust your
limit to something that might be more accurate
based on your previous levels. I would keep in mind that if
you're on the Firebase Flame Plan, which is the fixed
price per month plan, then it's essentially
doing this already for you and setting these daily
quotas isn't necessary. Sure, it's nice to
look at the summary to see what your
actual usage is like, but you can kind of ignore
these costs since they're covered by your flat fee. But if you are on
the Blaze Plan, which is the pay as you go
plan, another step I strongly recommend is to
give yourself a monthly budget alert. If you go to the billing section
of the Google Cloud console, you have this option
here to create a budget. Basically this is your way
of telling Google, hey, this is kind of
the most I expect to spend per month on
all your Cloud Services, so I want you to
tell me if I start to get close to this limit. Now unlike the daily quota
limits in App Engine, this won't turn off anything
if you exceed your budget, but it will send you a
somewhat frantic-sounding email so that you can go and
check out your usage to see what's going on. So because of this, I'd
recommend setting your budget much closer to what
you might reasonably expect to see in a busy month. There's a lot less harm in
hitting a false positive here. Just remember, the App
Engine spending limit is a daily thing. The budget alerts
are a monthly thing. Don't get them mixed up. Also, these budget alerts cover
all Google Cloud Services, not just Firestore. So your cloud storage,
your Cloud Function costs are also included in there, too. Finally, another tool I would
keep an eye on is Stackdriver. It's a very cool little
monitoring service that can track your app's usage
of a number of different Cloud Services, both on
Google Cloud and AWS. You can set it up to present
you some really nice-looking graphs, and have
it send you alerts if things start to look off in
a number of different areas. For instance, we
recently added support for the real time database. So now Stackdriver can tell
me if the amount of data I'm sending in the
real time database goes over a certain
limit, or if it's getting too many or too few requests. It can let me know if my
Cloud Functions are taking too long to run,
and it can let me know if there's an unusual
increase in any of these stats. That said, there's not a lot you
can do with it yet with Cloud Firestore. But I wouldn't be surprised
if that changed in the future. I'm not promising anything. I have no idea what
their roadmap is. I'm just saying it wouldn't
surprise me, the same way, say, a second jar of jellybeans
wouldn't surprise me. Anyone? OK, fine. So there you go, kids,
just about everything you want to know,
and probably some stuff you didn't, about
pricing in Cloud Firestore. Like I said, refer to the docs
for all the latest pricing information, and keep me
on our lawyer's good side. He looks cute, but he can
actually be pretty vicious. And I'll see you soon on
another episode of Get to Know Cloud Firestore. Cheers. Ooh. Oh, I got buttered popcorn. That's a weird one. Wanna trade? [MUSIC PLAYING]
