SPEAKER 1: So we
spent a lot of time in our last video talking about
how you can make your Cloud Firestore Database even
more useful with a liberal sprinkling of Cloud Functions. We talked about why Cloud
Functions are useful, how they work under the hood,
and we built out one function to remove some really awful
language from our reviews. Ew, that's disgusting. But what else are
Cloud Functions good for beyond ridding
the world of fat-free dairy products? Let's dig into some
common patterns for using Cloud
Functions, and think about how you might want to
use them in your own apps. So I don't know if you
noticed, but I'm a big fan of lists of five on the series. Ah, remember those days? I had so much more
hair back then. Well, it just so happens that
today's episode also works nicely as a top five list. So let's start by talking about
my number one favorite use for Cloud Functions,
and that's using them as an alternate solution for
when your security rules start to get too complex. For example, imagine we're
working on a multiplayer chess game powered by Cloud Firestore. We'd probably have some kind
of games collection, where each document contains the
game's current state, a list of players, whose turn it is,
and maybe like a history of all of our players moves. So now when it's
a player's turn, we'd want to have
that client send up the player's new move, the
new state of the board, and maybe an updated
history list. But there's a lot of information
we can't entirely trust here. For starters, is this a game
that the player is actually participating in? And if so, is it their turn? And is the move a legal one? Are they really appending
their most recent move to the list of previous
moves or are they secretly changing the
history of the game? And most importantly, does
the state of the game board accurately reflect what
would have happened when our player makes that move? Now, in theory,
this is all stuff that we could handle through
a sufficiently complex set of security rules. But I am not a fan of overly
complex security rules. These things sit at the
frontline of your app, they're kind of hard to
debug, and you've got players actively trying to break them. So my general preference
is to keep these as simple as possible. So how can you do this? How can you keep your
security real simple while still allowing this
kind of complex behavior? Well, this is where
Cloud Functions can help. See, instead of having to
worry about having our clients muck with these games
state documents directly, my recommendation would
be to lock these down entirely so that no
client can modify them. This set of documents is
going to remain read-only. Instead, I'm going to
create a completely separate incoming moves collection. Now, when a player decides
to take a turn in their app, they can submit their
move by adding a document to this collection. And this document
would be really simple, consisting of nothing
more than the player ID, the game they're
looking to participate in, and the move that
they want to make. But because of that,
the security rules here can be very simple. We'll only let clients
create new documents where the player ID
equals their own user ID, and, well, that's
probably it, right? Everything else can
essentially be forbidden. Then, once this document has
been added to the pending moves collection, our Cloud
Function can take over. When it sees a new document,
it can check and see if this game is
active, if the player ID is, in fact,
one of the players this game, if it's
really their turn, and then, most
importantly, our function can do all the heavy
lifting of taking this move, making sure it's legal, and
calculating the final state of the board for us. Now, because this is all
happening server-side in a secure environment,
we know this is a function we
can trust, which means that Cloud Firestore
will let this function go ahead and alter the official
game state document, while keeping those documents
completely locked down for our suspicious-looking
clients. In the meantime, if we
assume that our clients still have real-time listeners set up
on these game state documents, they can still go ahead
and receive these updates as they're applied,
so this change will get reflected pretty
quickly in our clients. And as long as we're
here, you can still have your Cloud Function
do some nice things, like send a notification to
your opponent using, like, Firebase Cloud Messaging to
tell them that it's their turn. Now, the biggest
drawback here is that you don't get
all that nice latency compensation I was talking about
a couple of videos ago, right? Like, when you make a change to
that incoming moves collection, we can't update your
cache data right away the way we would if you were
writing directly to that game state document, which means that
you don't get to see the change reflected in your
official game state until the write has come
back from the server. It also means that
if you're offline, that pending change will be cued
up and sent when you come back online. And, until then, your game will
remain in its former state. But still, for something
like an online chess game, I think these are
reasonable trade-offs to make as far as the
user experience goes. Now, going back to our
restaurant review app, we'll probably want to update
a restaurant's average rating every time a new review
is written or edited. But again, I'd be kind
of nervous about having clients update the
restaurant document directly. Not only do you need to make
sure clients don't change fields they're not
supposed to, you have to make sure this new
calculated average is accurate and not some crazy
extreme value. And that's going to be kind of
hard to do with security rules alone. So again, this seems
like a good opportunity for a Cloud Function to do that
work behind the scenes for us. OK, let's move on to
a second common use case for Cloud Functions,
and that's keeping your denormalized data in sync. So going back to our
restaurant review app, let's suppose that when we
display reviews in our app, we'll want include a
little bit of information about the author, like
maybe their display name, and a link to a profile photo. Now in these
situations, in order to keep our NoSQL
queries nice and NoSQL-y, we would normalize
some of that data. Specifically, when a
user writes a review, we'd probably add their
name and profile picture to the review document. And all this is fine
and dandy, but what happens when a user changes,
say, the URL of their profile picture? Well, we not only need to change
it like up here in their user document, but also in every
review they've ever written. And yes, in theory, we could
have the client do this for us. But I'm not crazy
about that idea. For one thing, it kind of feels
weird to have our client do this much processing
on our database, you know, particularly
given how flaky mobile phone connections can be. Plus, all that work involves
extra data and battery costs for our users. And again, the security rules
needed to allow all of this could get a little messy. So this is another situation
where Cloud Functions might prove to be really useful. We can have a Cloud
Function set up to run whenever a user
document is edited. If it turns out that our user
is editing their name or profile picture, well great,
our Cloud Function can run a collection group query
against every review for which they're the author, and then
change the author URL field in that denormalized data. And as I noted in
an earlier video, this is something
that could be made more efficient with batch
writes, or slightly more atomic with a transaction. Or, heck, we could even
combine this pattern with the previous one,
and not let users directly edit their own user
documents at all. I mean, these
documents might contain other information we don't
want our users to mess with. Instead, they could
write a document to a completely separate
pending user changes collection, and then our Cloud
Function could take care of both editing
the original user document and changing all that
denormalized data. Now, there is one issue here
that we need to watch out for-- by default Cloud Functions
terminate after they've been running for one minute. And that might not be enough
time for your function to completely change
all the documents it needs to if you had a really,
really large database. Now, you can increase
this default time to nine minutes, which will be
adequate for most situations. But if you are running, say,
a massive database with, like, millions or billions of
records that all contain normalized data that
need to be updated, you could run up
against this limitation. And at that point,
you might need to consider other
options, like standing up a separate cloud-hosted server
and running the denormalization scripts from there. But even in those
situations, you could use a Cloud Function
to observe your database and kick off the script
on your dedicated server when it sees that a user
document has changed. Now, on that note, let's
move on to use case number three, which is using
Cloud Functions to perform occasional database maintenance. Now, around this time last
year, the Cloud Functions team added the ability
to run your scripts on a regularly occurring
schedule, basically like a cron job. Now, this can be used for
a lot of different tasks, like generating weekly summary
reports based on your data, but it's also a nice
opportunity keep your database neat and tidy. For instance, let's say we want
to save drafts of restaurant reviews written by our users. Seems like a good idea. But maybe we don't want to keep
these things around forever, right? Like, at some point
our database is going to be full of
half-abandoned reviews the way my life is full of
half-abandoned dreams. "The Distress Signal," a new
novel by Todd [INAUDIBLE].. Chapter one. Lucy always won. Ugh, I give up. (SINGING) Your face like a-- Ugh, not for me. Hello, and welcome to the
[? "Snack ?] [? Bowl" ?] our weekly podcast on
all things snack rel-- I'm sorry this is dumb. I just-- I can't. Oh, my god. That got dark. So we decide, hey, let's
remove any draft reviews that are older than, say, 45 days. But again, how do
we do something like this with a client? We could ask every client at
the beginning of the session to go and query every draft
review that the user has, and then delete ones that
are older than 45 days. But again, that just feels
odd to have your client doing that much work. And what are you going to do
about people who uninstall your app and never come back? You want to delete
their drafts, too, and they can't do that if their
client isn't using your app. So this is clearly
a good opportunity for a Cloud Function
to come in query all of your half-written
reviews and delete ones that haven't been modified
in the last 45 days. You can set it up to
run once a night or once every couple of nights. And just like that, your
database is nice and clean. And I'm sure there are
other places we could apply this pattern as well. Maybe we want to use
it to put old user accounts into an archive state
when we realize they haven't been accessed in six months. Or maybe we decide,
hey, we don't need to recalculate a
restaurant's average review score every time a
review is updated. We could get away with just
updating all of our restaurants once a week instead. Again, something we could
do with the Cloud Function-- although an operation
this big might run up against our nine-minute
timer, depending on how popular our app is. If we reach that point,
it might be worth having it update 1/7 of
our restaurants each day instead of doing the
whole thing once a week. OK, let's move on to
pattern number four. And this kind of gets into
something we haven't really talked about yet in
this series, which is that Cloud Firestore is
a fantastic NoSQL database. But for some of you out there,
it's really less of a database, and more like a super fancy
real-time querying and caching service built on top of
your legacy database. So yes, maybe for a restaurant
review app, or a chess game, or some brand-new app that
you're building from scratch, it might make sense to
have Cloud Firestore power all of your backend data. But if you're an
airline that's got a 25-year-old system running
on existing infrastructure, or a bank, maybe the idea
of completely wiping out your server with all of
your existing business logic and regulatory
compliance and all of that so you can start fresh with a
NoSQL database is impractical. It's understandable,
but you're also sitting there wishing
that you could build an app with real-time
updates, and off-line support, and latency compensation with
like an infinitely scaling back end, right? Well, this is where you might
need to think differently about Cloud Firestore. For a lot of
companies out there, particularly long-standing
established ones, Cloud Firestore
isn't the database that runs your entire
infrastructure, right? It's the sleek new
client-facing layer that lives on top of your
old infrastructure, which allows you to build
modern-looking apps. So imagine you are an
airline that has, like, flight information and prices
all stored in a decades-old SQL database hosted on your
own personal server rack. And maybe you want
to build an app that lets people search for flights,
and check flight status, and so on, but the idea of
exposing that database to millions of users
at once is frightening. Well, what you can do is take
that flight data, pricing information, and
whatever else you think your users will want to search
for, and copy that onto Cloud Firestore, essentially
as a read-only layer on top of your actual database. Seems like a good
solution, but how would you keep all
that data in sync? Well, there's a
few ways to do it. As you know, Cloud
Firestore has server SDKs that run in many
different languages, and it'd probably be pretty
easy to write server code on your original server
that updates Cloud Firestore directly whenever it
has to change some data. It's a good idea, although it
is a pretty tightly coupled system. Your code on this
other server now has to know exactly how
your data on this completely different system is organized. And if your Cloud
Firestore engineer decides to make some changes
without telling the other team, you're going to have a bad time. So in addition to
Cloud Functions triggering when users
write to your database, or create new accounts,
or what have you, Cloud Functions also
supports a service called Cloud Pub/Sub,
which is essentially a generic
message-passing service. So if you need to change
the price of a journey or update the
status of a flight, you can also go ahead and
create a Pub/Sub message, which essentially looks
like some JSON describing what's changed. And after you've done
that, you can set up a couple of Cloud Functions
to listen to these Pub/Sub messages. When a Cloud Function encounters
one of these messages, it can go ahead and update
the appropriate sections of your database as needed. Then, your original
database doesn't need to know how your other
database is structured. It just needs to publish
this somewhat generic-looking message. Now, while this does mean
adding another product to your workflow, it's
a pretty useful one. The Pub/Sub queue
scales really nicely, allowing for spiky
workloads that won't overload your system. It can be called
from services that aren't running on Google Cloud. And you can add some
fine-grained access control to your queue so services
only get access to the events that you want them to. This also means it can be useful
if you have lots of services, not just Cloud Firestore,
that might need access to this kind of data. And what about when
your users want to write back to your database? How would they do that? There's a number
of different ways. In some cases, like
purchasing an airline ticket, you're probably going to
need to have your clients talk to your original database. And they would do
that with direct rest calls to an intermediary
server, or whatever you've got set up on that legacy system. In other cases,
you might not need to communicate with your
original database at all. Like maybe your users want
to just take personal notes about their trips. For this user-only information,
you could probably just keep it on Cloud Firestore. Then, you get the advantages
of the client SDKs, like offline support,
which is probably important on an airplane,
and this information can just continue to
live on Cloud Firestore. But there will also be cases
where you want that information living in both systems, like
maybe a list of a user's favorite flights, for instance. In those cases, maybe you want
the convenience of making calls through the client SDK to
your Cloud Firestore database, but you also need a copy of
this information on your legacy server. Again, this is a place
where Cloud Functions could come in quite handy. As your function sees
that a new favorite has been added to a
user's list of trips, it could make an outside call
to your legacy system, which could then go ahead and
make whatever changes it needs in order to keep
the two data sets in sync. OK, finally, let's talk
about one more interesting use case for Cloud Functions. And that's basically using
Cloud Functions to build your own custom API on top of
your Cloud Firestore database, or whatever other infrastructure
you might be running. So we've talked a lot about
how Cloud Functions can trigger automatically based on events
happening in your project, like a document being
written to Cloud Firestore, or a user account being added
in Firebase [INAUDIBLE],, and so on. But they can also
be called directly. You can set up functions so
that you can call them directly over plain old HTTPS. And, yes, this includes
GET, PUT, POST requests, the whole gamut. Or is that ga-MUT? Gamut. And while that's useful,
Firebase Cloud Functions also comes with some pretty
nifty client libraries that simplify this process
even further using something known as a callable function. A callable function is,
again, to oversimplify a little bit, a wrapper
around HTTP calls, but one that reduces
a lot of the work you would normally need in order
to talk to a Cloud Function directly. For instance, if your
user is signed in, that information is sent
across and automatically verified by the
server so you can do things like perform
user-specific actions without having to, say, verify
any signed hashes on your own. And passing across data is
as easy as sending a JSON object along with your call. Similarly, returning
data is easy as well. You're generally just
going to pass back JSON objects, which your
clients can decode as needed. And if your function
encounters any problems, callable functions
do a pretty nice job of simplifying error
handling as well. All this means that you can
access these callable functions using code that looks
almost as if you're calling some other local client method,
no NS networking or volley code in sight. And that means you
can quickly build up your own application-specific
API using callable functions as a foundation. So where would you want to
use these callable functions? Well, let's consider
that chess example I had back in the
beginning of this video. If you think about it, when
we're writing that pending move document to the database,
all we're really doing is sending a message
to a Cloud Function. We're just recording that
message inside a document as a way of passing
that information over. But maybe we don't need
that intermediary step. We could just as easily directly
tell a Cloud Function, hey, this is the move that our
player wants to make in a game by using a callable function. Now, as in everything in life,
there are trade-offs here. On the plus side,
we're no longer adding all these document
writes to our database, and that saves us a few
pennies here and there. It's probably also a
little more efficient just talking to our
function directly. But the drawback is that
we've lost offline support. With the pending
move document, that could still be stored
locally on our device when the user is offline,
and then uploaded later when we come back online. But a callable function, that's
just going to fail immediately. And we need to add
our own custom logic to retry this call again
later when the user comes back online. Now, another good example
is to use Cloud Functions in conjunction with
Cloud Firestore to query our database in a way
that's difficult or impossible using native NoSQL techniques. Remember back in
episode five when we thought about
how we would want to store a list of our
user's favorite restaurants? One option we
explored early on was to just store an array of
restaurant IDs inside each user document. That felt pretty
natural, but there's no way to have Cloud
Firestore natively perform any kind of join that would
give us the restaurant document included in that array. So we decided the
NoSQL-y way of doing this would be to create a
completely separate collection of user favorite documents
where I could store the user ID, the restaurant ID, and
whatever information I might need about this restaurant
to populate a My Favorite Restaurant screen. And this is still a fine answer. But it's a lot of documents
with a lot of denormalized data to keep in sync. And if it turns out the View My
Favorite Restaurants page ends up being a feature that nobody
really uses all that much, maybe I haven't gotten the
best bang for my buck here. So another possibility
would be to go back to my original plan of just
having an array of restaurant IDs per user. And then, having a
callable Cloud Function do the work of
grabbing this array and reading it in,
and then fetching the individual
restaurant documents based on the content
of this array, and then sending all that
restaurant information back to my device. And, in fact, if
you want to see me do exactly that, I have a
whole other video series where you get to watch me fumble
around in JavaScript and build a function
just like this. It's linked in the
description below. Go check it out sometime. Of course, by relying on
a callable Cloud Function, I do lose most of the
benefits of making a call through the client SDK. For instance, I can no longer
make this call real-time. It's going to be a simple
one-time fetch call. And there's no
native caching, so I won't be able to look at
my favorite restaurants when I'm offline. And all that nice
latency compensation, where we can first show the
user their cached documents and then update them with
values from the server when they show up a second later,
well, that's gone, too. I have to wait for a complete
response from my Cloud Function before I can show any results-- at least not without building my
own cache, and offline support, and latency compensation. But if you think
about it, this might be a good minimum viable
product type of solution, so that I can quickly test out
the idea of showing a user's list of favorite restaurants. And then, you can see
whether this is a feature that users actually care about. At a later point,
if this ends up being a popular enough
feature that is worth going ahead and expanding
out this full denormalized collection, I can still
do that once I've decided it's worth doing. And there's tons of other
cases where a pattern like this might apply-- things like
performing OR queries across different fields, or
doing Greater Than or Less Than queries on one field
and sorting on another, or even combining
Cloud Firestore results with results from other
databases or services. You can bundle that up
in one easy-to-call API thanks to Cloud Functions
and callable functions. So hopefully this
has inspired you to take a closer look
at Cloud Functions and consider how
they might help you build more powerful,
consistent, or cleaner Firestore-powered apps. And then when you
do, tell us about it. Have you used Cloud Functions
in interesting ways? Let me know about
in the comments. I would be super interested
to hear what you did. Thanks for watching "Get
to Know Cloud Firestore." And now if you'll excuse me,
I am late for a chess game. Queen to f2. You know what? This game's crap. [MUSIC PLAYING]
