[MUSIC PLAYING] MARK DUCKWORTH: Hey, everybody. [APPLAUSE] Hey, I'm Mark Duckworth. I'm an engineer on the
Firestore SDK team. And I'm actually relatively new
to the team and the product. And one of the
things I've really enjoyed about getting
started is learning about the customers,
how you use Firestore, and the challenges you face. And one of the
challenges that I've seen is understanding the
Firestore SDK cache and how it can be used to your benefit. So that's what I'm here to
talk about-- using the cache, and using it to build more
efficient applications. So just take a step
back, if you're not familiar with Firestore,
well, Firestore is Firebase's NoSQL
document database. And with a NoSQL
document database, you can build lots of different
types of applications, but it's common to build
read-heavy applications. Think, for example,
maybe an e-commerce app with a product
catalog with thousands of items being read
thousands of times per day by different users. And that product catalog
doesn't get updated as often as it's read. Well, there's something
else that I've seen as a common consideration
when using Firestore, and that's how much it costs. And I don't think that's common. I don't think that's
unique to Firestore. I think it's common
across all software, but it's important
here in this context today, because with
Firestore, you're billed by the number
of reads and writes. And so if you're building
a read-heavy application, then you want to be mindful
of the number of reads against the server. So to explore these ideas,
I built an e-commerce app called FireSale. Let's check it out. You can see FireSale here on
my laptop and my mobile phone. And you can browse a catalog
of pens, stickers, and patches. Browse it on the
laptop or the phone. And I'm going to log
in on both devices so I share the
same shopping cart. And when I add items
to my shopping cart, you'll see that those
items are synchronized across the Firestore server. So I'm going to talk about
how I built FireSale. I have my application
running on both devices. It's using the Firebase SDK
and two different Firebase services-- Firebase Authentication
and Cloud Firestore. Now, I want to point out that
I built just one application, a responsive web app. But I could have built a
native iOS or native Android app using the Firebase
Android or iOS SDKs and gotten all the
same functionality you're going to see today. Let's dig in a little bit
into how I'm using Firestore. Well, I use an API
called getDocs, which I use to query my product
catalog from the Firestore backend. I use a second
API called addDocs when I want to add an
item to my shopping cart. My shopping cart is represented
by a collection in Firestore. And I add an item or add a
document to that shopping cart. And third, I use
onSnapshot to set up a listener to my shopping cart. So when a new item is
added to the shopping cart, my application is
notified, and I can update the UI with the
new item in the shopping cart. So with those three basic
APIs and some helpers, I built a pretty
basic e-commerce app. But I want to mention that
it also works offline. So how does it work offline? Well, actually, let's check
out offline functionality. So the first thing I want to do
is take my application offline on my laptop and add a
couple of items to my cart. When I add them, you can
see the items get added to the cart on the laptop. But they don't show up
on the cart on the phone until I bring my
laptop back online. So how does this work? Well, getDocs, addDoc, and
onSnapshot all work seamlessly offline thanks to the
Firestore SDK cache. Now, the cache keeps
copies of documents that have been queried
from the Firestore backend so they can be queried
again when offline. And it also keeps
mutations, changes to documents and
collections that were made while
offline that can be sent to the server when the
application comes back online. So the only other thing I've
done with my application is enabled persistence using
enableIndexDbPersistence. And that's just to get the
offline behavior that I want. So let me talk about enabling
persistence a little bit. With persistence, it causes
the data to stay in the cache so it can be queried offline. And it's not the
only way to have your data stay in the
cache for offline querying. But it was the easiest
way for my application. And also, persistence ensures
that the data is in the cache, even if your
application restarts, which can be really critical
in certain scenarios. And so I also want to
point out that persistence is enabled by default
in Android and iOS SDKs. But it's disabled by
default on the Web SDK, so you need to enable it,
or I did on my application. So I'm pretty excited about
what I built with Firebase. And I'm telling my brother
how powerful it is. And he says, well,
if it's so powerful, then does it cost a lot? And I decided I need to
come up with a cost model. So I think about what success
might look like for my app. And I say, maybe that's I have
10,000 monthly users viewing 25 items per catalog
page, loading three pages from my catalog,
and maybe refreshing those pages five times. And that comes out to a total
of 3,750,000 document reads. And I'm a little
concerned until I plug that amount of reads into
the Firebase pricing calculator and see that that's only
going to be $1.38 a month. But I, as an engineer,
I'm kind of thinking about the future where maybe
I achieve global scale. Maybe I have 10 million monthly
users and a product catalog that's 10 times that size,
where I could see bills that are in excess of $20,000 a month. So I still want to
see if I can reduce the number of billed document
reads in my application. So I have an insight,
which is basically that my product catalog
doesn't update very often. So I decide if I
update my application to pull the product catalog
from the Firestore server only once a day, and then
any other time I need to query the
product catalog, I can query it from
the local cache, I can reduce my bill of reads. So this updates my application. So now I'm only using getDocs,
which queries the server once a day or the product catalog,
and the product catalog is stored in cache. And then any other time I need
to query the product catalog, it will be queried
using getDocs from cache and not hit the server. So this updates my
billing read estimation, potentially reducing those five
page refreshes down to one, or saving me up to 3
million document reads. But I still have
a problem, which is each of my 10,000 users
are querying the full product catalog. And it's the same
product catalog for each of those users. So I want to see if I
can solve this problem. And I have another
insight, which is, well, I think that I can
query the product catalog only one time and share those
results across all users. And in fact, I can do this
with a built-in feature called bundles. So what are Firestore bundles? Well, Firestore bundles
are the packaged results of queries from Firestore. And typically, you use
this with common queries. But it allows an application
to load your query results from a bundle quickly
without querying the backend. And in this case,
it's also helping me save on billed reads. So how did I create a bundle
from my product catalog? Well, in this case, I need to
use a Firebase server or Admin SDK to run that same query
for my product catalog and get all the
documents, and then I can run the documents
through the bundling API to create this bundle-- that
icon you see on the left. And then with my bundle,
I can distribute it to one application, or 30
instances of my application, or 1,000 instances
of my application, and I've only queried the
Firebase server one time. So this changes my
application a little bit. It changes how data gets from
the Firestore backend down to the SDK and my application. So I have a process that
runs once a day to update that generated bundle file
for my product catalog, and puts that in Cloud Storage. And then when an application
starts or once a day, it fetches that bundle
from Cloud Storage, and then loads the bundle
into the Firestore SDK cache using loadBundle. And so from there, I can
query that product catalog out of the cache. So once again, this updates
my billed read estimation, reducing those 10,000
monthly reads down to 30 for a total
of 2,250 billed reads for a savings of over 99%. Now, I want to point out if
you are serving a bundle out of Cloud Storage,
then you'll still have to pay for egress on that
bundle from Cloud Storage. So I'm pretty
excited about that. However, I found another issue,
a blog from 2021, that says, basically if you
overload your SDK cache, it can slow down
querying from the cache, and potentially slow
down your application. So I want to investigate this
and see if I can fix that. And it turns out there's a
relatively new feature called client-side indexing, which
can be used to speed up queries against cached documents. Now, I'd like to point out
that client-side indexing is in preview or public beta. So you can use it,
but it may change. And so what is
client-side indexing? Well, client-side
indexing simply brings indexing to the SDK. Indexing is always
performed on all documents on all fields on the server. And it's new to the SDK. And if you're not
familiar with indexing, well, it's just a mechanism
the databases use to perform very efficient queries. Take, for example,
this table on the right represents an index
on the field category. If I get a query where
category equals pins, I can use this index to very
quickly find all the documents where category equals pins. So like I said, indexing
is automatically performed on the server. But in the SDK, you
have to configure it. And you configure it using
setIndexConfiguration. And you pass in a
configuration telling the SDK which fields to index. So after I've set
that up, when I want to query from cache using
something getDocs from cache, it's now going to
query very efficiently. So in summary, FireSale,
I built an e-commerce app with offline support
thanks to the SDK cache. And I've saved on read costs
by querying from the cache and using bundles. And I've sped up querying
from the cache using client-side indexing. So if you want to learn
more about these features, check out our online
documentation, or I'll be out at the
lounge right after this. And thanks for coming. And thanks to all my team
and everybody who helped build this stuff. [APPLAUSE] [MUSIC PLAYING]
