Observable Flutter: MongoDB & Realm

CRAIG LABENZ: Hello, everyone. Welcome to another episode of Observable Flutter. I'm your host Craig Labenz, and today we're going to be talking about MongoDB and their product Realm, which is a pretty darn cool backend database. Now before we get too far into it, I do want to remind everybody that this is Flutter. This is the Flutter community. And deep respect for each other is extremely important to me. I hope it is to you as well. We're operating under the Flutter code of conduct and the YouTube terms of service and code of conduct and all that thing. So let's really, really be kind to each other. So this week I'm very excited to be joined by Kasper Nielsen all the way from Denmark who is a senior software engineer at MongoDB and certified Realm expert. So let me bring Kasper in. Kasper, welcome to the show. KASPER NIELSEN: Thank you, Craig. Thank you for having me here. Looking forward. CRAIG LABENZ: Yeah, me too. So this-- the reason that you're here, specifically now as opposed to any other time, is that Realm has just released some updated support for Flutter, right? KASPER NIELSEN: Yeah, we went GA with the Flutter Realm SDK on the 7th of February. And that's why I'm here. CRAIG LABENZ: Nice. So the ink is still drying on the press release, I imagine. KASPER NIELSEN: Yeah. CRAIG LABENZ: Great, great. Cool. So I think there's going to be probably mixed kind of-- the audience is going to have a mixed amount of knowledge about what MongoDB and Realm even are. So maybe if you want to start with just a little primer on-- maybe even just first, MongoDB-- and kind of its strengths and maybe a pinch of its history, if you're interested. And then kind of Realm and how that builds on top of it, which I think is the right way to think about it. KASPER NIELSEN: Yeah. Yeah, so MongoDB is sort of a fairly successful SQL database. It's a document-based database. And the big driver of today for MongoDB is their Cloud offering called Atlas. And they acquired this little Copenhagen company called Realm back in 2018. And Realm's strength is its ability to synchronize data between devices and make it easy to use a local database on your device. You can think of-- when I introduce Realm to other developers, I typically compared it to SQLite. And that's sort of correct in the sense that SQLite is also an embedded database. They are both-- we are both frugal with resources, be it memory or CPU cycles. But they are also very different in that SQLite is a relational database supporting SQL. And we are an object-oriented database. And there's no sort of industry standard query language for those. So we have our own query language called Realm RQL. I think one of the strengths of Realm is that it has very good language integration across various languages. There are actually today, including all the SQL+ alpha SDK, there are nine different SDKs for various languages built on top of the Realm core, which is the C++ database in the bottom. And the Flutter and dot SDK is the latest to reach GA status. CRAIG LABENZ: Nice. So it was eight on February 6th, huh? KASPER NIELSEN: Sorry? CRAIG LABENZ: You said there was nine SDKs. KASPER NIELSEN: Yeah. CRAIG LABENZ: I was just making a joke that it was eight. KASPER NIELSEN: Ah, so sorry. Yeah. Yeah, well nine is counting the alpha SDK, which we have a C++ alpha SDK. CRAIG LABENZ: Got it. Got it. KASPER NIELSEN: We have been alpha for a year about now. CRAIG LABENZ: OK, got it. Got it. Yeah, so you mentioned that MongoDB had acquired Realm in 2018. I remember I was using Realm before I found Flutter when I was working on a project in just raw Swift and Xcode. I used Realm and loved it. And then I found Flutter. And my first disappointment with Flutter was that it didn't, at the time, have Realm support. KASPER NIELSEN: Cool. CRAIG LABENZ: Because, yeah, there was just missing tech. FFI wasn't where it needed to be. So it's pretty-- it's very full circle for me to be sitting here talking about how to use Realm in Flutter. Because it is really darn cool. There's a couple of questions we have in the chat that I think we might as well just hit before they get too stale here. Baturay says, how is MongoDB different from CouchbaseDB? KASPER NIELSEN: Yeah. CRAIG LABENZ: And do you know much about Couchbase? KASPER NIELSEN: No. I think that it's based on top-- it's built on top of SQLite. So one thing that it differs on-- Realm has its own core database. But it's in the same space. CRAIG LABENZ: Yeah, because I think, Couchbase is also a noSQL store. So back-- I think Mongo is slightly older. Don't quote me on that. But yeah, back in the late 2000-- approaching the 2010 era, noSQL databases were starting to catch on. Mongo is the first that I remember hearing about. But then I think some other flavors spun up. Potentially they were being developed in parallel. Maybe it was inspired by, I don't know. Yeah, pretty similar. But then you mentioned that Mongo's kind of core thing is Realm. Couchbased may have a similar product like Realm? Or it might not. I don't even know. KASPER NIELSEN: Yeah, maybe I'm wrong. I think at least it was cut space lite that sort of built on top of-- similar to how like Firebase also built on top of SQLite. At least in that respect we're different-- that we have a new code all the way down. CRAIG LABENZ: Got it. We're got another question here. Is MongoDB Realm driver stable to use in Flutter? I think it is. KASPER NIELSEN: Yeah, so if I glean at this question, it mentions the name driver. So we don't have an official Dart driver for MongoDB yet. So what we are talking about today is the Realm SDK for the local database-- the Realm. So there's two sides to this. There's the local database, Realm, which is an embedded database that you can use that has nice language integration. It sort of excels at these reactive architectures where you can listen to changes from the database in great detail. And then there are the Atlas app service suite, which sort of mimics a little what you got with Firebase, where you have one of the apps services is Device Sync-- what we call Atlas Device Sync. And those two components will form the basis of how you think data across devices in an offline first manner that you would just write locally to your own database. You'll set it up so that synchronization happens with Atlas Device Sync in the background. And you don't have to write any network code. You don't have to write any conflict resolution code. We'll just handle it for you. And that is the value proposition of Realm and Atlas Device Sync. CRAIG LABENZ: Yeah, and I think Realm and Atlas Device Sync-- but for now I'll just say Realm, if that's accurate enough. Realm is uniquely strong at this-- the offline first support. I think Realm is basically best in class on that. That's my understanding. I imagine if you're aware of someone else, you maybe don't feel that compelled to cite them right now. [LAUGHS] KASPER NIELSEN: No, I think sort of-- you're with Google and you have this big product Firebase, which sort of sits in the same space. But I think when it comes to offline first aspects, I think Realm is more offline first than Firebase is. For instance, you have this aggregate functions and such that doesn't really translate well to offline. And if you want to-- if you open a transaction in Firebase it's actually an online thing. And so you-- we differ in that sense. Transaction in Realm is just a little thing. CRAIG LABENZ: Yeah, well I think it might be time to get into it. So you're going to drive today, Kasper. And I'm going to offer commentary and ask questions. And whenever you're ready, yeah, I think we could do it. KASPER NIELSEN: Yeah, so you're watching my screen now. We have three emulators here. So the thing I want to do today-- try to repeat today-- is that I want to create an app that synchronizes a list of items across multiple devices. And we're going to use Flutter. We're going to use Realm. And we're going to use Atlas Device Sync. You all know what Flutter is I've just briefly explained Realm. This is Atlas app Device Sync set up. So I'm just going to start here from scratch. And I'm not going to-- I'm just going to use flutter create to start out a new app. And I'm going to use the skeleton template. It's already setting up a list view for me and sort of creating a better scaffold for an app, including some routing and localization if we were to use that, et cetera. Let's enable a few platforms here-- ios, macos. I want to name one. And then we have to come up with a name. I usually call this listy. Should I call this Craig's list? [LAUGHTER] That would probably be inappropriate. CRAIG LABENZ: That is quite a good joke. I would usually refer to that as my list. KASPER NIELSEN: [LAUGHS] CRAIG LABENZ: I got this off my list. And I let people be confused for a second. KASPER NIELSEN: So just going to rearrange my windows hear a little. Sorry. So now we have-- CRAIG LABENZ: So while you're doing that-- KASPER NIELSEN: Yep. CRAIG LABENZ: Oh, yeah, while you're doing that, I was just thinking all of our talk of this being offline first and whatnot-- but it's, of course, not offline only. And we're going to ultimately deploy this to GCP today, right? KASPER NIELSEN: Yeah. I think that would be the appropriate thing to do. You could also deploy to AWS and Azure, but you'd probably kick me out of your stream. CRAIG LABENZ: [LAUGHS] KASPER NIELSEN: So I'm going to use GCP today. CRAIG LABENZ: I'm not so harsh, but it is extra credit for using GCP for sure. KASPER NIELSEN: This is actually one of the benefits of using a service like MongoDB Atlas-- is that we are sort of Cloud agnostic. We run on all the major Clouds and even a lot of the smaller ones. CRAIG LABENZ: One thought here real quick. If you're interested in it, it might improve the stream a little bit to kind of hide the File Explorer and make the text-- the code font size a little bigger. KASPER NIELSEN: OK. Oh, that was the most. CRAIG LABENZ: Yeah. KASPER NIELSEN: The thing is, I would like to also have the device here sitting on the left. CRAIG LABENZ: No, I think this is good. KASPER NIELSEN: So, yeah. CRAIG LABENZ: So how do we get started? KASPER NIELSEN: Yeah, how do we get started? So I'm just going to choose a device here. CRAIG LABENZ: This is one area where the enhance trick does work. And the way these AIs are getting, though, it's going to be Command plus on a fuzzy image soon. And it's just going to be clear. KASPER NIELSEN: I think I need to restart my shield code here. So just-- CRAIG LABENZ: Yeah, what issue are you having? KASPER NIELSEN: So it didn't pick up the emulator. So here it is. CRAIG LABENZ: Oh, yeah, yeah, yeah. KASPER NIELSEN: So I have a Pro Maxi on the left. Let's run it on that. So right now I'm just running the skeleton app that's generated with flutter create. I do hope to do a Mason brick in Realm-- start a Mason brick eventually. CRAIG LABENZ: Oh, nice. KASPER NIELSEN: You could get all this up and running just by one click. But I thought when preparing this talk that most people would probably be familiar with flutter create. And since there are only two, really, app templates there-- either the Counter app or the Skeleton app, I thought of people are fairly familiar with this app. CRAIG LABENZ: Yep, yep. Yeah, I think-- and if folks aren't familiar with the code, I mean, that's honestly OK for this demonstration. But if you want, you could maybe just click through-- there's what? Three screens on this? KASPER NIELSEN: Yeah, there's like nothing. There's this list view. And you can go into Item Details. I'm not going to spend much time there. And then there's this little cog in the right corner where you can go to Settings and change the theme. And it'll rebuild the-- or re-render the widget suite with the new colors. So this-- what I want to do today is, I want to add the ability to persist these items in a local database. And I want to add the ability to add and remove items. And once we've done that, I will enable the app to synchronize between devices. CRAIG LABENZ: Masterfully answered. KASPER NIELSEN: Thank you. CRAIG LABENZ: I clicked the wrong button. OK, all right. KASPER NIELSEN: I can feel the-- I can feel my hands not working here. Sorry, that's not what I want. I want to use-- go to the sample item class. OK. So the sample item, which is being displayed here on the left in the app-- that is what we want to persist. And we want to use Realm for it. So obviously, one of the first things we need to do is just move that. Sorry, I'll just move this here. We want to add the Realm package to our project. And I'm also going to add the collection package. And that's mostly just for one extension method called last on now. CRAIG LABENZ: Nice. KASPER NIELSEN: There we are. And then, the way you hook it up, you want to specify what classes you want to store in the database. And you do that by annotating your-- I am really feeling the shakes here. With Realm model-- the Cloud. You're going to annotate the classes you want to persist with Realm model. And I've already done a mistake here. But I'll just illustrate how it looks when you use it. So the next thing we want to do is we want to pick up these annotated classes. And we want to generate a bit of binding code that sort of glues these into the Realm database. And to do that, we do like you always do in Flutter. We run a built runner too. The only thing is, we sort of made it available from within the Realm package. So you can just use flutter pub run realm generate. And I'm going to add this watch flag, which will just keep it running. So when I hit Save, it will regenerate these little bit of bindings. CRAIG LABENZ: Nice. That's a really nice touch to put that in the library itself. I don't think many packages do that. KASPER NIELSEN: No, and you don't have to. You can just, if you prefer-- we want to make the onboarding really easy. So by doing it like this, you just have to add the Realm package, and you're up and running. Otherwise, you have a bit more setup to do. Yeah Yeah, so it ran. It already is complaining, saying there was-- I already introduced an error. It's saying that we are missing a prefix on the Realm model name and pointing out that this is actually a super cool library you released there with this ability to annotate the code I think. So on line 4, Cloud sample item, we're missing a prefix. You should align the class name to match the prefix either underscore or dollar sign. So let's return and look at that. So the reason we do that is that we actually want to generate a class called sample item. So this Realm model prototype cannot be called sample item. So I'm just going to add this underscore. Typically, I prefer to use underscore, because it'll make the model private, so you don't use it somewhere you shouldn't. But say you want a reference between files, between models-- you have a link from one Realm model to another or a list of Realm models that are defined in another file. You'd need to use dollar sign. So there's a lot of stuff here that sort of doesn't compile anymore. That's a constructor. It's not named correctly. Just remove that, because the generator will generate a constructive force. And also, the field here is not initialized. It's not knowable. So it needs to be initialized. So the way we do that-- it's a little bit inelegant. But we just mark this as late. Really eagerly awaiting static meter program. CRAIG LABENZ: Ha, indeed. KASPER NIELSEN: Yeah, and the idea is sort of to change this once that lands. But anyway-- so if we go here, what's the generator running in the background? It has actually succeeded. We're getting this warning from source_gen. We're using-- this is combining builder from source_gen to combine our output with other generators that we need to-- CRAIG LABENZ: Add the part file? KASPER NIELSEN: Exactly. Somebody needs to include the generator code. CRAIG LABENZ: And it's going to have to be you, Kasper. I don't know who else is going to do it. KASPER NIELSEN: It has to be me. Yeah, OK. So, ah! Suddenly the little red squiggle disappeared. And so how does this-- actually, I'm a little annoyed that my-- so this activity part is missing-- there it was-- so it generated this file sample_item.g. And we can see that it generated the class and license. It's extending our Realm model prototype. And it's overloading the get instead of the ID field. There's some magic that ties it into the database. And there's some other stuff you can see later. But there's the ability to listen to changes and sort of get the schema of this class. CRAIG LABENZ: Very nice. KASPER NIELSEN: And that's sort of about it-- what it takes to make a class that can be persisted. So what I've done so far is, I've just made it so that sample item can be persisted. I'm not actually storing anything in any database or even opening a connection to the database let alone reading any items from the database. So that would be the next thing that we'd have to look at. I'll just start by looking at what doesn't compile. So there's this sample list view. Yeah, there was this-- this is a very simple app, right? So there's just a const constructed list of sample items. CRAIG LABENZ: That's not very database compatible. [LAUGHS] KASPER NIELSEN: I'm just going to say the sample list items will require that we path in the lists. And that will probably make up-- yeah, doesn't leave any out. So down here somewhere, we're creating a sample list item view. And for now, we're just going to path in-- OK, it was called-- what was it named? I'm just going to path in an empty list. So what I expect to happen now is that, I can restart this app. It will show an even less interesting interface with no sample items. And since we didn't persist the theme, it turns-- it goes back to standard, which is-- CRAIG LABENZ: Yep. KASPER NIELSEN: Right. OK. CRAIG LABENZ: All right, this is all making sense so far. KASPER NIELSEN: Yeah, that's good. So now we actually need to open the database and start reading from it. And-- oops, sorry. I'm feeling I can't type in front of an audience. I get [INAUDIBLE],, not gonna lie. CRAIG LABENZ: That's a universal experience. You're not alone. KASPER NIELSEN: I hope to be better at it one day. OK, in order to-- so the database called Realm in Realm [INAUDIBLE].. So we're going to open a Realm just constructing a new Realm class. And that takes a configuration. And there's a number of configurations. And the really interesting one is Flexible Sync, which is going to allow us to synchronize data between devices. But before we get there, I'll just use a local database. And it takes just one requirement-- argument-- it's called schema object, which is going to tell the database-- sorry. Sorry, that's not what I want. I wanted to write sample item schema-- how to hook up what objects to expect in the database and that schema. So this is what you want to write when you open the Realm. I'm just going to leave out this part, sort of deliberately introduce an error. Because we see this happen-- that people sometimes forget to add the schema, so now I want to show how that looks. So we'll get back to that. Now obviously, we need to use this Realm for something. So I'm just going to open a-- I'll make it query that will give us all items in the database. The query method, you could use. But there's also a convenience that's just called all. And we just want all sample items like that. Sounds like you have question. CRAIG LABENZ: Oh, yeah, I was just going to ask a question about what was returned by that all method. But I might be jumping the gun. KASPER NIELSEN: We can hollow it out here. It's a Realm results. So when you see stuff like this, you might think, this might be expensive. What if this database contains like 20 million items? Are we just going to fetch all of them here? That's not what's happening. CRAIG LABENZ: That would be bad. KASPER NIELSEN: Yeah, that would be bad. All we're really returning is-- this Realm result is a cursor or pointer into the database-- that once you start accessing this result, then you're going to fetch object from the database. And say you're accessing index 2,758. It's only going to access, retrieve, that particular object. And only-- actually not going to retrieve all of the objects. Just going to retrieve a pointer to it. So that if you actually access the property on it, then it's going to access. So everything is very lazy with Realm. And that also means that-- another thing is that, say the database is updated. And there's some transaction rights, maybe in another processor, another isolator, whatever-- another sample item is written to the database. Then this-- all items will automatically reflect this. You don't have to rerun this query. When we say that Realm objects or Realm entities alive, that's what we mean-- that it will automatically reflect this. So I'm only going to set this query up once here, very early in main. And we're not going to-- from here on, we're just going to read it. And it will, despite us doing writes to the database, it'll just automatically reflect that. KASPER NIELSEN: Nice. Yeah, and I think for a lot of folks, seeing this kind of code in the main method-- this is before run app has even been called-- is a little-- it feels unusual. But to just really hammer home what you're saying, no query has been executed yet. This variable-- all items-- is just like the potential for a query. But nothing has been loaded yet. No connection has been made. It's not like there's an open socket just waiting. Nothing has happened. There might have been a socket open by line 10. But not by line 11, is that right? KASPER NIELSEN: Yeah, it's-- so right now, since we're doing a little database-- CRAIG LABENZ: A local, so there'd be no socket at all. Yeah. KASPER NIELSEN: Exactly. But yeah, crazy in that sense-- I see what you-- I wouldn't even be concerned doing it inside a build method. They are super fast. They just return this-- the money code to the potential to fetch data. And also, sort of fetching a property on a Realm object-- it's not like it's a microsecond kind of thing-- less than a microsecond kind of thing. So it's-- you shouldn't be too worried about the performance. CRAIG LABENZ: Got it. Nice. KASPER NIELSEN: Everything's [INAUDIBLE].. Actually, I was working with-- helping with an issue on-- there was this guy who had problems making a list view perform. So it performed very well for him. But when he added the scroll path around his list view, suddenly it didn't work so well. And he was sort of-- he was thinking there was a Realm issue. But really what it is, is that if you have a list view that is inside a scroll, scroll, scroll, and you don't specify item extent, then how does that scrollbar scroll [INAUDIBLE] calculated? It basically has to go and measure every item that could potentially be previewed by this. And if you have 20 million, that's going to take a long time. So you can make a very simple test that doesn't even involve Realm where you just set item count to like a huge number-- 4 billion or something-- and then Flutter will disintegrate. So, very important. It's very important to set item extent if you have very large lists that you want to scroll through really quickly. But doing that, you could just scroll through 50 million items like butter through a knife-- knife through a butter. Yeah, maybe we can return to that later. So now we've got this great setup. We want to use it for something. So I'm just going into here-- my app-- and we're going to path in this RealmResults of sample items-- called it items. Have that in the constructor. And now we can use that down here in the sample list view. I'm just going to path on the items. It's not going to fly, because this can't be-- and also the type is not correct. So it's expecting a list of sample items. And what we give it is the RealmResults. CRAIG LABENZ: We haven't given it the list of sample items yet. KASPER NIELSEN: Yeah, and RealmResults actually implements Iterable. We did consider whether it should implement list. The thing about lists in Dart is that they're mutable. You don't have a read only list. You do have unmodified list view, but it actually implements lists. Yeah, so what we chose to do was, we implement Iterable. But we do support-- so I couldn't make this red squiggle on the other file go away by making this Iterable. But then I would introduce an issue down here where we, actually, in the list view builder, access the items by index. CRAIG LABENZ: So the query result-- that does implement indexing? KASPER NIELSEN: Yeah, so I'll just-- even though it implements Iterable. But it also has an indexer, so this will compile again. So now we sort of hooked it up so that we just need to, of course, path the items that we just required here. So this should sort of put us in a green state again. So I could do a hot reload now-- a hot restart. And we would get a few tries. Oh, this is-- yeah, sometimes you have to, depending on what coachings you have, you need to do a hot restart. CRAIG LABENZ: Oh, yeah. So what did we add here that made-- that required this hot restart? KASPER NIELSEN: So the-- so in hot restart, my point is, hot restart won't even be enough right now. You see, we have this fail to load dynamic library. And the thing is, when I deployed this app, I just ran Flutter create, and then I deployed the app, and then later I added Realm. But Realm is a package that is a plugin. So it has a native component to it. And though-- as mighty as hot reload and hot restart are, they do not bring native code onto the device when you run them. CRAIG LABENZ: And the reason for that is that it's Dart's just in time or its JIT compiler that allows for hot restart. And just most of the languages don't have that capability. It's actually one of the reasons people ask, why did Flutter choose Dart? The compiler capabilities of Dart were one of the important reasons. If you love hot restart, you love Dart. KASPER NIELSEN: And I truly love hot restart. [LAUGHS] CRAIG LABENZ: Yeah, and hot reload. Yeah, yeah. KASPER NIELSEN: And hot reload, yeah, even more. CRAIG LABENZ: But here we've got to do the full big daddy restart. KASPER NIELSEN: That's the thing. We just-- why does go say something? I don't know. We do a lot of go coding in Mongo as well. So did I start debugging yet? So what do we expect to see now? So I actually expect to see another error. You might recall that I left out this schema from when I opened the Realm. And we'll soon see Realm complain about that. CRAIG LABENZ: We had that predicted in the chat. It's the error caused by the lack of parameters when initializing Realm. We were getting a different error. And we were approaching this error. So good eye, Bruno. [LAUGHS] KASPER NIELSEN: Here it is, Realm error. Object type SampleItem not configured in the current Realm schema. Add type SampleItem to your config before opening the Realm. And if you see errors along this line, it's quite easy to get into-- to forget to add. So you add more and more Realm objects, and suddenly it won't open. We've seen people sort of struggle a little with that. But you quickly get used to it. And eventually-- because we know a lot about the graph-- we know that, say, if SampleItem were to refer to another Realm object, we already know that. So we could use the transitive closure of the graph to decide what schemas to include. Well, that's for the future. We don't do it yet. Anyway, at least we should be able to start now. Still-- there are no items in the database. But I'm claiming that we are now reading items of the database and showing the fact that the database is empty. CRAIG LABENZ: Claiming that you're reading an empty database is an easy claim to make. KASPER NIELSEN: Yeah, that's easy to make. [LAUGHTER] It is. CRAIG LABENZ: So let's write something to it. KASPER NIELSEN: Let's do that. And so we want to create stuff in the items. And I'm just the-- we could just do it somewhere down below. But I'm just going to extract that little bit of logic out into a separate class called List block. It's Business Logic component. I'm not going to use the block package today. So it's more of the concept. So I'm just going to hoist this Realm list out of the sample item list view. And initialize the constructor, and then we're going to have a new method called addNewItem that will conveniently populate our database. And a little more things to do here. Need to have a block, and we're going to initialize that from the constructor. It's nothing fancy here. Just need to read the items off the block. OK. And there's probably somebody complaining that we're not pathing in, yeah. So now it's no longer items. But we need to path in a block, which will wrap those items. CRAIG LABENZ: Nice, very nice. KASPER NIELSEN: Yeah. Let's change the-- I changed the constructor writes. So that is a static code chain, so I need to do a hot restart. CRAIG LABENZ: As opposed to the hot reload. KASPER NIELSEN: Exactly. When we say, we only do hot reload. And so I needed to do a hot restart. OK. Obviously, we still don't have any database-- anything in that database. We need to hook up this addNewItem, and we also need to actually implement it. So let's start by hooking it up. This is just a standard Material app. So we have a scaffold here. Shows-- I mean, we can add a floating action button. So I think we should do that. And that needs something onPressed and it should probably also have a child. CRAIG LABENZ: The old icon button. KASPER NIELSEN: Some kind of icon there. CRAIG LABENZ: You could just take the icon. KASPER NIELSEN: Sorry? CRAIG LABENZ: I just haven't thought about a floating action button in actually a really long time. I was trying to remember what goes in it. KASPER NIELSEN: Oh, yeah, I think you can put all kinds of stuff into it. So I just say that-- CRAIG LABENZ: So I'm guessing you're going to call add item in this onPressed method. KASPER NIELSEN: That's true. You've got a floating action button. Nothing's hooked up, but onPressed should, of course, do something. We're just going to hook it up to addNewItem, which doesn't do anything yet. So we need to implement this. So what does it take to actually store a sample item in the database? Well, first we need a handle on the database. So I'm just going to-- CRAIG LABENZ: Yeah, I was wondering about that. KASPER NIELSEN: Yep. It's called underscore Realm. And let's not worry about where we get it from right now. But once we have it, we could open the write transaction. And whenever you do changes to Realm object, you need to be within the Realm and within a write transaction. So a lot of the idea of Realm is that it should almost feel like you're not working with a database. It's just objects that you can traverse. But on the other hand, there is also sort of database specific functionality that's not readily available in any object-oriented language such as transactions. So it's sort of a balance. And the thing is that, we want transactions, so we need to open them. And once we're in the write transaction, we can insert into it. We just call it add. It's just a shorter word for insert. And we can insert any Realm object and SampleItem-- sorry-- SampleItem being one of them. And if we remember, there was this ID property. It's just an integer. And the constructor that was generated by the build runner tool is requiring that we path that. Sorry. So we just-- the idea is that, it should just increment. This is where I'm going to use that one extension method I wanted from collection. CRAIG LABENZ: It'll last you all night. KASPER NIELSEN: And if-- the thing is, right now, the database is empty. So last null will return null. And if that's the case, we just start with 0. So 1 plus 0 gets us 1. And after that, it'll be 1 plus whatever the last ID value was. CRAIG LABENZ: Ooh, now I have a question here, which the audience also has. And I hope I don't put you on the spot too much. But MD and I are wondering, is there an auto increment? KASPER NIELSEN: Nope. CRAIG LABENZ: Interesting. KASPER NIELSEN: Yep, but I hope that this sort of showed that it's-- you can live without it. But we do not have auto implement. CRAIG LABENZ: So would you then advise having string IDs, essentially? Because the numbers are just not going to be reliable. KASPER NIELSEN: No. I would-- in that sort of thing-- because in the end, this auto increment-- what does auto incrementing means in a distributed system? So what we'd recommend is that you use some ID that you can-- that will be unique, that's generated on the device, that will be unique to the device. So you want to use something like NUUID or, as you'll see in a minute, we have something called an object ID that's a MongoDB concept, which you would use where you might use a universal unique ID. It's a little less. It's just 12 bytes compared to 16. And it's actually more similar to a timestamp. It is basically a timestamp plus some entropy that's based on the process and then a counter value. But in a distributed system, that's a good candidate for a primary key, because you can create it locally without consulting any other peers in your system and with high probability that it will be unique. CRAIG LABENZ: Where, by high probability, you mean astronomically high probability. KASPER NIELSEN: Astronomically high probability. But no guarantees. CRAIG LABENZ: Indeed. OK, so I do have one other question. You mentioned without consulting other parties in your system, is there just-- I presume there's an increment transaction? If I know I have an integer field, can I say, add 10 to it and then I get the result? KASPER NIELSEN: And actually, you can't in Flutter yet. But Flutter does have the concept of a Realm integer. And it's just one of those things that didn't quite make it into the DA. It'll be there soon. CRAIG LABENZ: It'll be there soon? OK. KASPER NIELSEN: I think there's three data types that we really need to add. One of them is Maps. We have List and we have Set but we don't have Maps yet. People are waiting for that. The other one is that-- sorry? CRAIG LABENZ: Sorry, keep going. KASPER NIELSEN: The next one is decimal128, which is sort of-- it's used often with financial systems. And the last one is this Realm integer. Maybe we could talk about that a little. So the default conflict resolution strategy of Realm is last writer wins, meaning that basically, if you have two updates happening concurrently on two devices-- say one is updating the integer to two and the other one is updating the integer to three. What should be the final state? Should it be two? Should it be three? Should it be 2.5? And any sort of distributed system that sort of claims to be of a first-- has to solve this for you. And the way that Realm and Atlas Device Sync solves this is that last writer wins. So we assign a timestamp to each of these updates. And the last timestamp, the update with the latest timestamp, will win. And that's actually maybe simplifying it a little. Because timestamps in a distributed system-- CRAIG LABENZ: Are also unclear. KASPER NIELSEN: Clocks are not-- CRAIG LABENZ: Magical? KASPER NIELSEN: --guaranteed to be in sync. So academia sort of over time came up. I think the most famous one is the Lamport clocks that helps you organize the order of events. CRAIG LABENZ: Yeah, now this is-- oh, sorry. KASPER NIELSEN: Yeah, but Lamport clocks is only getting you that far. If you want to do even better, you start using vector clocks. That's at the cost of storing some more state. And that's what Realm does. We use vector clocks to auto all updates, and the last one will win. But say you have a counter. Say you have counter. Say you have this-- actually, I think this is one of the things that the Firebase struggles with a little bit. Say you have-- you want to add a rating to a product. I want to give it four stars. And we want to have an average. So we want to divide the ratings by the number of ratings-- the sum of the ratings by the number of ratings, right? So say, in Firebase what you'd do, you would add your rating. And then either you would open a transaction and update an average within that transaction at the same time as you added the rating-- which would require you to be online-- or you would use a Cloud function to sort of monitor that a rating was added on the backend. Once that happened, you would update the average. And then that would be synced back to the devices. But there are two problems with it. Either you can't make ratings if you're not online, or you can make ratings, but you won't actually see the average until you get online. So you can't even see your own rating. The way you would handle this in Realm is that you would have two Realm integers. Because Realm integers, as you said, just tracks the deltas. It's not last writer wins. It also, if one added one and another added four, it'll convert to, altogether, five was added. And this makes it super easy to make this kind of functionality. You can just take the sum of all the ratings and divide by the number of all ratings. And that will just work even-- so if you haven't seen seven ratings from your peers, your average will just be different. But you will still have an average that includes your own rating. CRAIG LABENZ: Oh, that's super interesting. I want to try to describe back to you what I think you're saying. Essentially, have two realm integers. And let's say five people have supplied ratings. And they each gave it two stars. So we'd have two numbers. One of them would be five, which is the number of people who have supplied a rating. And the next rating would be-- or the next number would be 10, which is the total number of stars. And so then at any given time, when we want to calculate that average, we'd do the division. And we'd see that the rating is just two stars, because they all gave that rating. And then that was what would allow someone who's offline and gives it five stars to see the average rating go up, because suddenly, the numbers become 15 and six or whatever numbers I'm using in this example. KASPER NIELSEN: And also, if-- say, the guy who gives five stars-- he has only seen the ratings from two of his peers. So that would be 2 plus 2, and then his five star. That'll be 9 divided by 3. So the average will be three, which is not-- so he can see that his own rating is reflected somehow in the average even though he may not be online yet. So eventually when he comes on and everybody comes online and all the other ratings tick in, you have five twos and one five. You have 15. And you can divide that by six. I can't do that in my head. But that will be the true average. Everybody will converge to that as they see each other's ratings. And it's not like Realm-- it's not a peer to peer system. We are hub and spoke. So everything sort of goes through the-- CRAIG LABENZ: Through the mothership, so to speak. KASPER NIELSEN: Yeah. And that's-- it's not like the algorithm would actually work peer to peer. But the thing is, when you use vector clocks, you have to keep track of the last seen timestamps from each of your peers. And if you do that in an incomplete graph, you basically have in faulty state to keep track of. Whereas, if you do it in hub and spoke, you'd just have two times the number of nodes. That got a little technical. CRAIG LABENZ: No, it's good. So I do want to share one idea for an auto increment. If you really want an auto increment-- and you can't do this yet, because Realm object has not been ported to the Flutter SDK for Realm. But it's coming, so you could do it soon. And there's caveats on this, but you could have a single integer on the server. And you open a transaction and increment that integer when you want to write a new object. And then once it increments, you know that the number that it just incremented to is your primary key. It can't be anyone else's primary key because of the magic of transactions. Now, I said there's caveats, because if you're constantly writing to this table, all those writes will be queuing around this transaction that might be being opened from tons and tons of devices. So if it's a heavily written-to to table, I would not do this. But if it's-- KASPER NIELSEN: Are you talking about Firebase now or are you sort of speculating on-- CRAIG LABENZ: A way someone could kind of roll their own auto increment on Mongo. KASPER NIELSEN: Yeah, no, that won't actually work, Craig. Because the thing is, you're just sort of announcing that I'm incrementing it one. And locally I see, OK-- say the guy who was-- CRAIG LABENZ: If I'm offline-- KASPER NIELSEN: Yeah, so I think that the value is isn't online. So we're very much offline first. Transaction just happens locally. And all conflict resolutions happen afterwards. So it's very difficult to do these automatic alter increment with the device thing. That's sort of the cost of being truly offline. That's where we differ very much from Firebase. Because when you open a transaction, you're actually sort of going all the way. CRAIG LABENZ: You're committing it immediately. I didn't think about the fact that that scheme-- wow, so being as offline first as Realm is is actually just like conceptually, philosophically incompatible with auto increments. It's probably not going to ever happen? KASPER NIELSEN: It's not going to ever happen, I think. I don't see how it happens. CRAIG LABENZ: Yeah, OK, that makes more sense. That's really clarifying. KASPER NIELSEN: So what you want is, you want these random primary keys. And you could use something like NUUID. I recommend Optic-IDs. The nice thing about Optic-ID is if you-- since they are more or less timestamped-- they're not timestamped, but they are more or less timestamped-- it means that if you order them by-- if you order a set of Optic-IDs, they will more or less be ordered in the order they were created, assuming that the clocks are somewhat reasonably in sync. And that has benefits when it comes to locality and such. Anyway, let's return to the sample here. CRAIG LABENZ: Yes. [LAUGHS] KASPER NIELSEN: I was-- we get this little red squiggle on the List block. It's complaining that we're not initializing the underscore Realm. CRAIG LABENZ: We're not giving it the database, yeah. KASPER NIELSEN: Yeah, so I could take this in the constructor. But actually, I'm going to just, for now, piggyback on the feature that any Realm entity, including Realm Results, knows its own realm. Good like that. Yeah. OK, so now we have something should be hooked up. Just press this button a couple of times. Nothing happens. But that's just because-- let's just put a breakpoint there. We did hit this addNewItem, and we are running this write transaction, and apparently no errors happened. I'll just go in here, change the theme to doc, because that's of course, the area we need to re-render including the list view here. And now we have our items. And if I was to restart this app, which would sort of reread-- maybe I can just, sorry-- go to main here. Put a breakpoint. Do a restart. OK, so it's actually rerunning main. We would see that the items are still there. CRAIG LABENZ: They persisted. KASPER NIELSEN: Yeah, they persisted. Woo-hoo! But obviously, the fact that they don't appear immediately when we hit this FAB-- it's not cautious so let's fix that. CRAIG LABENZ: So we're currently writing to the database, but not updating the UI. KASPER NIELSEN: Yeah, and it's not that the query-- all items were updated. So when I went in and changed the theme and came back, we're still just reading from that all items that we set up once in main. And the items were there. The only thing that caused this not to be shown was that we, basically, weren't calling in setState anywhere. [AUDIO GARBLED] --being rendered. So obviously, we could do that. We sort of hook it up. You get something that says that it needs to be rendered. That would be horrendous and ugly. But the nice thing about these live Realm objects, apart from that you can just read the latest state is that, they can also announce whenever they change. So every Realm entity has a changer stream. And we just go to the SampleItem. You see this, we talked about earlier. It returns a stream of Realm object changes for the SampleItem class. So let's try and use that. So let's go down here in the list view and just wrap this with a StreamBuilder. I'll let it [INAUDIBLE] on it. So we're just going to take from the block. We got the items, which was the whole result set, and that has changes. And now I think that should be-- that did it, right? I always forget. You should, of course, check the snapshot if it has data. CRAIG LABENZ: Being a good citizen. KASPER NIELSEN: Yeah, exactly. If not, you should return something else than the list view. So I'll just do a CircularProgressIndicator. I want to wrap that in the-- CRAIG LABENZ: Const, yeah. I think there's one-- oh, you're going to wrap it in the center, I see. KASPER NIELSEN: Oh, sorry. E like that. It's not-- I've done this before. It's not that it's going to-- let's just try and restart. CRAIG LABENZ: We might get it for one. It was the tiny snicker. KASPER NIELSEN: Yeah, exactly. I think we can-- can you spot it? There we have a circular-- it's just not grown very big yet. But the thing is, the load database is so fast. This is sort of one of the advantages of using an offline first database. We never wait for the backend. Once the backend gets the data ready and we receive it, then we'll react to it. But even on the first load of the page, we'll just read quickly from the database. So we can add stuff. I think we should also be able to delete stuff. I'm getting very affectionate to tailing commas. CRAIG LABENZ: Right? There no other way to live. KASPER NIELSEN: Exactly. When I first saw them, they just looked odd, out of place. But now I love them. CRAIG LABENZ: Yeah, I love what they do to the rest of the formatting. That's why I'm-- KASPER NIELSEN: A fan, yeah. So let's just extract this ListTile. Going to add a little more. SampleItem-- I'm really losing my ability to type. SampleItem type. So what do we want to do? So usually when I want to delete action, I'll make it a swipe-able thing. That means that we want to wrap this ListTile widget with a dismissible. That's going to require a key. Because it's animating between different states of the list. We need something that uniquely identifies the tile and that's-- very conveniently, we can just use ID for now. It's still local. We're just incrementing the last seen ID. What else? This should-- CRAIG LABENZ: It's getting exciting now. KASPER NIELSEN: Yeah, so probably out of background. I always want it like that. I like delete actions to be red. Just jives better with me. Like that. Obviously, nothing really happens yet. So what we want to do is, we want to delete the item in the database. So once again, let's just hoist out that bit of logic into a block. And again, I'm not using the block package. I'm just using the concept of a block. Actually, I think this is what the block package would call a qubit, because I'm adding-- CRAIG LABENZ: You just have the streams. KASPER NIELSEN: --instead of streams. Yeah, this was the item. We will hoist out-- man. And then we will-- CRAIG LABENZ: You're doing great. KASPER NIELSEN: Am I though? I'm losing the ability to type. CRAIG LABENZ: It's late in the day for you. KASPER NIELSEN: [LAUGHS] CRAIG LABENZ: It's very late in the day for you compared to me. KASPER NIELSEN: I think it's the adrenaline. I don't know. I haven't done many of these streams. CRAIG LABENZ: I think you're a natural. KASPER NIELSEN: Thank you. There was an ItemBloc. We go bloc again. Something like that. We take it to the constructor. OK, something won't compile, obviously. We need to-- this item-- we can no longer-- we don't have direct access to it. So we're just going to take it from the bloc. And there was this one up here. Yeah, SampleTileItem now takes a bloc. And that's the ItemBloc grabbing the item. CRAIG LABENZ: And the reason you're doing this is because you don't want to pass the RealmDB object to widgets, is that right? KASPER NIELSEN: Yeah, at this time, that's sort of-- this is not UI logic. This is us updating the database. This should happen. It felt odd to do it directly in the UI. CRAIG LABENZ: Yeah, totally. I totally agree. Just that a lot of demos like this don't take-- and this is not a knock-- but they don't take that extra step to write something in an even vaguely realistic way. And it seems like we're doing a lot here. We're moving a lot of things around. But that's really just because you're refusing to path the Realm object to widgets, which is great. KASPER NIELSEN: So it's like the Realm object itself-- I'm not pathing. But I am, of course, pathing the Realm entities like the SampleItem and the-- also the widget of this reading the items, which is the Realm Results off of the bloc. So it's not completely hidden from the widget. Anyway, I think I-- there's another thing we need to do. First of all, here you are. We need to, of course, implement delete. And it goes by the same structure as before. We're just going to, instead of taking the Realm under to construct, I'm just going to read it off the item. I think, eventually, you might want to actually path it in the constructor because of a feature we've been talking about. You can freeze Realms. So one of the nice things about a Realm item is that they're alive. But sometimes it also gets in the way. Say, for instance, actually the block package. So you have this stream of changes. It will show you the prior state and the current state or the current state and the next day. I can't remember. But it shows you two versions. It'll stream two versions of the object. And if your object is live, it'll be the previous version is already updated. So it'll be like there is no diff. And that might be not what you want to do. So you really want to have an immutable version of the object to compare to. And there's this-- we saw it just briefly in the generated file-- there's this-- you can freeze a Realm object or any Realm entity, really. Which basically, what it does is, it freezes the Realm that the object belongs to. And that's a constant time operation. The full object graph will be frozen in constant time, which is pretty amazing actually. It's not like it's doing a deep copy of the entire object graph. There is a cost to it. But if you keep those frozen objects around, it prevents the database from cleaning up that version. So don't hold too many of them. And make sure to release them. But it's an easy way to create an immutable object. And if you do that, you can't open a Realm transaction on a frozen Realm. CRAIG LABENZ: Makes sense. KASPER NIELSEN: So say the SampleItem had been frozen. Taking the Realm off of that frozen item will not give us a Realm that we can actually use to make changes. You need a live Realm for that. So that was a bit of a detour. But for now, all we have to do is just in inside of write transaction when you want to call Realm delete on the item. And that should almost bring us there. I think maybe I did not-- yeah, I did not. We should also hook it up here on dismissed. So we're just going to-- ah, sorry about my typing-- call block delete when on dismissed is pressed. So now I'm cleaning. If I do like that, items disappear. And if I do a hot restart, it should stay-- yeah. CRAIG LABENZ: Woo! KASPER NIELSEN: We can now add the deleted [INAUDIBLE] items. CRAIG LABENZ: Nice! KASPER NIELSEN: Yeah! CRAIG LABENZ: Basic crud! KASPER NIELSEN: Basic crud-- impressive, right? Yeah, I know this is maybe not so impressive yet-- CRAIG LABENZ: But it's about to get good. KASPER NIELSEN: Yeah, I hope you will find that it's getting better. So how's time coming along? CRAIG LABENZ: The content is more important than the time. So if you're still good, you're still good. KASPER NIELSEN: I'm still good. It was just-- so the thing I need to do now is, I need to set up a backend. And my plan was to do that on the show. And I think I have time to do it. CRAIG LABENZ: Yeah, yeah, yeah, let's do it. KASPER NIELSEN: I'll continue with that. Otherwise, I do have a pre-cooked backend setup. So just in case, especially with today-- because I know that the [INAUDIBLE] guys are rolling out an update. And you never know CRAIG LABENZ: Oh, got it. OK, that's very smart. But yeah, let's try to configure. I don't think it's too tedious. KASPER NIELSEN: I'll try to configure backend. So I'm going to do some of it here from the command line. And so remind me to restart this generator. CRAIG LABENZ: Got it. KASPER NIELSEN: Thank you. We have this command line tool called Atlas. You can install it with proof. I think it's called MongoDB underscore Atlas CLI. That will allow you to set up resources on Atlas and the MongoDB Cloud offering. And the first thing you want to do is, you want to log in. So associate your command line with the account on MongoDB. And let's see. And because I have too many browsers open, it's not opening the window. CRAIG LABENZ: I have no idea where that will go. [LAUGHS] KASPER NIELSEN: So I'm just going to explicitly-- sorry. I was watching that. Where was it? CRAIG LABENZ: Account slash connect. KASPER NIELSEN: OK, so it was actually correct. And I already set up this burner account on Google. It's called Flutter Realm gmail.com. And now I have to associate this. I need to associate this. One time verification code. CRAIG LABENZ: Oh, so that verification code prints to the CLI. KASPER NIELSEN: Yeah. CRAIG LABENZ: And then you calculate into the browser. KASPER NIELSEN: So the CLI contacts Mongo with the session key, you could say. And that is-- CRAIG LABENZ: Interesting KASPER NIELSEN: --then signed-- CRAIG LABENZ: That's the other direction of how it normally goes. KASPER NIELSEN: Yeah, kind of like it. It's a very easy and convenient, very safe way to go about it. CRAIG LABENZ: Yeah, nice. KASPER NIELSEN: I'm just going to confirm this authorization. Your account is ready. Let's go and look at it. It's going to take a little while. OK. So now I'm here in the dashboard. What I was talking about-- I was having this listed project back up that we could use should we run into trouble. But I'm going to set up a new project. CRAIG LABENZ: Yeah, let's do it. I think that'll be great. I'd love to see it. KASPER NIELSEN: We want to just-- first on the-- just going to list-- we can see this one project we already have. But we're going to create a new one. Call it the observable Flutter listy. CRAIG LABENZ: Where are you typing? I'm not seeing any changes on your-- OK. KASPER NIELSEN: I couldn't do that. CRAIG LABENZ: There it is. KASPER NIELSEN: Can you see it? CRAIG LABENZ: I see it now, yeah. I think it was just lagging for a second. I also see an error. Oh, it must not have liked the-- KASPER NIELSEN: It didn't like the brackets. That was-- CRAIG LABENZ: Cheeky name, yeah. KASPER NIELSEN: I'll just set this, because otherwise, I'll mess it up. I'll just set this project idea as my default. CRAIG LABENZ: By the way, I don't know when you need to do this, but at some point, you need to restart a generator. KASPER NIELSEN: That's true. And that is when we get back to the client side of things. CRAIG LABENZ: Great, I'll say it again then. KASPER NIELSEN: There are some stuff we need to change in the model in order for the backend and-- CRAIG LABENZ: For it to sync. KASPER NIELSEN: Yeah, we need to agree on the primary key. And that means that I have to do changes to the satellite zone, and when I do that, I need to rerun the generator. CRAIG LABENZ: The generator. KASPER NIELSEN: It's like 99% certain I will miss that. Maybe not today. You already reminded me. OK, so we need a few other resources. We also need to set up a cluster. We're just going to call that listed cluster. And this is going to complain. We're not telling it enough. We need to set a provider. This is where, as I promised, we're going to use Google Cloud platform. CRAIG LABENZ: Exciting. KASPER NIELSEN: The only one to use, right? The region's just going to be-- I'm just flattering you guys. It's going to be Western Europe, and we're going to choose the tier. And we're going to use it's called M0. I'm not sure why, but it's the lowest tier you can choose. And it has the distinct advantage of being completely free. CRAIG LABENZ: That's a good advantage. KASPER NIELSEN: Yeah, and I promised to say forever. I don't know if I should say it, but I hope it's like, you would never have to enter a credit card. Though I would say, though, that since stuff that is free, people kind of tend to forget. And they don't shut it down if they don't use it. So we are a bit aggressive about shutting down stuff that is not getting used. So say you have this very rarely used app that somebody only checks in every once in a month or so. You may find that your cluster have been terminated. But you can always start it up again. So that's it. So let's just go to the URE again or web UI. It should be-- now there, see-- I called it the listy app. Maybe I should have called it the listy project. And in there we have cluster. And as much as I would like to do the rest from the command line, because I am mostly a command line kind of guy, in particular, since we chose Google Cloud platform as our provider, there are stuff that I cannot use to set up with the current-- it's called Realm CLI, by the way. As you said, in the prior days, the backend app device sync was also called Realm. But that changed this summer. So we're rebranding. And there's some tooling that hasn't completely followed through. They're working on it. I know it should be ready within a month or two, I think. Anyway, the rest of the thing I'm going to set up in the web UR, because that has the ability. That's the round logo. Just shortly you saw it. CRAIG LABENZ: Yeah. KASPER NIELSEN: I'm just going to put my own app here. And this has already picked up my cluster. I'm going to call this something a little less stupid. And this is the thing I couldn't set up in the Realm CLI. I had to use that. It would force me to use AWS here, which, of course, is not kosher, not only because I'm on a Google show, but also because we really want the app services to be co-located with our cluster. We don't want to suppress the world whenever we do it. CRAIG LABENZ: Yeah, that would hurt. KASPER NIELSEN: That would be stupid. That would hurt. You could actually see that on the show if we did that. Creating the app services. Then there's all kinds of guys. We'll just quickly make that go away. And there's really only one service we need to enable today. And that's device sync. And by the way, notice that our app got this little app ID. We're just going to copy that for later, because that's what we're going to use to hook up the client to this particular app series instance. CRAIG LABENZ: Nice. KASPER NIELSEN: So we can enable device sync, start syncing, and it asks us if it wants to generate a schema from some existing data. We don't really have any existing data. So, no. CRAIG LABENZ: True. KASPER NIELSEN: And then first thing we need to choose is whether we want to use flexible sync or partition based sync. And as a Flutter developer, that's an easy choice, because we can only choose flexible sync. We do not support partition based sync. And we don't intend to do that, as I understand it. The thing is that partition based sync's more or less being deprecated. It'll be supported. But it's-- CRAIG LABENZ: It's not the future. KASPER NIELSEN: It's not the future. To talk about the distinction-- flexible sync, basically, allows you to specify what subset of data you need to synchronize with using queries. So obviously, the backend data may contain a lot of data, and you don't want all of that on your device. So you can specify the subset that you're interested in. And you can do that in a very flexible way using queries. Whereas the old way of doing it was that every object was tagged with a partition. And the client had to choose exactly what partition you'd want to synchronize with, which is a lot less flexible. It's also very easy to implement say partition-based using flexible sync. You can just add such a tech to all your objects and do the same. Anyway, flexible sync is the future. So next thing we have to choose if we want to use development mode-- and during development, I strongly recommend you do that. Don't do production. What development allows you to do is something that sort of happens automatically for you. Say I was to change the schema. Right now we don't have a schema here. So the first time my client contacts the backend, it'll alt know a schema. And the backend will say, OK, let's use that. And as long as the changes you would do on the client-- as long as they are-- where did it go here? Sorry. Disappeared. Are you still there, Craig? Craig? CRAIG LABENZ: I had muted myself, but yeah, I'm here. KASPER NIELSEN: OK, sorry. Thank you. There you are. OK, as long as the changes you are making are additive changes, we can also add new fields, new classes, et cetera. Then we'll just-- the backend will just accept the new schema. If you change, say, the type of a field or do other sort of destructive changes, then this will stop working. And the easy way out is to recreate the backend and start over. CRAIG LABENZ: Which you can do if you're just in development mode still. KASPER NIELSEN: Exactly. And also, you don't want to enable this in production, because say somebody, some rogue client-- CRAIG LABENZ: Of course. KASPER NIELSEN: --with an old version or whatever-- CRAIG LABENZ: You never want to enable anything that says development mode in production. KASPER NIELSEN: Exactly. CRAIG LABENZ: That's why it's named development mode. KASPER NIELSEN: Exactly. Then we need to select a cluster. And already did that. We only have one. And we need to select the database. This is-- so now we are on the backend. So this is a MongoDB database. It's not a Realm database. And we don't have one. So we're just going to make one like that. OK. The nice-- you can configure the queryable field. If you've worked with Realm before, you know you had to do that. That's no longer required when you're in development mode. Then it's optional. The thing is, when you do these queries that defines your subscriptions, you are querying various fields in the database. And the backend needs to know that these fields are used for synchronizing queries in order to perform well. It needs to add indexes to them and such. But in development mode, that happens automatically for you now. So we don't actually need to specify it. Also, for this particular demo, I'm not doing anything particularly interesting. I'm just going to use the mandatory underscore ID field for-- actually, I'm just going to sync down everything. So every device will see everything, which is, of course, highly unrealistic. CRAIG LABENZ: Yeah, a little bit. KASPER NIELSEN: But this sort of gets to talk about this primary key that the client in the backend needs to agree on. It's got to be called underscore ID. And this is what-- CRAIG LABENZ: We're going to have to-- KASPER NIELSEN: We use to-- we track to see that optics are identical. Then we need to define some permissions. So far, we sort of just left it to the client to decide what subset of data it wants to sync. But it could say everything if it were an evil client. I also want to see my neighbor's purchases, whatever. So we need to add-- CRAIG LABENZ: Your nosy neighbor. KASPER NIELSEN: Yeah, exactly. We need to add a set of permissions on top of this. And these are, of course, handled by the backend. And you can see the data that is actually synchronized to your device is the intersection of what you asked for and what you are allowed to see. CRAIG LABENZ: Aha! KASPER NIELSEN: But I'm just going to be really stupid and allow everybody to read and write all the data. CRAIG LABENZ: Yeah I mean, that's how-- that's pretty convention for development mode. KASPER NIELSEN: Yeah, especially for a demo like this. There's some advanced configuration as well. I don't think-- so how far back in time do we allow clients to fall before we force them to reset? We need to maintain some history in order to do this conflict resolution elegantly. And there's a cost to that. And say you fall behind by more. I think the default 30 days. If a device falls behind more than 30 days, I wouldn't say all bets are off. But at least the nice conflict resolution is out the window, and we just do a hot reset. So you can configure how far back you want that to work. Gonna enable it. CRAIG LABENZ: All right. KASPER NIELSEN: And there's this general thing with Mongo-- you sort of get to review your changes. And the changes are really just the set of configuration file JSON as you see here. And you would store these in UIKit repo. We're not going to do that today. Then you're going to deploy it. One thing to notice, perhaps, is that we are enabling something called anonymous user and also provider called anonymous user, which will allow us-- which we're going to use to log in. This should be it. I hope it wasn't too tedious. Back to the client. And now you have been a good boy, Craig. You reminded me that I needed to rerun-- CRAIG LABENZ: Restart the generator. KASPER NIELSEN: Exactly. Thank you. CRAIG LABENZ: I have my moments. KASPER NIELSEN: Sure do. CRAIG LABENZ: [LAUGHS] KASPER NIELSEN: I mean it. CRAIG LABENZ: Why would they say? KASPER NIELSEN: I'm a fan. So let's go back to where we actually opened the Realm. So as I said, the interesting configuration was flexible sync. So just set up a flexible sync backend. And now it's putting this red squiggle, because we're not providing the right arguments. So we still need to path in the schema object. That's what's going to be used in development mode to set up the correct schema in the backend. But we also need to path in a user. And how do we get a user? Well, first we need to handle on the app service that we just created. Let's just call the app class. And that takes an app configuration. That takes a whole lot of bunch of arguments. But there's just one crucial one. It's this app ID that I copied out during the configuration. CRAIG LABENZ: Now inquiring minds will want to know, is that value sensitive? KASPER NIELSEN: No, it's not. CRAIG LABENZ: Not sensitive. KASPER NIELSEN: This is not sensitive, no. It's just like an URL, really. It's a moniker to a lot of things. CRAIG LABENZ: So all off-- all security is elsewhere. That could be right in the code. It doesn't have to be a secret. All that good stuff. KASPER NIELSEN: Yeah. Because it's basically impossible to keep a secret. Everybody can decompile you out. CRAIG LABENZ: My client-- yep. KASPER NIELSEN: How do we get the user from the app? Well, maybe there's already a logged in user in which case, we can just get it. This will work offline. app.currentUser, if you're previously logged in, you would still log that in. But say you wasn't ever logged in. You need to log in. And to do that, we just call out log in and pass a set of credentials. And since I enabled the anonymous off provider, it means that-- and basically, what is the anonymous off provider really doing? So we are creating a user here. And it is uniquely identified. But it's only on this device. So whenever we'll use this, even if I didn't do this app current user thing, will use these-- the same credentials for this particular device. CRAIG LABENZ: So then if you create an account later, you own whatever records you've made along the way? KASPER NIELSEN: Yeah, you can actually associate an anonymous account with a validated one, say, using a federated authentication. I think in-- I want to do a demo at one point where I integrate with Firebase Auth UI, which would give you a job that you can-- CRAIG LABENZ: That package is so nice. KASPER NIELSEN: But yeah, and it's actually super easy to work with Firebase Auth, I think, with the UI package. So it shouldn't take that long. Anyway, now I've got to use it. CRAIG LABENZ: That was the first parameter, right? KASPER NIELSEN: Yeah. CRAIG LABENZ: Nice. KASPER NIELSEN: What else? So I was talking about these subscriptions. Need to set up subscriptions telling what subset of data that I'm interested in. How much time? I'll leave that out for now, because I want to show some errors that occurs when we don't have that. Basically, by not subscribing to anything I'm saying, I'm not interested in anything. So let's try and rerun this. So this is the first error we get. It's complaining there must be a primary key property named underscore ID. Remember that underscore ID that we saw in the queryable fields? We didn't actually add that to SampleItem yet. I'm just going to rename this thing ID, because we want to use ID for something else. Just call it number. What did people do before we had these automated refactorings? CRAIG LABENZ: Cried I think. KASPER NIELSEN: I know I did. And I'm going to use this object ID class that I was talking about, which is basically like a timestamp plus a little entropy for the process plus a camera. So let's call that ID. I'll make it public, because I'd like to use it a little later. And this needs to be a primary key in order to specify on Realm that field is in primary key. You just annotate it with primary key. But the backend would like it to be called underscore ID, which would make it private in Dart. So we just have to do this. You see this popped in in Flutter code. Something don't compile. Obviously, now when I construct the SampleItem, I need to also path in an object ID. And I can just pull it out of-- I'm almost tempted to say my ass, but out of thin air. CRAIG LABENZ: Well, you did say it. KASPER NIELSEN: I was like, no-- [LAUGHTER] It's basically reading the time and doing that little bit of-- adding that a little bit of extra entropy from the-- CRAIG LABENZ: And folks in the chat were asking about like, why isn't there an easier auto ID option? And this is basically it. KASPER NIELSEN: Yeah. That's the-- CRAIG LABENZ: Nice. KASPER NIELSEN: In a distributed system, it's just much more convenient to have your primary keys be something that can be generated offline. And that means something-- you're going to use some randomness to ensure that. You can also-- we could also use the UUID. It's just sort of-- with MongoDB, it's common to use-- CRAIG LABENZ: Probably somewhat similar, yeah. All right, I anticipate that we're getting close. KASPER NIELSEN: We are getting close. Oops, all items disappeared. Why did they disappear? They disappeared because-- CRAIG LABENZ: We have a reading from a new database. KASPER NIELSEN: Exactly, Craig. It does, of course, exist locally on disk. But it's in a different position due to the lock and stuff. CRAIG LABENZ: That's now a zombie database. KASPER NIELSEN: That's the-- oops. That's the next error you're going to hit. So I tried to press plus. And it says, cannot write to a class SampleItem item with no flexible sync subscription has been created. So it's basically telling us. OK, so you're interested in no data, yet you're writing data. What gives? So you need-- what would happen if-- basically what happens if you write outside of the subscriptions, of the data you are interested in-- you'll have-- the backend will fix that by doing what we call a compensating write. It will remove the item again. CRAIG LABENZ: Oh. KASPER NIELSEN: And this sort of catches a lot of people off guard. The most common reason is that they never set up any subscriptions. So we put in this convenient little exception to-- CRAIG LABENZ: Smash that subscribe button, man. You know? KASPER NIELSEN: Yeah, exactly. CRAIG LABENZ: You press that bell like a-- [LAUGHTER] KASPER NIELSEN: We're going to add a subscription. And a subscription is just a query. It's just specified by query. And you don't-- these are persistent. So when-- just because you restart the app doesn't mean that your subscriptions are gone. But there's really no harm in setting them up on every run. CRAIG LABENZ: OK. KASPER NIELSEN: Sorry, it's not called-- it's called-- and for now, I'm just-- [INTERPOSING VOICES] CRAIG LABENZ: --thing we care about all sample items. KASPER NIELSEN: Yeah, exactly. I'm very-- I'm building a very-- CRAIG LABENZ: Very public-- KASPER NIELSEN: --very public distribution system here. CRAIG LABENZ: Yeah KASPER NIELSEN: But now we have a subscription. So I hope that I will be able to-- I just need to-- CRAIG LABENZ: We may-- did you ever restart the-- oh, no, you don't need to restart the generator, because you had it on watched. KASPER NIELSEN: Yeah. CRAIG LABENZ: Nice. KASPER NIELSEN: Oh, I can create stuff again. And I can delete stuff again. But that's not so impressive, because this is just one device. So let's start another device. Where did that go? It's not a select device. You may notice that I have three other devices set up over here on the left. I've got a Pixel. I've got an iPhone 14+ and the iPod. This is one of my favorite features of all your tooling is that you can debug multiple sessions at the same time. CRAIG LABENZ: That is pretty slick. KASPER NIELSEN: It is so slick. So let's do the-- one thing that annoys me, though, is that I can't just press a F5 here for some reason. I need to go to the-- and then I can. CRAIG LABENZ: Oh, interesting. KASPER NIELSEN: If somebody will lend me their ear, I would like to complain about that. CRAIG LABENZ: That's fair. KASPER NIELSEN: And actually, even another cool thing is, that we can-- kind of lost my ability to choose down here. I can even start multiple builds at the same time. So I'm going to do one for the iPhone 14 as well, and let's also do the iPad. Sorry. CRAIG LABENZ: Your computer is like, really Kasper? Really? Everything at once? KASPER NIELSEN: Everything at once. I mean, they gave me one of these in one-- CRAIG LABENZ: Can do everything at once machines? KASPER NIELSEN: It's an amazing fast machine. Still, it does take a lot of time. CRAIG LABENZ: Hey! KASPER NIELSEN: Pixel is up. We've got items here. Should work. Though 8-- it'll look similar, right? So let's try to remove the last item here. CRAIG LABENZ: Yeah, get out of here 8. OK, we switched back. It's already gone. Can't even switch back fast enough. KASPER NIELSEN: I can't. These should be up soon. CRAIG LABENZ: One would hope. Yeah, those must be-- KASPER NIELSEN: Those were old ones. Here we go. CRAIG LABENZ: OK. KASPER NIELSEN: And now we can see-- CRAIG LABENZ: Very good. Very good. KASPER NIELSEN: I'm sort of surprised we don't-- did I not start it on the-- CRAIG LABENZ: I thought you did, but-- KASPER NIELSEN: Yeah, I thought I did too. It was the iPhone 14+. Maybe I took the Pro. Anyway, there you have it-- synchronization. I hope you find that the last stuff that I did to actually enable it client side was kind of unintrusive. CRAIG LABENZ: Because this is offline first, is there an easy-- could we like put one of these in airplane mode or something and demo the offline first part? KASPER NIELSEN: Yeah. So that's super un-- you can't really do that with the iOS simulators. You have to put your entire device-- your entire laptop in flying mode, which is super annoying. Luckily, Android is a lot less stupid. So we can just put this Pixel into airplane mode. CRAIG LABENZ: Airplane mode, OK, nice. KASPER NIELSEN: So we'll just remove an item here. You can see the-- CRAIG LABENZ: Yeah, of course it didn't-- KASPER NIELSEN: It didn't propagate to the Android device and I'll just do something here. So now the Android-- notice-- this actually reminds me of one thing. I don't know if it's going to cause-- maybe it's going to cause us trouble. CRAIG LABENZ: Well, there's a number. But there's-- but object ID is different. It should be OK. KASPER NIELSEN: Yeah. Only the thing is that-- CRAIG LABENZ: Oh, the value. The key. KASPER NIELSEN: Yeah, the value key for the-- CRAIG LABENZ: Well, can you make it underscore ID? No, you won't be able to do that, because it's underscore ID. Or just dot ID. No, that should work, because it's not number in there. Oh, it is number. KASPER NIELSEN: Yeah, because the thing is, I did a refactoring, right? I just renamed the ID to number. And that included this place. And number is no longer unique. So I really still need the ID, which is unique. CRAIG LABENZ: Yep. Yep. KASPER NIELSEN: That should probably-- notice how it hot reloaded on all three devices. CRAIG LABENZ: Pretty slick. KASPER NIELSEN: That's probably enough. Pretty slick. CRAIG LABENZ: Yeah, I think that should-- I know what you're getting at here once you take it out of airplane mode. KASPER NIELSEN: Yeah. I'm worried that we're going to have an issue with the dismissible item. Because that value key was wrong. But let's-- CRAIG LABENZ: Let's try it. KASPER NIELSEN: Let's try it. So take it out of airplane mode. CRAIG LABENZ: Back to the app. KASPER NIELSEN: Go back. And it is going to take a little time, because it-- Realm doesn't automatically detect that the network changed. It's sort of working on this exponential back off scheme. And since I talked a little too long, it already sort of incremented to a certain state. But the easy way to fix this, and maybe we could do that soon, is to, say, take a dependency on the connectivity plus plugin. CRAIG LABENZ: Yeah. Yeah. Would it try again if you tried to create a new one? Or would it still just be like, oh-- KASPER NIELSEN: It'll still-- yeah. I think it's-- CRAIG LABENZ: Hey! They're there. They showed up. KASPER NIELSEN: You want it to wake up quicker. And the way to do that is, let's go and do it, actually. It doesn't take a long time. So depending on our time, I'm just going to break this one again. So we're going to add a-- CRAIG LABENZ: I think we're definitely near time now. KASPER NIELSEN: OK. Anyway, so just stop me if it's too much. This is called connectivity plus. Did I spell that correctly? Actually, this is also a plug-in so we'll have to restart. Anyway, you'd do something like-- CRAIG LABENZ: I think that's-- we can probably leave that as a reading exercise for the [INAUDIBLE] later. Nice, though. That was great. It seems like we got to the finish line. KASPER NIELSEN: So it has been a pleasure to be on your stream, Craig. CRAIG LABENZ: Yeah, thanks for joining. I just want to hit some questions real quick here. This one is from quite a while ago. We answered that. You can set a primary key. Ended up getting to that. Scrolling down. Someone a long time ago said, what is Realm? It's client side SDK for MongoDB with offline first data syncing. You may have also deduced that by the end. Someone also said, for it to sync, we have to hook up with Atlas, right? That was before we did the whole hook up with Atlas. So now we've walked through that. [CLEARS THROAT] Excuse me. Oh, one question someone says, is the data encrypted locally? KASPER NIELSEN: No, not currently. So say you want at least an iOS device. Your storage will be encrypted at rest anyway. But you can add encryption to the database. You'd need to specify what key to use. And you'd store that somewhere in the secure element on the device. But I didn't use it today. But we do support it. CRAIG LABENZ: Got it. OK, so not by default, but can be done. KASPER NIELSEN: Yep. CRAIG LABENZ: OK, cool. I think that that is a great place to call it. Kasper, thank you for joining and walking through. We made a to-do list app, essentially. We made a list app. Persisted those things locally first, just using Realm. So for Flutter developers who are pretty experienced, at that point we kind of had a Hive-like situation going. But Hive is always and forever local storage. So we escaped containment, and we reached escape velocity. And we got to the server by adding MongoDB and data sync through Atlas. And that allowed four emulators to all talk to each other very simply, including when one went offline via some cruel airplane mode enforcement from you, our flight attendant. And then when we took it off airplane mode, it woke up a second later and synced to all the other devices. Pretty good. KASPER NIELSEN: Correct. Yeah, well, I'm glad that it went so well. CRAIG LABENZ: Yeah. No, well done. You're a pro. Cool. Well, everybody, that will wrap it up for this week. And next week I have an extremely exciting guest. And I don't think anybody is going to want to miss it. So until then, I will see you all later.

Channel: Flutter

Views: 27,340

Keywords: mongodb, realm, Observable flutter, what is observable flutter, live coding, code tutorial, coding tutorial, how to code, live code, flutter livestream, flutter stream, flutter latest, flutter updates, what’s new in flutter, flutter tutorial, introduction to flutter, how to use flutter, google stream, google livestream, flutter developer, flutter developers, google developer, google developers, flutter, google, Craig Labenz

Id: 25cuM0ZCwu0

Length: 108min 41sec (6521 seconds)

Published: Thu Feb 23 2023

Reddit Comments

[u/skryu]

Always loved realm back before it was brought into mongo. Used it all the time in xamarin. It’s a shame this has been in development this long as it would of been a good option to firebase.

How are the capabilities of the SDK for large user use these days? Say if I wanted to do a geo search on a collection with a million entries. Anyone know?