Apple WWDC 2017 - Session 226 - Build Better Apps with CloudKit Dashboard

you good morning my name is Dave browning I'm on the cloud kit team here at Apple and I'm super excited to talk to you about some new things in cloud kit especially a brand new cloud kit dashboard if you're not familiar with it the dashboard is a web application it's designed to help you as you're adding cloud kit functionality to your apps so first I'd like to talk about some of our goals behind the things we're announcing today and at a high level as the title of this session suggests we really want to help you build better applications on top of cloud kit and we want to do that in a number of different ways first we want to help you through all stages of your apps lifecycle so when you're first learning about cloud kit when you're starting to build your functionality into your application when you're beta testing with users when you're getting your app out and the App Store going to production scaling up really honing in your user experience and then finally potentially supporting customer problems in the wild we want to give you the tools necessary to help you throughout all of those phases we also want to give you a way to experiment with the entire API now obviously you can jump right into Xcode and start using our API and figuring out how it works but we want to give you something to sit alongside Xcode then lets you visually play with that same API so that you can experiment with it understand how it works while you're building that functionality into your application we also want to give you visibility into all events across the system across all of your users and especially once you're out in production in the App Store and growing your user base we want to help you understand the aggregate behavior the communication among all those users back to the cloud kit server now before I jump into a live demo of the dashboard I want to do two things first let's do a quick refresher of some cloud kit concepts if you've used cloud kit before you'll be familiar with some of this but it's a good foundation if you haven't used it because I'll be using these concepts throughout the session and then after that second I want to show you and talk about an example iOS application that we built on top of cloud kit that I'll be demoing today alongside the dashboard to show you how the dashboard can help you when your test and building your application alright so first some cloudkit concepts at the end of the day you're using the API to store records back to the cloudkit server this is your structured data your keys and values and as a reminder the values can be different types strings and doubles an asset which is a binary file when you store a record it always exists inside of what we call a zone a zone is basically a bucket of Records and it's a foundational piece of some of our api's that we'll talk about in a bit a zone always exists in a database and some databases allow you to create different zones if you need to bucket records in different ways all of your users have their own private database this is where you store their private data that only they can see they also have their own shared database and this is because last year we launched cloud get sharing and if whatever your users shares data with another one that data shows up in their shared database you can think of it as a proxy back to the owners private data and then finally there's a public database which everyone can read and write to and there's one of those all of this data all of these databases live inside of an environment when you're building your application this is the development environment and at the environment level you define your schema your record types the type of data you'll be storing and potentially indexes if you're going to be querying it now you're using the development environment as you're building your app and then once you put your app in a store it talks to the production environment so we give you a way to promote your schema changes to the production environment and then in production all of your users have their own private shared and one public database they're separate from the development environment and then all of this lives in the highest level concept which is a container container has a unique identifier and it usually maps one to one with your application okay so that was the cloud kit concepts so I mentioned that we built an example application on top of cloud kit that I want to use today during the demo so let's talk about it it's a to-do list app the age-old example it's relatively simple the idea is users can create to-do lists and they can have items inside of there so let's talk about the data model that we'll be using since we'll be storing this data back to cloud kid so every time your user creates a to-do list in the app will create a list record on the server whenever they create items within the list we'll have an item record for each and we need a way to point the item back to the list to which it belongs and in cloud kit you do that with a reference which is basically a pointer back to another record via its identifier and in this case we're going to use what's called the parent reference the parent reference is a system field that exists on every record it's there for you by default and it's the way in cloudkit that you logically tell the server when something has a parent so in this case we'll point the item back to the list record using this parent reference now we also in this example app wanted to allow our users to securely share to-do lists with other users using cloud sharing so here's how that will work in the data model our list record will be what we call a route record and it will point to a share record the share is where you define the set of participants and their permissions and because we use the parent reference to point items back to a list they automatically get included in the share for us you don't have to go and share them separately to learn more about sharing make sure to go back and check out last year's session what's new with cloud kit ok so that's how we're going to be storing the data now let's talk about the API is we're going to be using how we'll communicate back to the server so the way we wanted this application to work is that if a user opens it up and creates to-do lists with items on their iPhone but they also have an iPad and they run the app there we want that data to be synchronized and because we're using sharing if I share a list with someone else and either of us edits it we want the other to see those edits so we have a core data we're locally storing all of our data locally in core data and we do this for quick reads and writes and for offline access but every time there's a modification when there's Network availability we want to send that back to the server the server holds the truth and if we use in cloudkit what's called subscriptions we can have the server send push notifications to my other devices or to other users devices whenever data they care about changes so here's how we're going to make that work in the example app the first time it runs we're going to create a seek a database subscription on the server for that users private database that tells the server to send push notifications to this users devices whenever their private data changes anything inside their private database we also because we're using sharing have a subscription in their shared database so cloudkit will now send push notifications to this users devices when data changes in either of those places now once we receive one of those push notifications in our app or when it launches we need a way to ask the server for the changes that exist that we don't have yet and we're going to do that with a CK database CK fetch database changes operation this asks the server please tell me what zones have changed remember we mentioned zones earlier inside of this database and if any of them did we can then turn around and ask the server what records changed inside of that zone or those zones with CK fetch records own changes operation to learn more about this check out last year's session cloudkit best practices and because it is such a commonly used workflow we've added a new documentation called maintaining a local cache of cloudkit records that walks you through this workflow and gives you a bunch of swift code so that you can build it into your application now because fetching changes is such a common thing we've built the ability to play an experiment with it right inside the dashboard so let's go take a look alright so up on the screen on the left side you can see the new cloudkit dashboard and on the right side i'm sharing the screen of this iphone running that example to-do lists application that i mentioned before now if we look in the dashboard we're on the home page i'm signed in with my developer account and the first thing you can see are all of your developer teams all of the teams that you're a part of because you may be a part of more than one team for each team you can see the cloud kit containers that that team owns and you can quickly see which ones have been deployed to production and which ones are still in development if you have a bunch of teams or a bunch of containers you can always filter down the list up at the top now in this case this is the container I'm using for this example app that I'm doing so let's click in when you click into a container you see the development and production environments that I mentioned earlier side by side and in this case we're using the development environment because I'm still building this example app so let's click into the data section and into a tab called zones this lets you play with the zone API and cloudkit you can ask the server for zone information inside of a specific database and now in this case I want to show you I'm logged in to the dashboard with my developer account and that Apple ID is the same one on my device it's my personal iCloud account and my developer account so because it is my personal iCloud account I can see my private and share databases so here the dashboard asks you would you like to load zones from a private database for this account the shared database or the public database and let's do the private database because we want to use the fetch Changes api's we're going to check this box to fetch the zones that have changed inside of this database since a certain point in time now the way this API works is it allows you to specify a change token which marks where in history you have synchronized up to you can apply that here and we're gonna leave it blank which tells the server we want all changes inside of this database since the beginning of time and if I fetch the changes you see we get a result this to do zone is the zone that my application created the first time it ran on this device and it gives us a place to store the future records that we'll be saving as the user creates to-do lists and items notice that the server returned to change token which it's populated automatically in this field marking that we've moved forward in time so if I fetch changes again with this token now the server says no zones have changed since then so we're up to date now if I clear that token and we go back since the beginning of time we'll see my to-do zone if I hover this row the dashboard gives us a handy little link to then go and fetch the record changes inside of this zone so let's do that notice that taking us over to the records tab it's chosen my private database to do zone that I clicked on and it said we want a load record using the fetch changes API s and we see two results over on the right the first is a list record and if you look on the right side of the screen you'll see that's because I've already created it to do lists in my application vacation ideas and it stored that back to the server and so the server's telling us about that change we also see a share record but we'll get back to that in just a second so much like the fetch database changes when you fetch zone changes the server returned to us the change token where we are now in history so if I fetch changes again it says there are no new record changes inside of this zone since that point of time all right now let's take the iOS application and create I need to do list or maybe not well the good news is oh there goes it caught up all right and let's name this one chores because unfortunately I'll have to do those all right so that technically say back to lists to the server so we have three here in iOS application now let's jump back to the dashboard and fetch the changes inside of this zone since the last time when it said there weren't any so now when I fetched changes we see those two list records that I just created in the app and if I tap on the record name for that second one it opens up what we call the record editor this shows you information about that specific record and in this case we can see it's unique record name we can see that record type and in this case it's a list record type like I mentioned before when we were talking about the data model it's in my private database in the to do zone and it was created and modified by me just a second ago down below you can see the fields that we've stored for this record and in this case we're using a name field to store the name that the user provided in the application so I've decided chores don't sound very exciting so let's rename this two movies to watch and change the to-do list now I'm going to save this back to the server via the dashboard and I want you to watch the iOS application on the right side so the dashboard changed save that record change back to the server the server saw that I had a subscription for my private database sent a push notification to my device my applications saw that push turned around fetched to the new changes updated its local core data cache and now you can see the UI reflects the fact that we've changed that list to movies to watch so hopefully that walks you through some of the things that you can do now with the fetch Changes API let's jump back to slides and recap what we just talked about so in the dashboard when we use the record editor to modify that record to update it on the server it used the publicly available cloud kit web service API is that all of you can use it send an HTTP POST to the records modify endpoint and it sent a JSON body saying I would like to update this record and set its name field two movies to watch that corresponds to the ck modify record operation in the iOS the native API which we're using the example iOS application up here when we fetched zone changes inside of there to do zone that used the publicly available records changes endpoint and it said server please tell me the set of records that have changed in the to do zone for Dave's account since this change token and that corresponds to the ck fetch record zone changes operation in the native API which we're using in our app so the point here is that the dashboard now lets you play with the exact same API that you're using in your native and web applications hopefully allowing you to understand and debug functionality as you're building it into your application now I mentioned that part of getting all of this data synchronization to work is subscriptions and push notifications so let's jump back into the dashboard and see how it's given us the ability to experiment and understand more of this behavior so I'm going to close out the record editor and as a reminder in the development environment data section there's a new tab here for subscriptions if I click it this lets me use the subscriptions API in cloud kit to fetch down subscriptions that exist inside of a specific database so I'm going to ask what subscriptions exist in my private database and we see the return server returned a result it's a database subscription and I've given it the ID of private changes which is something I chose in my code so that's a subscription that exists that will tell the server to send push notifications whenever my private data changes if we look in the shared database we also see a database subscription there like I mentioned with the ID shared changes now there are subscriptions and push notifications and you're fetching changes potentially across multiple devices there's a lot going on here wouldn't it be awesome to see a log of all of these events as they're happening on the server well now there is if I click back up to the container thanks there's a new section called logs that I'm gonna click into for the development environment this starts on a feature we call live log it opens up a real-time connection back to the cloud kit server and the server will push events as they happen right to your browser it loads a bit of history so here we can see some requests we were doing from iOS from the app up here as well as from the web service API because that's what the cloud could dashboard was using we can see we were doing some record modify some zone fetches etc now let me clear this out and let's create a new list and watch what happens in the log so I'm going to create a list BAM we see the event pop in or boom so here we can see some information about this specific event but let's jump back into slides and talk about all of the things you might see show up in your log so here's an example row like we just saw in a user interface let's walk through each column and see what shows up the first one is the time this is the time at which the event happened on the server the next column is the platform this tells you which platform the request came from because cloud gives that available in iOS Mac OS TV OS and watch us you may see all of those show up here if you're building apps on those platforms you'll also see the version of the platform and if you're using the web service API to build a web application or to extend your app to another platform then you'll see a show up here as web the next column is the user this will show you the cloudkit user record ID now in this case if the person that sent this event is on your developer team we know their name and will show it so we knew my name because I'm logged in to the dashboard on my team and this is handy as you're testing things out with people on your team in development but for everyone else for all of your normal users it will show you the cloudkit user record ID the next column is the type of the event this is most commonly database as you're interacting with the database API but if you have subscriptions and cloudkit is sending push notifications on your behalf you will see pushes show up right here in this log as they happen you will also see sharing events called out separately the next column is the operation ID so as of iOS 10.3 and the newest version of Mac OS Sierra cloud kit will automatically create unique IDs for every operation that you're issuing in the native API and you'll see them show up here the next column is operation group name which we'll get back to in just a second and the final column there the set of details specific to this type of event now in this case it was a database event and so we can see the type of database operation that we issued back to the server this one was a zone fetch but you'll see things like record modified database changes depending on the type of operations you're submitting in this case because it's a database operation we can see it was in the private database and in the to do zone we also see the server latency now this is how much time it took the cloud gets server to process this but note that it doesn't include internet latency or the time spent getting from the client to the server and back you can see the request size and the response size you can also see the hardware identifier and know this is not a new iPhone this is actually the identifier for an iPhone 6s plus like I'm running up here some more information about operation groups which we'll talk about just a second and finally the request ID this is interesting because when you issue an operation via the API that usually maps to a one to one request but in some cases the climb may need to issue multiple Network requests to carry out your operation and if it does you will see multiple roads show up in the log with the same operation ID but different request IDs allowing you to differentiate if an event leads to an error you will see it called out in red and it will tell you the specific type of error in this case I try to fetch a zone that didn't exist and so the server said this was a zone not found alright so I mentioned operation groups a couple times so what's that about well new and iOS 11 and all of the other matching platforms we've provided as you probably guessed the ability to group operations based on application logic so let me give you an example in this example to do list app there are a number of things that it needs to do when it first launches in the initialization logic it needed to create a zone on the server within which to store our records it needs to create two subscriptions remember I mentioned the database subscription in the private and shared database and it needs to fetch down any existing changes in the private and shared database and then potentially fetch a zone changes so there's a number of operations all of which encompass the initialization logic and so we now have a way to group all of that logic together now as you probably know there are a number of apple applications built on top of cloud kit and I'd like to give you some examples of how we use cooperation group names in our apps so iCloud backup is built on top of cloud kit so every night when your iPhone or iPad is plugged in and on Wi-Fi it'll automatically back itself up that may take a number of cloud kit operations to do and so we can group those up inside of an operation group named automated backup if the user triggers one manually we can call that out separately as a manual backup and when they go to restore the data on a new device we can have an operation group that encompasses all of the operations within iCloud photo library is built on top of cloud kit and they use operation groups to designate when they're setting up your library downloading thumbnails or fetching a movie that you tapped on iCloud drive is built on cloud kit and so they have operation groups to call out when they're initially pulling down any changes from the server when they do so after a push because of a subscription or when you tap into a file to download it so let's look at the API so there's a new class CK operation group it provides you an operation group ID set for you by the system it allows you to specify a CKD operation configuration which we'll talk about in just a second it allows you to provide a name this is any string that makes sense in your application and the names on the previous slide were ones that we use and you want to be careful not to put personally identifiable information into this this is your app logic what it's doing the next property is quantity this is an integer that you can set and it's completely up to your application so some examples from our apps might be when we were backing something up and I called back up we might designate how many files were backing up or an iCloud photo library when they're downloading thumbnails we could use to say how many thumbnails are being downloaded in this group but the point is it's completely up to your application you can also set the expected send and receive sizes and this tells the server I'm sorry it tells the client how much data you think your estimating will be sent back and forth between you and the server and notice the type is CK operation group transfer size and this is an enum which allows you to specify an order of magnitude so the point is is that it's an order of magnitude estimate it doesn't have to be perfect exact byte counts and by setting things like these properties as well as properties like quality of service this allows the system to optimize when network calls are sent back to the server based on the network conditions of your users device finally once you've configured an operation group you can add an operation to it by applying it to its group property now I mentioned the CKD operation configuration so let's talk about that for a second in the past you used to define properties like quality of service allow cellular access these long-lived on an operation that is now deprecated and instead you apply it to a CKD operation configuration and the reason for that is because we found that it was very easy to forget to apply these to all of the operations that you're issuing in your client so now you can set it up maybe once or a few times you can apply it specifically to an operation if that makes sense or hopefully more likely to an entire operation group which applies it to all of the operations within and that was that default configuration property on an operation group that we saw before okay so if we look back at that example log event from before let's call out some of the things that we saw around operation groups so here you can see in the log the operation group name so I can see this event happened because of my applications initialization logic you can also see it's unique operation group ID and the quantity property if you set it now at every event you can see the operation group that it applied to you can then see the specific operation that applies based on the operation ID and if it leads to multiple requests you can always see the specific request ID all right so with this new log feature and the things being exposed let's take a minute to talk about privacy as you know Apple cares very deeply about our users privacy but with cloudkit we also understand that you're working with a system that you don't own or have full control over yet you still need to debug customer problems so if both of those things in mind here's how it works as a reminder when you log into the cloud get dashboard with your iCloud account you can see your private and shared data in your private and shared databases just like I showed but you cannot see the private and shared data of other users you can see the public data in the public database because it's meant to be public and you can now see log events for you and every other user however log events do not include the data so for example some of the events we would see in the case of this to-do list app you could see that my account was issuing record modifies but you couldn't see that I was creating a list named vacation ideas or chores or movies to watch so we hope that this balance really allows you to debug problems as they happen while still keeping your customers safe and private okay so I mentioned earlier that we wanted to add the ability to share in this example application we want to let our users securely share a to-do list with other users so that they can collaborate as a reminder this was our data model we point items back to a list via the parent reference and we have the list as the route record in a share now we've added some the ability to debug and explore more of this in kaga dashboard so let's go take a look now on the right side of the screen I want you to look at the iOS app I'm going to tap into that top vacation ideas list record and you'll notice that it shows us some information saying that this is already shared so what happened was I created this to-do list on my phone which means it's in my private database so I'm the owner but I invited my friend Emily Parker to share this because she's a travel expert and I wanted her to give me some ideas on where to go on vacation so in the dashboard if we go back to the data section remember the last time we were looking at the records in my private database in the to do zone and if we fetched changes until we get to a point where nothing has changed the dashboard is up to date let's add an item in the iOS application so let's say I would love to go to Paris so my iOS app has stored that record as an item record back to the server and if I fetched changes in the dashboard again we see it show up I'm gonna tap on the record name and open up the record editor again we can see this is of the item record type that's in my private database in the to do zone and down under the field section we can see Paris the name that I provided in the iOS app now if i zoom in a bit you'll notice there's a section here labeled sharing and it tells us that this record is a descendant that means that it points to a parent via its parent reference the dashboard tells us the record name for that parent and provides us a little jump icon to load that parent up inside the record editor so let's do that so now we're looking at a list record and this is the vacation ideas list record within which we created the item in the sharing section we can see that this is a route record and a share it points to a specific share it tells us what that name is and we can open up that share so now we're looking at a cloudkit share record and down below you'll see you on the share you can actually see information about the sharing participants so in this case you can see me as the owner and Emily having accepted you can add and remove participants right here to test out how your app behaves when that data is modified on the server this allows you to browse relationships and if the record editor detects that there are any other types of references in your record it will also let you jump to them and we have a bit of back and forward history buttons up here in the top left so that shows you a bit of how you can view and navigate the data relationships going on with sharing let's take a look at how logs can help you out when you're doing sharing functionality in your app so I'm going to load up the logs let's clear it out let's create another item in my iOS application so let's say I'd also really like to go to Vienna Austria now I'm gonna hit return I want you to watch the dashboard log so first we see a request to come in for the record modify we can see notice my operation group name I'm using one saying oh this is logic for creating an item in the application it was a record modify in Dave's private database after that we can subsequently see that cloudkit sent a push notification to Emily it was in her shared database because of a subscription called shared changes that we set up on her device when she first ran the app the push notification made it to Emily's device and then her app turned around and fetched database changes and then zone changes and noticed the operation group name fetching changes after notification so we really hope that this will help you as you're debugging and trying to understand what's happening across multiple devices and potentially multiple users and sharing cases now in this case we were on the live log but we also have a feature called historical log this allows you to go back and view events that happen in the past we store events for up to seven days and you can do things like show me all the events that happened yesterday let's say between nine and ten and you can provide a bunch more filters and this will return to you all the events matching those filters alright the last thing that I would like to talk about is telemetry so last year we offered cloudkit telemetry and this year we've expanded the offering as a reminder telemetry is your way to understand aggregate behavior happening across all over your users devices coming into your container in this case we're looking at the development environment so we're seeing information that I've been playing with from this app one stage you can choose a time filter so you can choose the last year the last 30 days the last day or the last hour and let's look at the last hour since that's when we've been up on stage playing with this app you can ask for telemetry information that applies to everyone's private database every one shared database or just the public database and let's leave it on private you can also view telemetry information around all operations within that time window for the private databases and will tell you the specific types of operations and how many of each came in during that time window so you could say I would like to see time 'try information only for record modify operations but in this case let's look at it for all operations the first graph you see at the top is called requests this shows you all of the requests coming into the server again during the specific time window and for the private databases and how many requests for each type came in at a time slice notice it's called requests and not operations and that's because as I said before as you're issuing operations from the API in some cases that leads to multiple requests and we want to give you visibility into all of their requests so here we can see that I was doing record modifies database changes subscription modifies and how many of each we hope that this helps you during development understand how many types of requests you're sending in as you start to test and build your app and then in production and we'll see some examples in a second we really hope it lets you monitor the types of requests coming in across all over your users to check for things like drops or spikes that may mean there's a bug as you release new ad versions if I scroll down a bit the next graph is called server latency here we're exposing to you exactly how long again it took the server to process these events we show you the 50th and the 95th percentile and as a reminder you can jump in and see how long it takes for certain types of operations and the reason we want to give you this information is so that it helps you understand how long it takes to process the type of operation that you're adding to your application so that you can choose the right one depending on the experience that you're trying to build the next graph is error counts this used to be called error rate so we used to show you what percentage of requests failed now we show you how many exact ones failed and how many users had affected and the specific types of errors so in this case you can see at this time slice there was one error it was a conflict which meant I tried to update something that was already updated on the server and it affected one user so we hope that this information will allow you to detect when you might have problems in your app causing more errors and to understand what percentage of your user base it might be affecting and finally you still have access to your average request size which shows you how much data is coming into the server now again this was my development environment let's jump back to slides and take a look at some example screenshots of these graphs in the wild so here's a request graph from one of your containers in the wild and you can see on any one day they do about 2.3 million record fetch operations across all of their users in the private database and what's interesting is if you look at this scale you can actually start to see trends across weekday and weekends and so these developers could hopefully use this - like I said monitor for spikes or drops changes in this behavior as they release new versions and of course if you change how you interact with cloud kit then maybe you expect to change here's an example of an errors graph in the wild notice that they have about on any one day 1500 zone not found errors but it's only affecting about 55 users which could be a small percentage of their total user base so it might mean that there's an edge case in their application where it gets into a weird state where it thinks the zone exists that doesn't actually exist in the server we have CK error documentation that used to tell you all the types of errors you might see we've updated that to explain what each error means and how you might handle that in your application so the errors you'll see show up in this graph map back to that documentation we also give you some telemetry around the push notifications that cloudkit is sending for your application in the wild it's separated based on the subscriptions in the private public and shared database and this is sort of a sanity check if a customer complains that data is not synchronizing you can go here and feel good that cloudkit is still sending push notifications that subscriptions are still configured right in your application okay so what did we cover today well we've launched a brand new cloud kit dashboard we hope you check it out try it out please give us your feedback let us know what works what doesn't work what else you'd like to see there we've launched a new CK operation group API we hope you've started to see the the the help it can provide you especially when looking in the dashboard log to understand exactly what code led to what network requests being but sent back to the server like I said we want all of your feedback it really does guide what we do so foul radars post on the forums let us know for more information here's a link back to this specific session and we also have an email address cloud kit at Apple you're always welcome to reek it reach out and ask us any questions you have and with that thank you very much have a wonderful conference

Channel: beBetterDev

Views: 796

Keywords:

Id: iTeXsh9uPDs

Length: 37min 35sec (2255 seconds)

Published: Sat Sep 02 2017