droidcon NYC 2018 - Coroutines by Example

all right I can't see any of you but I trust that you're there and hopefully you're here because you want to watch this talk on coroutines by me Christina Lee I am an Android engineer at Pinterest which is about the only relevant life fact I have for you and I just want to let you know that I'm I really had great grand aspirations for this talk when I submitted it of doing all examples just all the examples everywhere and then there's this thing that happened which is that smarter people than I am pointed out the flaw in that and this guy is dead but people still team seem to take him very seriously and being empty or blinded my work didn't seem exactly like what I was going for but fortunately I'm a math major so I know something about combinatorics which means that while we ruled out only theory and only practice and SF rolled out neither there's still Theory plus practice which is great so so this is this is a you know seems like the right track and you know they didn't actually specify what that was they were saying the thing about blind about empty nobody actually said what this would get us and so just to fact-check i went to google because that's where you get all the answers and it turns out that if you combine theory in practice you alternatingly get a derivative form of chutes and ladders a center puzzle piece with no external pieces a giraffe being chased by a lion on stilts or my favorite and uprooted bonsai tree that's been graffiti'd by a bird now all of these seem to vaguely positive so I was pretty sure I was on the right track which is why this is exactly what the talk is going to do it's going to start with five facts that you need to know to understand the rest of the live coding that I'm going to do and then it's going to go into live coding and that live coding is on a sample app I didn't want to give toy examples I wanted to work in a real app using real data and so it is going to be that now I have a level of excitement right now that goes along with doing really dumb things so let's all just pray that the live coding goes planned but this is this is what we're out here to do so probably should have better been called this but anywho we'll get started now how many of you all have used co-routines before okay so a small smattering but the majority haven't so then I won't skip this section but I'm going to go through it rather fast which is the why of co-routines and it starts like this threads are expensive we know this because we have thread management and specifically we have threes like thread pools where we don't want to create new threads so we know that that's vaguely problematic co-routines are state machines which means they have three key things they have the data for each state we have an index of the current state and then they have the ability to wait patiently which lol my mom is still trying to figure out how they got that and I didn't but you know this is what they have and it turns out that small objects are a lot easier to create than these entire thread environments and so it's it's less expensive to do this and what that means is that we're not getting rid of threads but instead we're distributing these small objects across the threads in order to better utilize their capacity this is a potato chip dispensary in case you are wondering and it's actually a really great analogy for co-routine dispatchers but we'll get back to that later okay so then there's the second part that JetBrains really likes to emphasize as well which is function signatures and it goes a little bit like this which is if you ask for a model and you get back a bottle that makes sense if you ask for a model and you get back an observable it's okay and I love observables they're great they have a lot of stuff going for them but they're their own unique person and you need to get to know about them and their likes and their dislikes and that can lead to a lot of this during that learning process and so if you have been using Rx for a while something like this might seem fairly reasonable but if you have not been something like this probably seems a lot more familiar right off the bat and what this boils down to is that with co-routines functions have the ability to stay the same quote unquote which is to say that you localize the change to a single point time instead of passing a new-type throughout and all of this can be summarized by one less thing to learn TM okay so that's the ninety second primer now onto those five things that you need to know before we go into live coding first co-routines are a form of continuation passing now there's a great talk on this that reminded at kata Ling comp so I'm not going to go into it the only reason I want to point this out is because there's something that happens that goes like this you write a suspending function called get models and what you get back is something that's get models with a continuation specifically there's the continuation and oh look it's being passed all year so why is this problematic well because you find things like this which is that when code expects a certain function signature the fact that a continuation gets inserted can be problematic now this is the synopsis which is that it is a form of continuation passing why do you care you need to be careful with libraries like room that expect a certain signature there's already BEC filed for this so don't worry but it's something to be aware of that leads us to important thing number two which is that co-routine builders are bridges specifically suspending an on suspending code look remarkably alike however if you try to use them in the same way they are not remarkably alike specifically you get errors and you get this error saying that you could only use suspending functions from other suspending functions or from covert Zients now you you can keep doing as many suspending functions as you want but at some point you need to use the thing and so that's where these co-routine builders come in so if you are in synchronous city and you want to go to suspend Apple Township you would use a core routine builder that is how you get from one part of your code the normal code into your co-routine code now there are several flavors of this there's run walking there's launched there's async they're pretty self descriptive for instance launch is something that's fire-and-forget but you know these are given two out of the box and the main point to remember about them is that they are the entry point into your suspend a bull code this is what we need to take away from this and that leads us to important thing number three before we live code which is that co-routine dispatchers direct traffic to threats now hopefully this name makes sense they're dispatching things to the appropriate location and there's this definition of it but I like to work with concrete things and so instead of going through the definition I'm just going to use this example which if you use rx before hopefully this will look vaguely familiar what it's doing is it has some repo and it's fetching something from the network and then it appends to subscribe on here how would you do this in co-routine land well it would look something like this you have your covert team builder and you're telling it to operate in the common pool and then here is your call now common pool and subscribers that i/o are not the same thing and so you can actually do something like this if you have co-routines rx2 and what you're doing is you're taking the scheduler Stadio you're adopting it with this extension function and now you can use it again so this might be more analogous but the concept is the same which is that you can still tell this thing where to begin its execution now we don't usually stop with SUBSCRIBE ons we usually also add these observe ons and of course co-routines have you covered there as well so specifically what you get is you now have this with context block so at the top we're still launching with IO and then when we get that data back and we need to bind it to the UI what we do is we say with context and then we tell it to operate on a UI thread okay there's one caveat here that I want to point out which is that when you are in rx land you have this notion of upstream and downstream so subscribe and affects things upstream observe on the effects things downstream until the next observe on yada yada yeah in co-routine land it's not like that it's all based on blocks so here we have this launch block everything in this block is going to happen on the i/o dispatcher unless we specifically enter another block where we've specified something new so inside the with context UI that'll happen on the UI thread everywhere else it'll happen on the I dispatcher so this is vaguely different from our Xstrata but hopefully still makes sense okay there are a few that you got for free these are built in but you can also make your own or as you saw earlier you could adapt existing things to to use with your Co routines now there are some caveats about all of these different things some knowledge that you need to acquire turns out threading is hard I don't know who knew that already but lots of thing and so I'm not going to go into it because again this is focus on the practical but there are plenty of blog posts that do go into this specifically I really liked this one by Chris Bane's and I would recommend reading it and we actually utilized some of the concepts here in the sample app okay so out of all of that what do I hope you walk away with well basically that subscribe on is to observe on as initialcontext is - with context now I'm not sitting here and saying they're one-to-one but they're analogous if you've been paying attention it's after lunch so that's a big if but if you have been you might have noticed this word context popping up everywhere and we were talking about dispatchers and then everything started being context so I didn't mention it and that's a little sneaky of me and it was probably time that I actually explained what this is and specifically why I can say with context dispatcher because those don't seem like the same thing except for they are spoiler alert so dispatchers are co-routine context which leads to the question what is a co-routine context important thing to know 3.5 snuck this one in there for you quarantine context they at context I know that's really silly definition but it does exactly what you would expect exactly what you're used to in activities and all of your other Android code which is it adds relevant context and so specifically what can you do with co-routine context you can pull out the name which might be useful for debugging or you can get the job which might be useful for canceling it or checking the children or its status and if you're me you can also create your own by giving it a key some dubiously useful function and then going to town so this would as expected just print out the first five Fibonacci numbers but if you were really ambitious you could do something actually useful with this like say pass an exception handler so there are useful things to do or you can do your Fibonacci generator it's up to you I'm not judging so this leads us to our penultimate important thing to know before live coding which is that you need to cancel things so analogy time again disposables are basically akin to jobs that's it so if we go back to this example which we are using earlier we had this subscriber but I was being a really bad developer in practice what I really should have done something like this and then eventually I probably should have done something like this or if I'm being realistic I probably would have had some sort of composite disposable that I used tied to a life cycle method well co-routines have you covered there specifically in that when you launch one you can get a job back and once you have a job you can cancel it so you can also do a lot of other things with jobs that I'm not going to go into but one thing to note is that you can also get the job from inside the co-routine context which is actually pretty useful so that's a fun little extra that you're getting co-routine land now most things are the same but there are a few things that switch over like I was talking about composite disposables earlier we don't have composite disposables in co-routine land but we can make do with parent jobs and the like which we'll go into in the coding section but basically things are pretty analogous or you had disposables you know how jobs there cancelable take care of them as you would and that leads to our last item so co-routines are synchronous by defaults this is somewhat of an unexpected statement given that most of us associate Co routines with asynchronous work but this is true nonetheless and let's take a look at what that means so this is how I represent code I'm not sure why but it is and let's say that this is our normal code path and we have this one job that's executing linearly then we have a covert team builder it starts this Co routine code and the important thing to note is that both of these are proceeding linearly we have two tasks they are operating in a linear fashion specifically what this Co routine code is not doing is something like this where it has a bunch of different things happening in parallel all while that normal code is executing linearly now you can make this happen but you have to opt into it and you have to do it explicitly by default this will not work it will not be in parallel unless you choose that there's a very good reason for that which is if you want to do a concurrency I really hope you've signed the waiver and agreed that your code probably won't work it's not going to be reproducible and you're gonna ask 10 people to look at and they're not gonna have an idea either so if you want to get into this situation you really want to do it knowingly which is where that requirement for being explicit and opt-in comes from what does that look like in practice here is the smallest example I could give you what does it do it has a function that takes ten seconds to complete it has a function that takes ten yeah ten seconds and five seconds and then I time the whole method predictably because by default this is running linearly you're gonna see that the result of this is it finishes the entire 10-second task and then it finishes the five-second task and the whole method took ten plus five equals fifteen seconds so there it is the five-second tasks wait for the ten-second toss to be finished now I can make this asynchronous if I so choose but I have to be explicit and I have to opt into it here we do get that parallelization that you would expect specifically you'll see that this five-second task now prints first because five seconds is much faster to complete than ten seconds and the total time of this method has now been reduced to ten seconds which is the time it took for the longest element to complete okay so those were our five things basically the continuation passing style is important when you work with libraries like room co-routine builders are going to be the entry point to any function that we write that you justice suspend keyword so every time I write that in the live code Belle should be going off dispatchers are how we're going to tell this where it should be running cancellation is going to work via jobs instead of composite disposables and with synchronicity every time I try to do something that I would like to be asynchronous or parallel I'm gonna use this keyword and I'm going to opt into it alright you have everything you need it's time for the code a little bit about the sample project I'm going to be working on it looks something like this specifically I'm hitting the unsplash developers API I'm pulling all of their trending photos I'm putting them into a recycler view and then I'm also adding a caching layer and some user interface where I can click things to change their state and persist them so this is it we have networking we have different types of flow bowls and completable and we have caching with that let's get to it and switch over here so of course I did not leave off at the right spot because that would be way too professional with me so here we go I'm gonna start with the networking calls and show you how to adapt those and fortunately for me there's some low-hanging fruit that I can begin with specifically in here I had this call adapter factory for rx and I can go ahead and switch that out to 1/4 co-routines this work has already been done for me thank you Jake as always I don't know where he finds the time but he does and so now I can go in to my service and I can change this return type so here I had a single and now I want to return the fir'd so great that was the lowest hanging fruit we could possibly start with we added our call adapter we change the return type now it's time to go look at the call site for this predictably everything has broken because we're returning a different type we used to have this rx stream now we don't and so we can begin by destructuring this first we're gonna add in our suspend keyword and then we're gonna take this line by line so the very first thing it does is it calls the unsplash service to get the curated photos so I'm just gonna say curated photos equals the unsplash service get curated photos and then I'm gonna keep this logic exactly the same the next thing it does is it subscribes I'm gonna pass over that right now because it'll happen somewhere else and so then I'll move directly on to this flat map well in co-routine land we don't need to do this flat map we can just call it immediately so insert if not present and curated photos now this error here is saying that the types don't match up and this goes back to that point about having to opt in to the asynchronicity this is returning a deferred object I need to actually ask it to wait for the real object the list of photos to be produced now you'll see that that type error has gone away because I have a list of photos here I'm waiting for the network to complete before I do this next database call all right so that's the flat map code and then of course we have something down here which is the on NEX and the on error we can do something similar if we want I haven't really added a lot of code here so mostly it's just to print lens but if I want to add them I can do something like this and I don't know maybe I want to look for errors of this nature and then go in and move this again the prisons aren't doing anything useful I just want to show how you can use logic like this once you change the type the same way that you would normally so now let's see we took care of this unsplash service we did this next photo datasource call and then we took care of all of the subscriptions great now the last thing we have here is this disposable and I'm not going to deal with it right now but I promise I'll get back to how you change this over but for now I'm just going to delete it so there we have it it's our first suspending function we D structure that our extreme one line at a time but now comes the fun part which is that we actually have to use our suspending function somewhere so we can go ahead and to that call' site and ding-ding-ding we have a normal function and it's going to call something that's suspending so the very first thing that comes to mind should be that we need a cover team builder so here let's use launch and once again let's go through this Indy structure at one line at a time so first thing that's happening here is we're calling photo data source that get all photos this is coming from the DB so I haven't yet converted it but we'll go ahead and say updated photos equals photo data source get all photos we'll leave that like that for now and then it has this observe on main thread so we'll go back here and how do we do that we'll call our with context and we'll use dispatcher stop me now this dispatchers object is something I called out to in the blog post which is just this object that I have injected via dependency injection and this is an idea that Chris Bane's had which I actually kind of liked but no magic is happening there that's the only point nothing magical it's just hiding away the instantiation so ok we did our observe on and now we get into this situation so if we get these updated photos what do we want to do well this is what we want to do we want to do this particular block here now these errors are coming into play because of course the types don't match this photo data source it's returning a flowable and what I'm looking for an updated photos is actually a list of photos so I haven't converted this yet but let me explicitly type it and then leave this us to do and we will return to it now this is all yellow because it's unreachable code but hopefully you'll see that is empty is no longer erroring because these types are checking out so I have done the observe on I've done the initial call I've done the subscription if you would like to handle the error you can I'm gonna skip it for now and then there's this disposable thing which again I will get to momentarily so I'll just leave this to do up here on the updated photos which we will convert in a second and there we have it we are using our suspending function from a normal function now this was not the only place that called this fetch next photos there's one down here but luckily this one is easier there's not much going on so we can do something directly like so and if you remember if you have a very astute memory I actually took out the SUBSCRIBE on from the fetch next photos call and so if I want to add that back I can do something like dispatchers dot IO because this is a network call so now we have that subscribe on logic coming back here the only thing we haven't really done yet is managed the disposables or what used to be disposables so we can go ahead and jump to that location and this is what we used to have this disposables object here let's get rid of it and instead let's create a composite job now the reason this works is because jobs form a hierarchy so if I pass one parent job and later on I cancel that parent job it'll get propagated down to all of the children and so this is useful if you want to use it with life cycles which is that you create this you pass it as the parent to all of your co-routines and then in the appropriate life cycle event you cancel it and it will then go ahead and cancel all of the children associated with it so let's go ahead and use this we can get rid of this disposable stock dispose and on cleared we can use our composite job cancel and of course it's not actually doing anything yet well there's one other location where we have disposable so let me get rid of that it's not actually doing anything yet because I actually need to pass it here and specifically I'm going to say parent job equals this composite job there's one other place that we have introduced a co-routine builder so I'll go ahead and do it here to parent job equals composite job and there we have it we have completely converted this networking flow what did we do we updated our call adapter we changed our service to return a deferred type we then utilize that in a suspending function and then when we need to access that suspending function we did it via launch which is a KO routine builder and we kept track of that work with our composite job that we could then cancel in a lifecycle alright now we can move on to the database work so let's just go take a look at what we're working with and it's something like this this is room I decided to use it because it seems pretty popular with all the cool kids these days and it looks pretty simple there's nothing ridiculous going on under the hood here specifically these two things down here are completable x' they're not really returning anything that I need here there's a long array but that's just whether it was successful or not I can take it or leave it I'm very specifically ignoring the top flowable and that's because that it's going to be a different case that I'll return to it's not quite as straightforward because co-routines they're very very easy to adapt code that is a single a maybe of our completable when we get into a flowable situation it's not really its wheelhouse Rx was made for these asynchronous event streams whereas co-routines this is a little bit of putting a square peg into a round hole so we'll return to it I'll show you how to do it but for now let's just focus on these two completable now there's nothing really to do here but we can jump to where they're being called and we can see that there's a completable specifically there are these completable x' and also the subscribe on schedulers so let's go ahead and dive in we can again say that it needs to be suspending and then we know that this needs to be subscribed on the i/o I which is actually I think a typo let's do dispatchers dot DB because this is DB work and then we know that we need to do something asynchronous so let's be explicit about it say async do the work insert all photos and then wait for this to complete now we don't need to return a completable anymore we really wanted to be accurate like we would be returning a unit but of course this is Kotlin so we can leave that off we don't really need it we could do the same thing down here which is remove this completable because all we need is unit at this point and then again we have subscribe arms so let's do our with context dispatchers dot DP and then we need to do asynchronous work so to be explicit about that a photo dot update like status pass it what it means and then wait for it to finish all right so what did I do here oh I didn't make it suspending ha okay so let me put with context back and then update this which is what I should have done first which is to make it suspending great we have two more suspending functions and now we need to use them and what are you thinking of course call routine builders hooray so let's go and use this so everything's broken as per usual with my code and we're gonna fix it so we know that this is a normal function we would like it to call something that suspending so the first thing that we'll do is we will choose sonko routine builder and then once again we'll disassemble this our Xtreme line-by-line what is it doing is calling photo datasource so we'll go ahead and do the same thing persists like status it's passing at these two parameters and then when it's successful it's printing this very fun emoji when it fails it's printing the fail whale so once again if we would like you can do something like this let's put a placeholder exception in please don't do this in your code but again oops this is just to show that you can indeed do stuff here if you would so like I am not actually Wow okay this is not necessary but you can if you want to do additional logic here so what did i do i already ported this i ported the SUBSCRIBE if you recall i deleted the disposable down here before so i'm gonna go ahead and add that back and so where do we want this to run by default well we can say DB it's already being handled in the photo datasource but if we wanted to we could do that here they actually dedupe themselves so if you start something on one context and then you say with context with the same dispatcher it'll it'll just check and then continue on you the overhead that you have is the one check which when you're doing something asynchronous tends to be pretty small in comparison to whatever network call you're doing but we can avoid it here for now since we already know that this is happening where we want it to happen and instead we'll just pass the parents composite job okay now you probably notice that we updated two different functions this one's a little bit of a cheat because it was already within a suspending function so I actually don't need to adopt it at all because again you can call suspending functions from within other suspending functions so that's it we've done these two database calls we've adopted them we've made them suspending and then used a KO routine builder to access what they have to offer so this is where we get to go back to our very fun streams example the one that I was avoiding like I said I'm going to show you how to do this without necessarily advocating for it because Rx and co-routines have very different specialties and it's it's a little bit of an interesting case for me to try to force co-routines into this reactive space but I've never had good judgment so I'm gonna do it anyways so there's nothing for me to do here because room can't directly return a receive channel yet as far as I know so instead I'm gonna go and work at this one level removed which is what we were doing here was just basically getting this flowable and adding the SUBSCRIBE on so nobody could accidentally subscribe onto this in the main thread instead of that I'm gonna continue to get the photo dot all but I'm going to convert it over to a channel using open subscription it's gonna yell at me because of course the types are wrong and that's because what open subscription returns is a receive channel now this was probably something I should have discussed in the PowerPoint slides but unfortunately I had limited time to do this presentation and so the short of channels is basically that things can be offered to a channel and things can be removed from a channel it's very much like a queue and so what's gonna happen now that I've done flowable dot open subscription is that every time that flowable would have fired it's going to offer this item into the channel and then me on the other side of this channel start consuming them and saying oh I want to process that I want to process that so let's take a look at what that looks like oops streams so this is the to do that we had from earlier now we had this to do and then before that we had this call here again the types are wrong we actually know that it's not going to return a list of photos anymore specifically we know that it's gonna return a channel so let's go ahead and type it as such get all photos this is our channel and now we're in this conundrum where we need to somewhere get these updated photos we have a channel but what we really care about is the things that are being pushed onto that channel the emissions which is this list of photos every time the DB gets updated so what are we gonna do well we're gonna use a for loop what I can do here is I can say updated photos in channel and then I can wrap this block of logic within this for loop get rid of the excessive space and now everything's working now that was probably a bit of magic but this is a suspending function so what this is doing is every time there's something on the channel it's going to execute this block and then it's going to suspect and then the next time there's something in the channel it'll pull it off it'll execute this block and it will suspect so this is going to continue to run this block of code every time something is offered to this channel now I just noticed a typo which is that when I was transcribing this literally from the our Xtreme there's this with context but fetch next page of photos should actually not be happening on dispatchers domain be pretty bad so let's go ahead and change this over here fix our spacing and now when we get our data back those updated photos we're gonna dispatch at our UI on the main thread and if we need to hit the network we're going to dispatch that on io instead of blocking our main thread so yay code quality this should look a lot better alright there we've done it we've done the networking we've done the DB the DB was slightly different but it really followed the same playbook which is that we marked something as suspending we knew that it needed to be asynchronous so we marked it as such and then we waited for it to complete using oh wait we then use that via a co-routine builder and we dispatched it to the correct location of operation using our dispatchers thought i/o or main etc now in this op I included one other example of dubious nature just because I wanted to show you what it would look like to do something that's one-off this is a doctor the photo adapter that's getting these photos and binding them to the UI and I wanted to use diff util somewhat just to make this case but you know it's something that's plausible in an Android app and so for our last example what I'm doing here is I'm creating this single I'm moving the diff view till calculation off the main thread because sometimes it can actually be pretty expensive and then when it comes back I put it back on the main thread and update the UI so what is something like this look like well we already know we're gonna need a suspending function so we can start their private suspend fun I'm really bad at naming we're just gonna go with creative and then once again we're gonna do structure our current rx code starting with this chunk of work which seems like the most important thing so let's pull it down here it looks like the two arguments that we need are the current data and the updated data so let's do current data and then this is gonna be a list of photos and then we can do our updated photos list of photos this should probably actually be updated data just to make it consistent and then we can change this over so current data alright so that makes it pure and now what do we want to do with it well we already know that we don't need this anymore we know that we want to subscribe on but we'll handle that later oh here's an observe on this is something that we can handle so let's go down here say with contact we'll say dispatchers dot me and then what do we want to do when we get into this well we simply want to assign this to our local field and then we want to dispatch it so we'll go ahead and take that we'll put it in here update it data and we'll dispatch it and this is all happening on me okay that leaves us with a little bit of cruft up here but no fear we have something that's suspending so we need we know we need a co-routine builder and this is gonna be following much the same formula as everything else so here we know that we want to dispatch this let's see what are we doing it's a computation so let's use dispatchers computation we know that we probably want to keep track of this so let's give it a composite job which I already cheated and defined and then let's go ahead and Creole call it create diff the current data is going to be in the photo adapter data the updated data is gonna be updated photos and let's check all of our bases so we have single dot creates we got our subscribe on computations we got our observe on main thread and we did our disposable and there it is so now I've gotten rid of all of the calls to our X in my app if I want to be bold actually let's see we have we have a minute left so I can do this let's compile it this is a terrible idea never like never do this it's ever bad bad idea and it's installing moment of truth is it gonna work oh and it works so you you can't see this but I can tell you I can also lie you can you can touch this you can scroll things are happening and if I had done this wrong these would be crashing so as you could see we just successfully removed all of the instances of our X from this app we converted it over to cover teens and the five things that were important to remember once again we're basically that when you're using continuation passing style you need to be careful with libraries that expect given function signatures we use co-routine builders to access suspending functions from normal code we have dispatchers in context to make sure that they execute where we want them to execute we can cancel things using jobs and of course a synchronicity needs to be both explicit and opt-in that's all I have for you today I'll be around at the back if you have any questions but I hope this was useful thank you [Applause]

Channel: droidcon NYC

Views: 10,990

Rating: 4.8393574 out of 5

Keywords:

Id: lh2Vqt4DpHU

Length: 37min 45sec (2265 seconds)

Published: Wed Oct 17 2018

Reddit Comments

[u/CrazyJazzFan]

Are any other videos from Droidcon NYC uploaded anywhere?

[u/Zhuinden]

I liked this talk, it was very accessible! Finally an example for ReceiveChannel.

Let us be thankful that the with(dispatchers.io) didn't end up failing on us

[u/phileo99]

was there ever a published link to the (GitHub?) project used in the video ?