Apollo Client Deep Dive (Ben Newman)

hello said on cool just gonna take a picture of you guys wow what a room freemasons huh Oh welcome as people are filtering in I will say some introductory things about myself and the title of the talk which is on the screen and we talking about Apollo client caching and client graph QL caching in general and in depth and my name is ben neumann I worked at Apollo formerly meteor development group I'm tech lead of both Apollo client stuff and of the meteor JavaScript framework which still definitely exists and sort of my baby these days and yeah you can find me on social media using the name Benjamin without and I github Twitter Instagram Facebook and others in case you want to get in touch and you don't get a chance to talk to me in person after this all right you're ready to go great so I believe in at least occasionally interrogating questions that are supposed to have obvious answers so I think for this talk as an overarching theme I want to raise and then resoundingly answer the question why is it so important to even have a graph QL client after all you still get actually quite a lot of the benefit of graph QL if all you're doing is sending raw HTTP POST requests to a graph QL server you can use the curl command line tool you know not quite as pretty as client query but you get back JSON data just the same you can also use the w3 what would G never know how to pronounce that the the fetch API that's now standard in most browsers and this is just built-in you know so there's like no runtime library to run if I come over here to the dev tools I can even use sort of a prototype level of wait and see the data that came back live from that that request right and so this hopefully I can just reopen that whole thing no I'm gonna start playing my soundcloud that's great well it's a link to my soundcloud okay here we were you can use fetch and you know in some manner is speaking this is a graph QL client write a bare-bones client that is not taking advantage of any of the fancy features that we've tried to build into the Apollo client but it works and you can try it just directly in your browser console without loading any libraries it's definitely better than rest in a lot of ways right you still get just the one round-trip instead of many there's no under fetching or over fetching there's one endpoint and it's endlessly flexible instead of a whole bunch of different rest you are all tabs there's a predictable result structure right the data that you get back is going to be a tree that you know basically corresponds to the query that you sent to the server and the server of course is doing all the things that it does it'll enforce schema types and run resolvers for you even support introspection so that you can use fancy tools like graphical write which is not a bare-bones tool by any means but it's not something that you would have to end up using in production so you could use it you know as a development tool in lieu of having a real graph QL client and then in production you could use fetch and you'd be cooking with graph QL just in sort of like a reduced way so that floats your boat it's actually not such a terrible world to live in but definitely worse than rest if you're not using some sort of graph QL client is this one really big issue which is the graph QL HTTP responses since the requests are typically post requests and not get requests so there's like a post body instead of query parameters they can't really be used cashed in any granular way by the browser at least not by generic HTTP caching so you might be able to sort of put your query into a get request and then like cache exact responses based on exact queries maybe but really nothing more useful than that and that's kind of a shame because one of the you know major benefits of rest is that it has a consistent carefully thought about caching model that's based on HTTP so because HTTP caching turns out to be mostly useless for graph QL graphic you all clients if you're using one need to re-implement caching from scratch themselves and because they get to do that in the context of graph QL with all of the assumptions that they can make because of that you know single solitary purpose it actually becomes a huge advantage that you're building something that is graph QL first rather than sort of like HTTP browser first so for example you know if you have a query that happens to be a subset of some other query that you previously fetched and stored in the cache you might not even have to send that smaller query to the server because it's already in the cache and if you have a query that contains some other query you might be able to you know pull out the part that you've already requested and end up sending a smaller query to the server which could be cheaper there all sorts of queries that are logically equivalent to each other according to the rules of graph QL you know depending on how you use fragments or aliases or how you order your fields and you know string wise they don't look the same so it'd be difficult you know to catch them based on the query syntax but because they're logically equivalent to each other the cache should be able to handle them in the same way so you end up with this very flexible graph QL data structure aware caching system and that's all because we didn't have any other option we had to build exactly the right thing so just to drive this point home a little further I hope I can convince you that only the client really knows about fancy things like client resolvers optimistic mutation results local state that you've written manually to the cache for reason the contents of local storage or indexdb so you know your graph QL server can't help you there and again generic HTTP caching just doesn't work here so for all of those reasons it's a great thing to have a graphical client that performs graphic you will aware caching for you the more domain-specific your caching system is more efficient it can be so you know although there is an investment of engineering effort and education understanding in picking up a tool like the Apollo client we do that in order to tame the complexity of you know all of our existing data sources and that's just one of the ways that graph QL allows us to tame that complexity on the server side of course schemas serve to sort of elevate raw data by enforcing application level object types and relationships if you've written a schema you know that that it's an exercise and kind of smoothing out the sins of past data models and when it comes to writing your resolver is you know you get to do whatever you want to combine existing data sources and you kind of hide implementation details of the backing store but resolvers can misbehave and so chaos into your graph QL system so here's one of my favorite examples is that legible even remotely this is a website called faker QL and unfortunately it's one of those websites that allows you to embed it in an iframe the point of faker QL is to provide a schema that you know consists of like users and products and to do's and posts you know generic sort of stuff and anytime you issue a query for say the first three posts author's first name and last name sure enough you get back you know some data it looks plausible but if you issue the query again no matter how quickly you do it you get you know totally new fake data right so this is sort of weird it's like there's not actually one true graph anywhere right like this is like a solipsistic version of a graph QL server that just kind of like you know feeds you stuff and here your simulation just because you asked for it and not because anyone else will ever care about this data so that's one example of like a implied world view that sort of doesn't line up with caching that we might want to do in the client right it wouldn't make sense to hang on to this data assuming that it was going to be the same if you ask the server for it again because he would just get new data so that's faker QL I love it I think it's a great little tool so schemas do impose useful constraints after all faker QL does have a schema but they don't guarantee that your graph is stable or a coherent right you still have to have like a real reasonable data model that lives somewhere that gets realized in your graph QL server in your resolvers so as another example this is this is not actually an example from production but it is an example from a reproduction that was submitted via code sandbox on our issue tracker where a person was you know reasonably just trying to mock out some data and ended up doing it in a way that kind of broke the data model accidentally so these type two halves over here this little schema that's perfectly normal nothing wrong with that these resolvers you know you can see how it's not fetching data it's just like returning some some constants except in the document current version field the ID anytime this resolver is called is recomputed it's a new universal unique identifier or UID right so the problem in this app was that there were two different queries on the client that both talked about this current version and one of them asks for the download URL of the current version document and the other asks for a list of editors and the problem was that the Apollo client cache was never able to like merge those together so that there was one document that had both a download URL and a list of editors right and why is that well these two queries got sent separately and that current version came back as a you know totally different thing each time so there was never any single current version object that the Apollo client could determine as sort of you know having both of those things in it the IDS just kept being new so it was as if new objects were being added to the underlying data model all the time so you know if you go a little too light on the the mocking in terms of imagining a real graph then you can end up in situations like this where the client misbehaves because it's making some assumptions about the stability of your your data graph okay so just pointing out again problem there which should give you pause because that's bananas okay so if query results can be different any time resolver is called how can it be safe to cache query results on the client if you knew that your graph QL endpoint behaved or more to the point might behave like faker QL for example how could you justify cashing anything so for me the crucial insight here was the job of the cache is not just to predict what the server would say if you reached a given query right now it's not trying to approximate what you know it thinks the current state of the world is instead the cache is sitting there receiving these little like you know partial query tree structures as it receives results for the queries that it sent to the server and it has to kind of ingest those and build its own mirror of your data graph as far as it understands it based on the results that it's received so I can't take credit for this diagram but these are the results of two different queries for a book which as each of the books has a different title but happens to have the same author Thomas Piketty ID five and die fat Pandya is a former summer nips intern at Apollo who wrote a great blog post that's linked on this slide all about how to how to visualize graph QL a couple of summers ago so what's happening here is that the results come in and that author appears in two different branches of two different result trees but because the cache understands that ids are uniquely identifying it's able to extract that author and sort of hoist it up to the top level of the cache and this this process of having sort of a flat list of all of the uniquely identified domain objects is called normalization and it's one of the things that well it allows logically related structurally similar queries to mutually benefit from the cache even if some of them were never actually you know sent to the server it also allows because the the data model has sort of been simplified and flattened it makes it a little bit easier for the cache to take in new data and use that to update any sort of queries that you might be watching to show you the the latest version right okay so this is different from other data management solutions that are often used in the client for example redux which sort of allows you to store arbitrary data which can be a good thing but if you wanted to do this sort of normalization to you know speed up live updates or you know handle data that had cycles in it then you would have to do that normalization by hand you'd have to maintain this flat list of uniquely identified objects by hand in redux and I guess another benefit of this system is that the the data really does get sort of ingested into a private representation that you know can't be messed with except through the mechanisms of of graph QL so you know you could compare that to redux where it's a little bit easier to just you know arbitrarily modify the data that you've put in your store so as people are thinking about you know possibly moving off of Redux on to the Apollo client cache these kinds of trade-offs are worth thinking about and again I really do recommend that graph QL concepts visualized blog post by die fat so here's an example of the Apollo dev tools visualization of what the cache looks like and in that column with the dog colon IDs that's the beginning of a flat list of all of the objects from the pups to Graham code sandbox example app alright so once those previous results are decomposed into this flat normalized graph structure the cache can respond to any variation of any previous query as long as the data are available and that flexibility that ability to like sort of ask what if questions what if I sent you know this query instead of the query that I actually sent a couple of minutes ago what would I have gotten that flexibility remains valuable even if you know since then some of the data have become slightly stale and in fact I think just as a matter of principle the decision to continue using stale data or throw it away and try to update it ultimately belongs to the application I've worked at a couple of different companies that had you know live updating systems and so I know from experience that forcing data to remain always up to date as a matter of like you know automatic policy eventually becomes a performance liability it's just you know too much traffic between the server and a bunch of connected clients so solutions to staleness need to prioritize data that matter the most to the user because you're not going to be able to force everything to stay up to date all the time okay and the the upshot of all this is that because we're cashing your data based on its structure instead of syntax of queries the data fetching workload of your application will grow only with the amount of data that's needed not with the number of queries so you can you know keep adding components that have their own queries and those queries are going to overlap to some extent and the sum of all the sort of combined data that they need is what you're responsible for fetching from the server rather than you know having a separate HTTP credit request for every component that has a query right and so just to round it all out that's that's my case for why you really want to have a graphical client and in particular a graph QL client that does caching for you the Apollo client cache is the the hub of that coordination she was picked for my dogs Beckett inspector Beckett on Instagram and I love this texture because he it looks so happy but we're actually at the vet and he is about to get some shots and he doesn't know that so there's some dramatic irony there so I thought that would be a good way to lead into the next section of the talk so everything I've said so far although I have you know used the word Apollo a couple of times is pretty generic in terms of caching and why you would want to graphical client so if you go step further and ask the question which cache implementation should you be using well I have what may be news to you there is a choice you've got a pretty good choice on your hands in fact I would almost say you can't go wrong we have this Apollo cache abstract interface that anyone can implement and they have now I wouldn't say it's been a like Cambrian explosion of different Apollo cache implementations there's there's two of them but two is more than one and as you ask yourself this question of course you want to consider you know what aspects of performance really matter to you maybe you really want to avoid network round-trips and probably everyone wants to avoid network around chips you want to avoid unnecessary rendering if you're using a view framework that you know is based on rapidly rendering you wanna efficiently update and you watch queries when new data become available it might not be so important if maybe you have like one master query compared to a bunch of separate queries but definitely something to consider and if you're you know working in resource constrains devices like react native Android phones memory footprint might be especially scarce or yeah okay so the implementation of the Apollo cache that we maintain is the so called in-memory cache there's the documentation for that it's an NPM package with that that name the other one is a cache that in McCloud gave a talk about yesterday in this room called Hermes and I one of the things that I really appreciated about Hermes is how upfront they are about the trade-offs that they've decided to make in the name of performance so they try to return just direct references to result objects that are sort of already realized in the cache so that reading from the cache requires almost no work in most cases and the trade-off is that if data has been merged from multiple queries into the same object you might ask for like only some of the fields of the object but you could get back an object that actually has more fields than that right but as long as it has the fields that you asked for then you're probably fine especially if you you know understand this trade-off and that that might happen when you know the talks go and why initially definitely check that out ok so I have mixed feelings about what I might call bench marketing I will say that this very small blurry list of benchmarks is thanks to the work of Ian and his team at convoy who who put together the benchmark suite the the first column on this side is Hermes so you can see that it has you know been doing pretty well in terms of performance which I I think is actually because it is an efficient implementation and you know certainly not just because author of the spin Forex wheat works on it the two middle columns though are the previous and current versions of Apollo cash in memory so that's the relative difference between being in last place basically in every category and being you know sometimes in first place and sort of neck-and-neck with Hermes that's that's what I'm proud of the difference between the the two middle columns and then the relay cache is often the furthest column and that this graphic makes it look like it's you know slower than the others in most categories but I think it's only fair to say that like I don't think anyone from the relay team has spent much time interrogating these benchmarks and you know it's possible that they could do something that you know improves performance if they if they took the time so I just want to focus on the difference between the old menu Apollo and memory cache so how did we do that how did we read those really pretty massive benefits well there's a big PR on github almost 100 commits maybe five months of work for me and a lot of feedback from the community which was great people validating that it actually was faster you know there's a question of like what where clothes actually happen in practice and it was able to answer that only because people with relapse where we're giving feedback so you know this this may be a review but there are only so many ways to make existing software faster right if you if you have the task of speeding something up then you know you can just go through this list and see if any of these things are options you can stop doing unnecessary work you can do the same work faster maybe by picking a different algorithm or three and faster hardware or using a different programming language and parallelize the work if it is actually paralyzed Abul prioritizing work that matters now is a great trick for perceived performance reusing the results of previous work where caching is my personal favorite or maybe my second personal favorite after just lowering your standards and not really worrying too much about the accuracy of the results a lot of things are possible if you give that up so that's not what we did here though when you're trying to reuse previous work it's often called caching there's several challenges I tried to use the same suffix for all of these words and it sort of worked staleness is the question of you know what the shelf-life is of a certain computation how long you can hang on to it before you need to force it to update sameness this is the question of like if you have two queries that are superficially different but actually turn out to be equivalent like you could you know cache the same value for both of those and that would potentially be valuable but you know to a certain way of looking at them they might seem totally different and so you you would miss out on that opportunity if you just recomputed the value of what I'm calling creepiness is this phenomenon where you know you've cashed a value and okay you found out it's stale that's great but it's already kind of like seeped its way into the rest of your application right and so if you don't have a plan for sort of refreshing all of that then you may end up only refreshing the one thing but still living with the consequences of the stale data elsewhere and then whether the caching is really worthwhile is an interesting question any caching system is going to have some overhead and if that is more than the cost of the operation that you're trying to cache it may just not be worthwhile okay so just to be totally clear what we're working up to here is building a cache on top of a cache and it's no joke so what happens the idea that started this whole process was that when read query gets called the cache you know needs to extract a tree-shaped query result from the normalized graph and this is one of the most performance sensitive parts of the cache in large part because we have this technique of calling broadcast queries that needs to reread every watched query after any update and so that's just a really hot path for the read query and so the idea was what if we just hung on to those result objects after the first time we computed them instead of recomputing them from scratch so initial reads wouldn't really be any faster but repeated reads could be nearly instantaneous so calling broadcast queries for example with lots of query observers would be a lot less expensive just because that would probably be a lot of repeated reads rather than fresh reads so to implement this I'm gonna have to sort of just refer to this and welcome you to ask me questions about it later if the details sound interesting but I brought in a library that actually develops awhile back for some similar problems in the meteor framework called optimism and so here's a toy example of caching that a lot of people use when they're talking about caching or memoization we're sort of a naive implementation of the Fibonacci sequence it's doing a whole lot more work than it needs to be doing because it's exponential instead of linear and so you know like a lot of memoization libraries this optimism library has a wrapper function that you can use to wrap fib so it just remembers the previous results based on the input number and then that just you know basically solves the problem so that you can compute the 78 Fibonacci number in you know oh 78 rather than 2 to the 78 okay so this is a less trivial example and this is more like what I originally wanted to solve in the meteor case meteor was doing a lot of file i/o and we were just you know hitting the file system all the time and I knew that a lot of that could be cached because the files weren't actually changing but it seems dicey to try to you know take a function like this it's trying to compute a hash from the path of a directory by sort of recursively computing the hash of all the files and directories inside of it and just like you know store that hash result based on on the path right the past doesn't tell you anything about the contents of the files right oh just you know try to think about what it would take to safely hold on to a previously computed value my my sketch of how this would work is that for each of these functions like read directory and hash file and hash directory and even is file and is directory you would need to have some strategy for finding out when you know the results of those functions were likely to have changed or might have changed and if you knew that if you had a way of getting informed when your data had become stale and you could safely hang on to the original data because you wouldn't just be blind to future changes so wrapping the outer function with optimism is not nearly enough that's a little bit like trying to draw the cartoon character the tick by starting with an oval but in the line through it and drawing the check holding the oval this slide if you look at the slides afterwards sort of explains the whole optimism API so not only can you wrap a function which gives you nice memoization based on the input arguments automatically but you can set a limit on how many cached values there are you can define a function that sort of transforms or reduces the dimensionality of the the inputs this make cache key function it just needs to return something that can be stored as a key in a map and then there's a subscribe function that you know I came up with a couple of years ago but now seems pretty reminiscent of like the use effect hook that react has introduced in that it returns an unsubscribe function and whenever the file changed notification for this particular function or whatever it is happens you call the dirty method of the cached function with the same arguments and it figures out what the sort of parents were in the dependency tree and reports the fact that that value may not be dirty so the parents themselves may get recomputed the next time you call functions so it's a pretty neat system pretty small API and has had a big impact in more than one project now so this is just an example of how you would set up all of those subscriber functions that we don't really have time for okay so how does this apply to a publication memory well the most interesting thing about this is that we have this concept of a normalized cache this abstract interface and there's now an implementation called the depth tracking cache and it's typescript so some some type dance in the constructor we create this sort of dummy wrapped function it doesn't actually do any expensive work but it exists to help register dependencies on certain IDs in the cache so that when we interact with this depth tracking cache by calling the get method we depend on an ID when we call the set method if the value is different we dirty that ID and similarly with delete we dirty the ID after deleting the object from the cache and the great thing is that nothing changes in this depth tracking cache without these methods being called so this turns out to be an extremely convenient bottleneck for the dependency tracking that we need to do and so when it comes to reading from the cache this example sort of demonstrates that we took a method that we implemented in a uncashed way and wrapped it with optimism and then went to some trouble to come up with a fairly sophisticated make cache key function that sort of simplifies this mess of inputs and to just the few things that uniquely identify the value that we're trying to hang on to in this case the store the query object the selection set which is contained by the query object the variables and the the ID of the object that we're applying the selection set to and so what that gives you is you know you you may have changed some of your data but when you read from the cache you're going to get back a tree of result objects many of which will be triple equals identical to objects that were previously returned so if you're passing that data into react components say you may be able to take advantage of the fact that not only has the you know the data not changed but the literal objects that compose the data haven't changed and you can just avoid rendering very cheaply so I'm gonna talk about these internal optimizations sort of inside baseball and we're already out of time but two things that you should keep in mind as you use the cEPAL ocassion memory package you should take advantage of triple equals equality if you're using react then the pure component class or the new react memo Combinator can help and I would recommend not modifying result objects from the cache but if you do you should write them back immediately so that you no other consumers of the cache don't see your temporary modifications the other thing is try to not use an unbounded number of query documents don't like reparse a query string repeatedly because some of the optimizations rely on one-time pre-processing of the query documents so those are the two main recommendations it's been interesting to see you know how many people were doing things like this that I didn't expect and to decide what counts as a bug and what counts as a you know something where user behavior should change so what are the future plans here stability is really the big one this is a huge change right 100 or so commits in 1 PR and I'm promising we're not going to make any more huge changes until the dust has completely settled so you know I would really appreciate your help in updating in-memory cache to the latest versions yeah broadcasting queries needs some more work there's a whole host of features around invalidation and garbage collection and deletion that I could give another whole talk about and it seems like now that I've used it for two you know fairly serious projects it's time to actually write some documentation for that optimism library so with that thank you for indulging me in the extra time and I don't know follow my dog on Instagram if you feel like it [Applause] [Music]

Channel: Apollo GraphQL

Views: 11,077

Rating: 4.8048782 out of 5

Keywords: GraphQL Summit 2018, GraphQL

Id: Vjf7sjgMqfk

Length: 35min 32sec (2132 seconds)

Published: Tue Dec 04 2018