GraphQL and ASP NET Core - .NET Oxford - November 2019

okay hi everyone let's talk about graphic we're on the high speed on a core platform and we will cover quite a few topics tonight okay this talk contains a lot of live demos expect things to explode and I hope you enjoy it so let's get started so graphic we're on the net platform was until recently not really existent there were some some libraries available but they had a lot of shortcomings compared to the JavaScript community so let's talk about what graph word is first before we get into how we can do it on they don't that platform so when I go to meetups or conferences and talk to dotnet developers about graphic where the first reaction we get is what the is graphed with and this is because mostly because there is not a great API support or there was not a great API support until recently so what is graph query in short graph query is a query language for your AP eyes and create the query engine to execute those queries and graph query was developed by Facebook in order to fix the needs to fetch data more efficiently for their mobile apps who did remember the first Facebook apps here for mobile devices nobody they were kind of crappy very slow and people hated them and they thought Facebook will vanish from the US because they were so bad and that's basically when Facebook started inventing graft where there were sweetgrass three guys at Facebook one of them mostly Byron and they came up with that so grass care was then open sourced by Facebook in 2015 as a specification document which was very quickly by the community and nowadays it's used by companies like Netflix Twitter even the New York Times and many many more so the biggest Internet companies today run on graph query so it's very better tested and super efficient so before we go into it I'm Michael Stipe I'm one of the cofounders of chilly cream and we are heavily invested to make graph work better on the.net platform so who is this top for this talk I say I mean that's what we do is user groups usually it's a we have the introduction so it's for everyone who wants to get started with graph query but we will drill down into graphic where look at things like how we can scale graph query and stuff like that so it it also will be very valuable to people that know graphic weather already so who here actually knows graphic well at all anybody okay who has used it really in products ok - ok so that's good because you always learn something new to that so let's start with the demo because I like presentations with a lot of code and I have prepared a little exercise here so let's let I hope everybody knows Star Wars because otherwise you will get lost in the example so I have a simple example a simple demo here we want to fit fetch all the characters that appeared in movies together with Luke Skywalker so just a list of names it sounds quite easy so let's first have a look how we would do that with rest so there is an API that called this Taoist API it's hosted at swap eco how I hope everybody knows it it's an exempt Latorre REST API and it's frequently used to teach clear to teach people clean REST API and rest is very easy to us dotnet developers or back-end developers in general because it basically drives the web today and if we look at such a query documentation we know how to use it we see it has a few resources which are exposed here through endpoints so ever people's endpoint here planets endpoint FM's endpoint and so on I can drill in into those endpoints by fetching them like I could fetch the people's endpoint and as I expect I would get a list of people here which basically are the characters in the movie like luke skywalker 3cp or and so on and I can even drill in further like I could upend the idea of a specific character to my endpoint and then basically get that specific object in this case Luke Skywalker and if I look at that JSON object here it's easy to read it has a name hides maths and stuff like that so it's very easy to explore that API and if I look further at the API have array films here and this films array consists of your eyes and these your eyes are actually references to other endpoints and I could drill into that and fetch one of those references and get the film's object again easy to explore and I can see there's a characters array again represented by references and I could drill in further and it's easy to see how we could get our data here so I have gathered all my JavaScript knowledge and wrote a little program here that basically now fetches the data for us so the algorithm that we would have to write to fetch the data from our REST API is quite easy we would first fetch our Luke Skywalker that's here then we would iterate over the then we would fetch each of the movies we would iterate of the characters and then fetch each character at the end we would do a little data massaging that's basically I'm pushing the character objects to array here and then we extract just the name from it so by just looking at it I can already sense that it's not very efficient or very performant because I'm fetching a lot of data that I do not need so let's have a look at my little program I hope Internet connections all right let's refresh it and see how that performs push the red button here and you can see that there are a lot of connections a lot of data pooling is already happening so we did 124 requests just to fetch basically a simple list of names and if you look at popular websites I would have said medium but they migrated to graph query but before them migrated to graph well and you had it over to medium to just read your block you had more than 100 connections just to fetch the data from the backend so that for what it did it didn't perform that bad it's just six seconds and we fetched a lot of data that we didn't need so let's have a look at how we would do that with graph square so for that I opened up a tool that is called playground so playing around is a tool to explore graphical API so it's based on grafica which originally was developed by Facebook and on the left hand side of playground I can write queries on the right hand side I will get my response once I fired a query up against an a graph where back-end and I have here two buttons one is at docks and docks lets me explore my API I can see my API has a field characters here and if I click on that I can see my characters field will return a type that is called characters and I can see it has similar data like like my REST API I for instance have a name field here I have a height and I have a movies array and the movies array is not made up of references but actually actually returns me movie objects and I can see that my movie object then again returns a character array here and I can drill in here so before I did my first request again my against my endpoint I could already explore my API quite surly so let's see how we would write a query so a query starts basically with a keyword query and then I write the query somehow like a JSON JSON structure but it is actually not a Jason structure so I'm writing here my squiggles and then the tool already helps me along by suggesting okay there's a characters field and the characters I field needs an ID and I can type in the ID one because I know you Luke Skywalker has the ID one and as I go along it suggests me things that I can use so basically I have intellisense and let's just fetch this character with the name okay that's interesting I'm not getting a full object like I got with rest I just got what I specified in my query so I specified I want a character here and I want the name of my character and I got exactly that so it looks like I sent an empty jason graph into my server and get a fill Jason graph out of it it is not like that but it's a good mental picture to start with how graph query works okay so let's explore the API further so I can write here of movies and then I can drill in like I can fetch the titles of my movies for instance or I can search characters that played in that movie or kid in that movie and then let's fetch that okay boom I have my data here and I get the data like I want them to be shaped okay so let's do what we actually wanted to do we don't need that name and we don't need that title what we actually wanted is just a list of the names of characters that appeared alongside Luke Skywalker let's fire it up and are basically done I have defined for my server the query which basically describes which data I want and the server gathers all the data as efficiently as possible and returns the data to me so let's see how that stacks up with our unscientific benchmark you so let's refresh that and see how fast graphic we can do that 250 milliseconds that's good but if we look closer of this it sorry it actually took only 45 milliseconds because the other requests here's the options request for course middleware so actually our data call just took 45 milliseconds that is I don't know how much faster but a lot faster so six seconds with rest 45 milliseconds with graph query and the trick here is actually the graph wave API that I've wrote there is just a wrapper around the swappi that code so actually the graph plan API fetches the data from from the original REST API so why is it so much faster because both api's are hosted in the cloud and I have unlimited bandwidth there to run like data fetching algorism on my client is actually very inefficient but running in the cloud is much faster so just by putting a graphical representation closely aligned and that puts the consumer of your endpoint or the consumer of your API in the driver seat you as a consumer of a graphical API define what data you want and the server just fulfills your data need and that allows you to iterate much faster because in a traditional sense when a front-end developer needs some more data or needs some more of this or need some less of that he has to always ask the back-end developer to change the API with graphic well you don't need to do that anymore the back-end developer just defines the type system and the schema and by that basically defines all the data that is available and the consumer defines how he wants to consume the data okay so with rest it's more we are exploring the API we are looking at the results we are fetching some data and we have a look how the API works sometimes that works out and sometimes it's not because because with rest we don't have a guarantee there's no type system and even with swagger the types are metadata they are not really they are not enforced by a query engine they are not enforced by a type system it's not like you're writing a c-sharp type and it Karen it's guaranteed to be like that with swagger it's just I tell you that it looks like that but it doesn't have to be like that okay so we can also do dynamic endpoints with wagga but if you go down the route of dynamic REST API so you end up like this Emma's in shop API with lot of parameters and the API becomes unreadable and also we don't really know what all the types are there is associate take for instance in int alone or can I even feed strings in there I don't know how what is graph wave so graph wave has only one endpoint that's a post rest where you have multiple endpoints and with graph query can fetch all my data that I need in one request and that's crucial because we want to have efficient web apps efficient mobile apps and with graph query there's no over or under fetching over fetching is actually when you looked at our first algorithm I had to fetch all those data that I didn't need that's over fetching and under fetching is if you remember the request and we had those you are ice there these references to other resources that is under fetching because I actually wanted this reference I didn't want it just the UI so you don't have that with graph and graph is enforced by a type system so it's built upon a type system and we will see more about that in the next demo and by just providing types and building upon your type system your schema you produce already a lot of documentation when we explored our API at the beginning we just looked at the type information and that's a lot of documentation that comes for free if you think about it and then graph by there's no versioning concept so in graph API it's more aligned like with c-sharp you would deprecated fields like you put an obsolete ekhan on a property so it's more in line how we developers already develop develop our c-sharp programs and graph players real time and we will have a look at that and all of this makes graph bar very predictable to use and it's super simple okay let's have a demo how it actually works in in c-sharp so I have proof they had a little thing here so let's start with a simple hello word just to warm us up so a set coat so this is a standard graphic wave Oh super it's downloading my own new Omni sharp-edged so we have to skip intellisense for a second but okay mm-hmm so this is a standard asp.net co-op project here it has nothing special in here except it has the hot chocolate asp.net core package in there and also this startup is just an empty startup every graph query server starts with its schema and the schema is made up of a separate of several types and every schema has to have root types and the most important rooster root type is the crab you so let's first create a class called crabby so we just type up a class here query let's say it is a string it's called hello and we return word that's simple enough okay and now we have to have a schema so we have here some dependency injection extensions at and we have something that's called the schema builder still downloading should okay so we can say schema builder new and the schema builder basically helps up a building that complex schema type so we can just say here add query type and then sorry we say query and then we say create and we are done basically that's the simplest way to create a schema and graph square for dotnet so let's know okay I just see a typo here sun-ok finally my intellisense arrived okay so now we need something to execute on that schema and to execute on that schema we need a middleware so we say use graph coil here okay let's run that don't net watch and we swap the tools for a second so this also a UI that's basically develop our open source project to explore graph well api's I should have closed these oh yeah okay okay let's start with our hello world let's fetch our endpoint and then look at our schema I can already see I have a query type here I have a hello field here and that means I should be able just to fetch that hello field let's do that boom we get our data so basically it's very easy to get started we just can provide some simpler types like we are used to do I do not do not have to write anything special and I already come up with a schema but actually the schema is not fully specified there are more things that I could add to that schema graph query graph query for instance has this concept of none'll ability c-sharp now has a two but in c-sharp it's like with swagger it's just metadata and we are integrating that metadata automatically with the next version but for now let's specify something called a schema type so we come up here with a graph with a query type and it's actually an object type and it's an object type of query and the this type actually allows me now to specify more graph where properties of my actual root type here so I say protected I would figure and I get something here like that's called a type descriptor and that let basically lets me add more information about a certain field like I could say okay my hello field here I'm sorry my field hello actually is an on string type let's say that okay and instead of providing my type you know my route type I'm from I'm adding here my query tab my new schema type and if we run that in our tool we will see that we now have here an exclamation mark and that means this type is an odd no type execution-wise nothing would have changed so I can execute it I still get hello but if I now try to violate that contract like putting in here no the the execution engine will actually now protect the consumer from getting invalid data so I can refresh that okay we are back so I now get an error here that says okay the server actually is in an invalid state I won't give you that property and the interesting aspect is that if the server violates the contract then you will get if you have larger queries get partial results back so graph well is not like rest where we would fail everything when something unexpected happens in the request so graph player can also return partial results so that was it for hello world we done off dr. Graf we're server so let's get deeper into it so graphic well exposes its data through operations we have the query operation and queries in graphic where our side effects free reads on the server state and this is important because graphic well execute each resolver in parallel and that makes it super fast in fetching the data from your backends and then there is a write operation you could compare that to put post patch delete so in graphic where it's just a mutation it doesn't matter so the complexity of thinking about what kind of mutation you have to write is gone so you just write a mutation and it changes the server state and mutations are not executed in parallel but serially in order to have a predictable result and then there's something called events subscription which is basically events so real-time data and I think we should have a look at that Michael when you say about events are the actions happening in serial or parallel is on the client no that's beckoned execution algorithms so basically the query engine executes the show that in a sec so let's start another Star Wars server here start the wrong product Star Wars start with the mutation I have prepared another Stowers example here which allows us to write reviews for our movies a mutation in graph grave just let me fetch that okay a mutation in graph quiz that was the keyword mutation and just as a thing here because I before in the last example I just did the squiggles by default queries I expected to be a query so if you don't provide this keyword what operation you are doing it's actually always query and so by providing here the kyboot mutation I'm telling the graph player server that I want to change something on the server and now I only get the mutation fields in this instance I get here review and I can write a review here for instance for New Hope and episode review and I can provide a commentary here like this is maybe something this this is a good movie for instance movie and I can give it some stars maybe five and then I can select another fields and I will explain that now so if we look at the query structure now the top fat just this part is actually the mutation so this is what changes things on the server and this down here is already selecting data so I'm querying already that changed data and this is very interesting because it allows me like if you have more complex changes like I'm linking objects to different objects I can we we select query only changed server State and we select that data and that's very powerful but what does does does executing mutations serially mean it means that I could say this is my mutation a and this is my mutation B and the server would now start with a and then do B and only after B is executed we would do the querying on this on the under we would do the queries so first this would be executed and then this would be executed and with the query algorithm we would execute everything in parallel so going down the graph with the depth-first algorithm but in parallel like each field would be its own mood field and we will drill down in parallel which is very efficient for querying data okay let's get rid of that okay that is in this example it's not very spectacular because we just we selected what we have written into the server so I did a mutation that basically created a revenue here this is a good movie and we selected this state but it is really powerful when you do it with larger examples okay let's look into MU take a look at two subscriptions and they are kind of related to mutations that why we are skipping to subscription so subscriptions basically allow me to ask the server for data when something changes on the data so I can say okay when a review for Empire Strikes Back appears on the server I want to fetch the stars and I can fire that up and actually nothing will happen because I just says the server executes that query when a new review arrives on the server and I might have just written analytics program that aggregates the stars here so I just want two stars and if I execute now this query that is on you hope actually nothing happens here because I said I only want to be notified and get the data when something is written for empires to expect so that means if I do Empire Strikes Back here maybe give it three stars fire it up the data is already here and this gets quite quite good it's done over WebSockets but we are also working to do that over signal a so when you have all the browser's that we also support that and this not only works with such small queries you could have large queries and get the data only when you really need it so you not anymore pull and pushing and pulling data you're not anymore pulling data you're busy getting the data pushed want something changes on your server okay it's also important to look at what Roth queries not and if people look at graph query the name they sometimes think it's a graph database query language or it's Google's query language but it is not it has nothing to do with Google it doesn't have anything to do with graph query with graph databases and graph query is not a good solution for for binary streams so if you are Netflix you wouldn't use it for streaming movies Netflix actually uses graft well but not for the streams and it's not Facebook's version of OData and if you drill into broth quail you see that it's quite different all data was designed for an apps it's an abstraction over your basically data source database and it's it has a complete different mental set like a new data you have things like primary keys and entities and graph but it doesn't dictate these things graph coil is a layer over your API and that means it has terms that you define so it's your ubiquitous language that is exposed to a graph query server not like and likewise all data which is driven by data based semantics and graph way is not bound to a specific data source and we will see that later in the talk what we can actually pull together with graft where it's not limited to HTTP and we actually at the moment with the upcoming version of our graphical server we are supporting not only WebSockets which which is already in our current version but G RPC signals are in other protocols so graphic Wireless transport agnostic and the graph where spec doesn't dictate any transport so so actually at the moment the we started at the workgroup to specify how graphic should be served over HTTP until now there was no specification for it and Brava is not limited to JavaScript and if you think about what I said in the beginning Facebook built this thing for mobile applications for native mobile applications so it started with basically apps they were written in C++ to consume data in an efficient way it just exploded in the JavaScript space so every JavaScript developer knows graphical and most of them love love it it's also important and we will we will have more time in this talk so we will talk about all of the changes first challenge in graphic well it's called the n plus one problem and it's about fetching data in a graph if you if you think about deep graphs then fetching data in a consistent way is one of the challenges that you have to deal with in grafton and also scaling a graph query service more challenging like was rest which rest you have thousands of endpoints you can basically easily put up multiple of them and you don't think about that with graph query you have one endpoint and so it's more challenging to scale it so we have a look at how we do microsoft's with that later and estimating the performance impact of queries on a graphical service also more challenging because if you think about rest it's quite easy you could take each of your rest requests and easily estimate how much performance impact a rest request has on your system which graph where it's more dynamic so it's more difficult and also doing something like rate limits is a one challenging point in graph graph so let's start with the first thing here fetching data so I have prepared a little example let's say we are building the next Twitter and we have a query that fetches a message stream and the message stream basically each message exposes its creator but it also exposes the message to which it was a reply to and the reply to is again a message so I can also fetch the user name here so if you think about executing this graph from top to bottom with it depth-first algorithm you would first fetch the message maybe then fetch the user for this message then drill in fetch the reply to here and then fetch again a user here so we have two problems here the first thing is we have to do two fetches to the database to fetch users because when we fetch this message we might not even know if this message even exists so how can we accept that the user here the second thing is what if the user is the same and changes while we are executing our query so basically the the user could could be named here foo and now it's called BA but fear not Facebook defined something they called initially prepare Able's which they specified now as data loader and data loader are basically like an interceptor between your resolver and the resolver is basically what fetches the data for your field and the datasource and the data loader does two things first it caches so within the request within one request it caches the entities that it has resolved already so if we hit one time a user and resolve that user from the database we will not fetch us the second time so we have data consistency here and the second thing is the query engine compare a combined with the data loader will aggregate all the cause to a certain entity and Bechet so we end up with a lot less data based calls to our own a lot less calls to our data source okay let's have a look at how we would do such a thing it sounds complex but it is really not okay and this is the fourth demo okay so this basically the example I showed you here and I have a message type here and the message type is implemented naively so basically we have our our created by field here that fetches from the database through its repository the user in order to fix that implementation we have to add here data loader so let's copy and paste that because I don't want to waste precious time so we put that here and then I'll explain it so we still would use our user repository because I don't want to throw away that nice API so the thing that I do here is I ask my resolve of context to use a batch data loader that basically batch the request to my database and I tell the engine that this data loader is called user by ID and it should use my user my user repository to actually fetch the data and because it's a batch data loader I specified here batching function to my database which actually with receives a collection of IDs but then the actual resolver will only fetch the data through that data loader and ask for one ID and the execution and it will basically aggregate all those requests and then when all the day at the resolvers that ask for that data loader are include it will batch all the requests in one go to the database and fetch this graph data very efficiently from your database so let's run that let's do a watch one well that comes up go here okay that's refreshed should already come okay we can easily drill into our data I asked for messages here and I can have to create it by here fetch my name and fetch the text here okay from the execution side is the X oh sorry it uses an extra database and I haven't started it sorry and this more so let's do that here doc ah it's done okay again and we get our data so while this is not spectacular let's drill more into data fetching in graph where so actually if we look at our request again it looks kind of dangerous because I'm fetching all the messages here and this would with Twitter pretty much kill myself and kill my client maybe so what I actually want to do here is also their data paging and Facebook introduced something that is called cursor based paging and that's very interesting because it allows you to really slice your data so how do we do paging miss graph where it's hot chocolate we have a concept that is not only resolved us but also middlewares feud middlewares that means i instead of using the resolver here i can also build up a pipeline and say okay i want to use paging and my entity type is actually message type and now the resolver returns a credible in this instance and our paging middleware will generate paging calls to my database and fetch the data in a way that is very efficient and I don't have to rewrite my extra repositories so when we refresh now our data my two we're already steaming tell me that this is not right here because my data has changed so let's explore that first of all I have a lot of etiquette of arguments here so I have an argument that is called first so I can say give me the first 10 and this doesn't sound very innovative it sounds like the skip and take approach but wait a second and then I have something that is called edges and I have an edge represents a cursor and the cursor defines my position in my data set and it defines my position in my data set in a specific data set so if I had a sorting and stuff like that on it it defines my set in that data set and actually if you implement it right it also prevents you from from those jumping paging where you add data while you page through it and your paging really jumps you don't see that when you are working with Facebook when you scroll through the Facebook stream it's very smoothly and that's because of these cursors because new data will be added to the top of the data set and that prevents you from jumping from getting jumps we have a look at that so the second thing here is something that's called a node and the node is actually my message now so I can't put that in here reformat it execute it and I now have paging without with just one line of code and I have this curler here which is base64 string that describes exactly where my item is located in my dataset and now comes the tricky part because I can say now first I went the first let's say the first two and I went the first two after my cousin okay well this is nice I get the next I should get the next first who after my aren't should have is it the same it's a different yeah okay so sorry it's sometimes different you have to spot the difference okay so I can now get the first two after that but I could also say give me the last two after that or give me the last two before that and I can even combine that and slice my data from both sides that means I can really slice for the first time my data however I want so I could get the first ten from the last item in my data set without thinking about okay I have to do a full count on my database which is very expensive and then calculate back so in this case I just fetched the last last cause of my data set and we have something that is called here a page info and I could ask for the last cursor so I could ask very inexpensively my data pays to just fetch the last data set get the cursor from that and page backwards okay but that's not enough we might want to have something that is featuring I want maybe to fit on my data a query and mutations are stateless whereas subscriptions are stateful because they have a socket connection so I want to do filtering on my database I want to filter over the messages and stuff like that so I can again it's a pipeline here I could say oh I can see I forgot something in my demo there is another package that you could add and it's called types dot filters yes restore that restored go back here okay and now we have something else called use filtering let's save that go back in the hour to refresh our schema and now I could say okay I won't they let let's put the cursor away because it changes now but we want the latter they first to let's say the first to where the text contains and you see all those operators that I have no here I can ask where the text contains where the text ends with I have all those operations now with just one line of code and it translates the graph where query directly into your database query in a system so we are using variable to build expression trees and these expression trees are translated then to all record or SQL or whatever you so we could say in a simple contains here on the text for instance and look for B so the first the first two beasts we want to find so bar and butts okay but we could even say okay and let's do another query of their object here and actually we want to have the text ends with that okay now we just got one item so I have a very powerful very effectively filtering language it's busy we started with it by looking at prisoner they were the ones who started those features in the JavaScript world so Christmas actually a JavaScript framework that does filters like that for note services so we implemented that for dotnet so with just one line of code you have all those power to filter him and you could even say okay I want sorting in here and then it's just another package we combined that packages actually with the next version but in this version you can say okay I also won sorting is we store that and then you could say you sorting and add more to the pipeline and you can expend a stamp the pipeline by yourself so you could write more data aggregation pipelines here by just saying it basically works like standard asp.net core middlewares just on the field level and you could say like I have next I have the context here and then you could do your aggregation code there okay that gives you an idea how powerful you can do data aggregation data filtering and even more stuff okay the second thing I said is changing is scaling your graph and we will look at two concepts tonight one is persisted queries and one is called graphical schema stitching let's start with the first one so the problem is graphic where sometimes this that varies it could be very complex and the first thing you notice when you write up a query is that it's actually a static static query tree but every time you send it to the server you sent actually unnecessary unnecessary data to the server because you are sending always this big large query so it's a high bandwidth usage there for more about applications for instance the second thing is that you when you send in rock furries to the server is that you give the consumer the API unrestricted access to your query engine and in the beginning of the talk I said this is a superpower of Gov query and it is but it is for your front-end developers you don't want the other guys to use that maybe maybe yes like if you are get up and you have a public API they are using graphpad as well in their version for API then you want to expose it but most developers don't want to expose their API to some other people so they can restrict it so the solution that Facebook has and the what they are using is called persistent queries and with persistent queries we are not sending in the query we are sending in the hash of the query or it could be any any any string but they are using for convenience a hash of the string and the server actually has a se query pre-compiled and optimized on the server so it's must faster when you are sending in a roar query and you still have the power of requesting what you want during your development time but now you have also the control that you have controllable requests during the your operation times yeah this the second thing is that you you not only have they been without advantage with that you also can restrict what is executed basically whitelist or the queries that are allowed let's have a look at what that means okay five okay let's this again graph class that was server here I'm um I'm using this that was example because the graphical community actually created the Star Wars example like this puppy dog come swap edad Co for the rest api s-- so when you go after this talk and explore examples you always will run across this that was api's so it's quite of good to use that okay this is a standard graph graph where server built on asp.net core has a lot more in in there as than the graph where so as I showed you before so in order to support persisted queries all we have to do is to add the Pacific various package G and we implemented persisted queries not only for the file system you can put in Redis air or Arab blobs it's just a different package and also for the for the subscription stuff we are supporting Redis as a backing system or other messaging systems okay let's have a look how that works actually so we go here and start up and what we have to do to add persisted queries is just one line of code again we say at read-only file system query storage and we don't need because there are multiple flows to do persisted queries we are using is a simple one that just allows Pacific queries now and it should pull in and persisted queries from our very store and that is located on our here in the file system so basically I have folder here is queries and it has a query that is called foo and the foo query is actually that so basically it fetches the hero of the name of the hero from new hope okay and then the second thing I have to do is to add a middleware to the query level so we have filled middlewares that I showed you earlier but we also have some kind of request very middlewares that on top of the whole query request in the execution engine I can change that by saying here builder and I can tell my builder to use persisted where we pipeline basically now my query engine accepts empty queries that just have a reference to a query and they can basically then look up the query and give the correct ferry to the execution engine okay let's start an up not and and no sorry run okay now we are not using one of those fancy graphical api's because now we want to look at the raw requests so I'm opening up postman here let's kick that don't save don't haze okay so I have a new breakfast and graphically requests I actually either get requests or post requests post requests are more common to use but I will tell you in a minute also why you might consider also get requests so with the post request a normal graph will request would look like this I would have here a JSON object that has a query and the query could look basically is a standard graph quick request here like I could say episode just let me fetch a different episode than me and we are using in our persisted very okay be using no new hope here so I'm using Empire Strikes Back here sorry Empire Strikes Back here and I'm fetching the name so that is the standard Draft where request that would go in and you can't imagine that the name token what did I do yes I can see sorry ah I see sorry yes mm-hmm so now the query is corrected Pass is correct and we get Luke Skywalker here and you can imagine that like if you have bigger applications like the Facebook application the queries are huge so that's why it really saves you a lot of traffic when you are working with bigger examples so we get our query here back so with the persisted query I would say ID and then our persistent query is called so I say execute the query foo and then our execution engine execute that query and this query is pre-compiled and does not need to do validation like my server doesn't need to pass that query we don't have to validate that query because we know that it's that it's correct because we are validating and compiling them when their servers starting up and so why are get requests than good that get requests are good because we can actually specify those persistent queries in a simple get request here and this works very nice with output caches because now you can put that on a CDN like fetch the data and cache that data on a CDN so that's why it's also worth looking into get requests sometimes so let's conclude our first part of the talk so graph B is a great way to expose api's they are more human readable and if you design your model right it's like an ubiquitous language like a lot of non-technical people could easily explore and do queries against the graphical backend because it's very easy to drill into the graph and explore that API data fetching an aggregation is done on the backend because on the backend we have unlimited spent words basically servers are close together so aggregating the back end is much faster than on the front end the type system in graft where it protects you from doing arrows in your back end that might lead to errors in the front end like with an honourable example for instance the typescript developers can completely get rid of no checks for fields that are not knowable because the server is guaranteeing you that it will never deliver an O for unknown no field that makes it much nicer to use and rest because I don't have to check my responses I know they are correct but graphical also introduces new challenges like scaling your API Sam we see another scaling example in the second part of the talk and graft way is not always the best solution there are better solutions for binary streams and if you have just a simple heads check endpoint you don't want to do graft wear because there's nothing to explore you just do a simple rest request so it's not always that that graph but is the best but as soon as you are having bigger data models it's the better solution then rest ok we start with something simple because it's the what we do after the neckline is really sometimes blows people's mind and so let's start with something simple so what do graph query clients actually look in the net because with all those dynamic queries how do they look like at the moment that's not our API people querying graphical like that we basically have a string which is a query and this is actually very horrible because logon is not it's not type safe and this could blow up quite easily so this is how users try to do it like with the github API at the moment but basically all our strong types or what I told you is gone through the window with that so we invented a new API so we have hot chocolate in the backend and strawberry shake is our new client API let's have a look how that works so I have prepared a little project here and this time we will consume the Stars API so let's first start this tower server because otherwise it won't very good run let's take our first example that we heard stores and ok so while this service coming up let's have a look at the API so this is a standard console API because I'm not good at front-ends so you have to stick with console api's with me let's get rid of this bad example that I just showed you client basically has some information here like about about the endpoint that we want to query like I told my API that there is a schema called Star Wars and I want the schema to yeah I have a schema file that is this that was graph query schema and actually if you would get started with it we have a dotnet tool you could do something like this in it and then tell it the server and stuff like that and it would actually fetch down the schema so you have a local schema copy here and you have the information about the schema and this is important because graph code can be compiled and we actually hooked up into the into the compilation of dotnet and we are compiling all the graphic well that you will write so you will get actually intellisense and stuff like that while you write all your credits and you just write graph queries and we translated to c-sharp okay so let's do that and the important aspect to you like I have visions to decode so I have to build to compile that stuff with a visual studio for Windows we are doing the same like the protobuf type put above guys so actually as you type we generate so let's start we will start with queries so we have a query document here let's call it queries and then we could let's start with something simple we have the simpl query here that queries the hero and episode and we take new hope and we say its name so there's another excel the thing that we have to do in order to let that compile here we have to give our operation a name and let's call it get hero okay so like I said with visuals to be for code we compile that and now we can see we have a generated folder here and that's basically this translated into code and now I have a character here and the character only has the data that I actually used so it's not the full character that I have in the server it's just the character that I use here and the client is already because it's it's preview and I said we are fully embracing the new feature so we have not been reference types here so because we know in the server this field could be null it's not a bill now in the client let's have a look at the our actual Star Wars client so Stowers client is now generated I have here a get hero because I called my operation gets here oh so I get it get here Oh function here and I can fetch my data through that so let's just type that here heist tom was blind and now I could easily fetch my hero here and oh let's make that hazard task yeah wait that and do that okay ah sorry that's and now I could explore my result and so I have a graph query result is X actually specified so and graph query result has a field data data actually contains my hero so whatever I I have as a result would be here but graph man also has like error informations I said we could have partial results so all the errors about my results are here and we made it very convenient so that you have like with the HTTP request you have something that calls in June or errors and then we would throw an exception like it would expect but we give you the full power to get all the things out of the data of the actual graph query request like it would expect with any other platform so that is nice but actually that's not useful because how a query is kind of static so in order to make that more dynamic we would need to define variables I think graph query has variable specified so you could in your operation define now the inputs that you have in into your query like I could specify I have an episode here a dollar episode and it is an episode happy salt let's do that okay let's compile that and then we get an arrow because why no actually we don't oh yeah okay that's that's one buck in my preview here needs double compiled okay actually we that that works quite fine here I get my episode I can search for the episode and now my function has changed my Star Wars client here star wars lined client has now the possibility and I can input here an optional of episode an option of episode I explained that in a second it's actually implicit also an episode so I can just put an episode in here and then could fetch my data why do we have an optional because in the server side I can define that my inputs actually have default so I could specify that this field actually has a default so if this is now that gets a default from the server so that's why I need a concept that is called optional I can also define that in microwave I could say in my query I want to default to be new hope so I don't have to provide that but so it's optional but if it's not provided then it will be filled in by the server and we had to introduce that optional because none is not sufficient we really need to know if it's not provided but actually it doesn't shouldn't any bother anybody okay so then there is a thing like in graphical enums are actually always uppercase so now we have an enum here new hope and this looks like kind of crappy because it's not like we see sharp develops like how enums to be formatted with kemon humps so we could then introduce a new file and that basically does schema stitching on the client side because I can reshape my server schema into how I want it to be on the client side I can't say Star Wars Star was extensions craft and then I could say extend my enum and accept my enum episode and I only want to extend new hope and I want it to be named to have a name that is called actually new hope so I can rewrite the type names I could also add new fields that only resolved on the client side I could add fields that fetched from the client side data so I can merge server and client Forest and execute them and once okay so let's rebuild that yeah hi-yah now it's compilation actually is right I have here new hope and now has my community chemicals so this is quite nice because I can shape really my types how I want them on the client side to be okay let's make it more complex let's say we have friends and the friends have notes I think notes and they're happening and then compile that and we get and we will get quite crappy code because it should be right unsigned okay that's okay so now I get something like a character one in the character and that looks kind of crappy because I have a character one that has a friends and I have a I have character that has friends and have character one that has just a name but there's something called fragments in graph where and fragments define reusable components so actually I wouldn't write those graphic varies like this and if you look at the Facebook API they don't use that like that so they would specify maybe a fragment here and that's why I say you can start thinking about it as an empty Jason graph but if when you're drilling into graph whether it is a super powerful syntax so we can't specify a fragment here and we can say okay I have a has name maybe and this goes on the character so basically I'm I'm describing now a component here that only has a name on top of this character type and now instead of using name here I'm using has named you and then I could further go on take that again and say maybe I have has friends and this has friends maybe get this construct but then reuses this again and actually that is very aligned to how we do things in c-sharp because now our grammar our compiler can use that can use that to infer from that interfaces so now I have an interface I has name and I has friends and if I look at the character the character actually implements I has name and has friends and now the API is really nice to use in net because it has quite good names and I can type checks on them and and stuff like that and sometimes a character implements this or that okay let's just copy in a bit of setup code here because we can run that actually and that's also kind of nice because we generate you all the setup code for your dependency injection so in the end this is all generated and in your C sharp submarine or even even it works even on watch OS so if you have a watch this client also runs there so but if you're using it with dependency injection we can also generate it without dependency injection and then you get a build a pattern then you can have a client builder that you use for instance on watch the rest setup your client you can also use any other dependency injection but if we generate it with Microsoft you basically say ok I have my service collection then I add an HTTP client and that's the connection to our server you name the the httpclient like you named the client I called the clients that was Klein so it is Stowers client then you tell the HTTP factory where the UI is maybe you add some poly and stuff like that and then it's one line of code to set up your server it's just at Stowers client we generated you all of you okay and just paste that in and then we have then we in this example if you use it in asp.net core you don't have to do that but in our example we use the service collection we build a service provider on it then we get our Star Wars client and we fetch a hero here from new hope we ensure that it has no errors and then we just print the name here but if we want to explore the API further you can see it's now very nicely to explore we have our friends with this on a character connection there's no character one anymore we have our notes here we could touch the first of first first note put some link and then we can see it has a name and it is again it is again it has name in this case so it's very nice how we can infer from the query that you write the API that you want to use in submit so let's run that okay we get r2d2 because I just printed the name out you could print it also other stuff out the important thing here is we are fully compatible to the node nullable pattern that means the the graphs busy are immutable in this case and if you look at it it's quite a nice formatted so there is no we put a special emphasis on not generating crappy code like lot of generators have a lot of crappy code so it's kind of kind of nice to use and easily to use and it really compiles if I would method messes up say don't net built I get a correct error here in my graphic file I can't jump in here that that's an actual compiler error in in Visual Studio for Windows you get it a new grid down there it's just with Visual Studio code you have a lot less tooling so but I still can jump in here on my link I get to the actual error here and it says the specified fragment has new them does not exist or if I did another mistake here let's do this I would get another compiler error that it's that this query is wrong and the cool thing is that if you compile the field does not exist and the cool thing is that you can combine that with tooling in your in your DevOps pipeline for instance where you fetch the schema we have a dotnet update schema and then it doesn't compile if the server changed in a way that your client would not be compatible anymore let's really power him okay so that's how you can do clients with with graphical and strawberry shake we are at the moment in preview it also works with a real time with the next preview that we are releasing actually we could generate already real time code here but it would crash like it would generate actually I could could do a subscription here it would generate but it wouldn't run okay okay let's talk about micro services and that's very interesting with draft where because guava is built for micro service but if you think about it like or what I told you so far is kind of monastic like we have this one endpoint we have one data graph it's all it sounds like a modulus so how can we do micro-services with the monitors let's first line out some principles because what we want is one graph we want one data graph to explore and it's worse to not think only about applications like you could think let's build one data graph for my company with all the data in there I think even the Microsoft graph is much better with with with graph where and we want one endpoint to consume our data so one endpoint effect we want to see we want it simpler and we want to be able to fetch everything when what was one request but we want to basically be able to implement it in a federated way we want to have micro services that offer the data like it's efficient for those micro services not efficient for the data graph the solution to that is called schema stitching and there are as this term is kind of made up like there is a company called Apollo and they are the leading company in graph where they are doing the node platform and in graph in in the Java ecosystem it's much bigger than with us the net developers and they turned the coin schema stitching but they deprecated their schema stitching and renamed it to Federation so if you're looking for schema stitching and you see something like it's deprecated they just renamed it it's just a name it's not a specification so we still call it schema stitching because I kind of like the term so what is schema stitching schema stitching if you just take what Apollo does is just the capability of merging different schemas into one schema but with hot chocolate it's more and our ambitions are bigger for that so let's have a look at a typical setup so a typical schema stitching setup would consist of multiple servers like we could have Apollo graphical server because graph will respect its it's it's a standard so schema stitching works with any graphical server so if you have like multiple scrum teams and each team has its own own technology stack then you could mix and match the graph query service maybe there's the team is go they are a team with Apollo basically note and maybe there's a dotnet team so maybe we could have an Apollo graph PI server and maybe in our example here our graph PI server has the user data this is our user service know so users and maybe we have a dotnet team another graph query server and maybe it has all the message data from our Twitter API and maybe somewhere in a cupboard there's this old west end point and we don't want to rewrite it so hot chocolate can stitch also this into graph query so you can provide us your REST API and we make it graph query and then we have something that's called a gateway and the Gateway is another graphic web server that has some extra middlewares that explain to to the schema how to rewrite queries from monoliths into those micro services but it's very abstract to explain that in slides and it's much nicer to see how that actually works so let's see ski master Ching okay okay we have multiple service here like we have a condom this is an example that I gave to Swiss life it's a insurance company that says insurances and buys real estate and so basically they sell you the life insurance and they put the money in real estate and so we have a contract schema here and we have a customer scheme let's fire them up and explore them don't let run contract schema as much project okay why this is coming up seven stitching which one did we start contract customer asking so so we have two two schemas here one is the customer schema that is coming up now is on port fifty let's explore that that basically holds all the customer data so if you look at the schema we have a customer here and the customer maybe have it has a consultant basically the basically the person that sells you all the insurances that you don't need and we basically have address data here so we could explore that we could fetch all the customers here we could fetch the names of the customer maybe we could also fetch the consultant that sells us all this crap though there may fetch the IDS let's have a look we get all the data here like we asked for it we have like Freddie Freeman and he has a concern Jordan Belfort and yeah customer like that and we have an ID here for customer let's copy and paste that and keep that in memory that we pasted so then we have a second end point that has all the contracts data 50 it's on 51 and this has things like life insurance Contractors has queries for contracts so let's explore what we can do we can ask for contracts and we have to pass in a customer ID here okay let's pass it in and how I just have an ID here so I can prepare you for that okay that's not much but there's a concept in graph query that is called interfaces or Union types and in this case I have an interface so I have life insurance contract and I have maybe other contracts so I can ask for data okay if this entity is actually a life insurance contract then give me the premium so basically the the money I have to pay each month but if this is actually some other contract maybe give me the expiry date so I can see I have a very specific data in a very specific way for the micro-services to fetch here so what the client the the front-end developer want is one endpoint one way to query this data okay so let's set up a little gateway okay a giveaway is a standard draft words are the only difference here is that we have a stitching package and okay okay and what we also have here is we declared the HTTP Factory and we basically pointed to the end points that we want to stitch in like we have endpoint 50 the custom endpoint and we have the contract endpoint and we made it named clients and this is very good because you can now use poly to make it very resilient if maybe a request goes wrong then you refetch and stuff like that okay the second thing now is we're using our special extension metal a method here that called at stitched schema and this stitched FEMA has a bidder and with this builder I can very easily just reformat that describe how I want to stitch my schema together so I could say builder dot add schema from HTTP actually I wouldn't fetch this key on a production system I wouldn't fetch it via HTTP schema I would actually do the merge already in the in the CI pipeline because then I don't deploy schemas that would break but it's always good to have those for demo purposes because I can just say ok I fetch the schema from there and then we do it from here ok so now I just have to provide my schemas here and basically the first request of my front-end developers is not fulfilled I could fire the table and the need to query everything in one go is now fulfilled do we have inside yeah okay because I can't touch that and now they fetch did it fetch no it didn't just let me see why not hello what did I do wrong let's compile it please yeah what wait stop that don't build I know it ah yes yes yes yes yes thank you yeah actually okay let's put it on a different port let me just grab that coat from here let's kill that on coat let's put it in 5,000 won and then the program just then we grab this let's put it here and here I'm putting it on this pot okay let's get this one we just won again running no sorry which one which one did I kill here 5001 okay that was me the contracts hope okay now is everything up again and now we should be able to do that no addresses to already use very strange let's wait to reset code happy okay let's give it that completely different yeah okay yeah my other examples might have looked okay we fetch that and now we fetch this end point it's just two lines of code I said to them to the schema take this schema and take that scheme and merge it together and now what I could do is I could prep this query here and it would run okay that's an easy feat but I also could could take this schema put it in here touch that and it also runs but my front-end developers would tell me that actually crap because now I have one endpoint but I still have to paste IDs and stuff like that okay when and this conversation actually happened when we talk to our front-end deaths what they actually want to have they came up with something like okay we want to explore our data like this we want to have a me endpoint with me field here and I want to ask maybe from from me a seed signed in customer off of this application and I can explore everything from the customer from the sign-in customer through Semyon point here and then I want to have the the maybe the contracts here and I want to explore the contracts through the me endpoint and do this so we went back and we then introduced something that is rewriting the types and this can easily be done like we have here add schema and it's our add schema endpoints but we can now say dot had extensions from five in this time and in this case I can't say grab this next engines don't graft we're fine and then we can use graphic web to rewrite our types like I showed you in the earlier example in the client we basically put under the client compiler we have the stitching engine okay so what we can do now is we could say we could tell our engine to the extent our type query and our query type should have a method and the me field should actually be the customer but then we also have to tell our engine how to fetch that and then this dance word through something these these guys here are called directives and they have the power to change the execution flow so basically directives are actually associated with middlewares so what I can do now is to say here delegate gate this call to a schema called customer and then I have something called a delegation pass I don't have to specify that every time but in our case I do want to fetch the customer but I don't want to fetch any customer I want to fetch a customer that is logged in so I can specify here a special variable that is called context data so I want to fetch something from my server context and I could call it the user ID reformat that okay so now the scheme already would fulfill that but it will would run into an error because we have to provide that context and we have something called a request intercept drop so basically we can intercept each call that's coming in at each graphic graphic well call that it's color coming in and we can intercept here the HTTP context and the graph they requested us so actually the builder that builds a request for the execution engine and then so actually when I would do that in production I would grab maybe from my user would be signed in and I would have some claims maybe I would do call to my database look up some special data here in our case we will just add a fixed ID so we say our builder here then we want a property a special property I think I called it user ID and I don't have the ID in my hand anymore so I'll just let's grab some here so let's imagine we we actually fetch that from our from our context right we could fetch our identity of the principle from the context and do stuff with that but we are just putting it in fixed okay now my graph reserved actually knows that we have a user ID here it's this user ID that's my logged in user and it would delegate the call here okay so let's crash that watch run it's up and let's explore the API because now I get a new field in my query that it's called me it's a customer and now I can do the first part here like I could fetch my me and it's Kara Danvers in this instance and I could do everything that I could do on the microcell is like fetching the fetching the consultant it would get this consultant so then I can go on and rewrite my API as I want so I could in and like let's extend the type from from let's extend my customer type from the other server from the customer schema so I can extend my customer okay okay sorry so I can't expect my customer like I would do here and then I will get my customer field maybe I have this to copy/paste here it's a lot of let's let's paste it in and then you at least get the result so we can put that in and then we actually get what the slide users want but there's one more thing maybe I also want to rename types because life insurance contract sounds kind of bad we just want to get rid of the contract so let's do that as well just by rename rename type so we can say this type is actually known this type and last but not least let's kick out all of the Microsoft's that we don't want so we get rid of the route type so we actually flush out everything that we don't want and now we can remove that extent here and we define now they type locally and we will end up just with the me field okay sorry so let's we compile it let's have a look at our query type where is it and now you can see we have just the me field here everything else is already invalidated now we can also grab me our contracts data and I don't have to specify any ID and I can say if it's actually the life insurance so I the point here is I can completely rewrite my source schemas and if we had more time I could show you how to integrate all data and rest it so you can fetch this data very efficiently you can completely rewrite your domain services and that means you don't have to compromise your domain service design you can design the structures as you need them there and then rewrite them with just a very sim gateway layer because if you look at the code we didn't write all the resolvers it's all generated for us we just specified kind of with the schema definition language of graph query we specified how our schema should change and then we have like here a couple of lines of code and that support was it so it's very easy to rewrite schemas so as a conclusion here it's easy and was minimal minimal amount of code that you can create a gateway and get what you actually want and you can describe rough pervious graph where I can use just a graph where schema definition language to define how my types should change you can merge multiple data source together and if we had more time I could even show you how to stitch in SQL services and other stuff so you can in the Gateway just merge things together but it adds complexity like you have to think about things like Poli like making it resilient to fetch data and so I would say start with a single server and then explore if if there is a need for that so sometimes it's it's easy to start with the simple graph words or before stitching so we are an open source project we are the fastest graph query server on the.net platform twice as fast as the earth our competitor and we use only a fraction of the memory for each request so it's only a third of the the memory and actually that with the next version we are having that performance statistics so we are then four times faster than graph board or net so everything you saw is open source it's free there's no hidden charges everything is under at the MIT license you could even copy it and do your own thing at the moment we have the graphical gateway and the server and they are production ready what I showed you today also was the graphical client that we are hoping to deliver in January and author the tooling that you saw the UI with all the intellisense and stuff it's also done by us and should arrive with the plug within the next platform release in January but we are not stopping there and for August we are also building parts of this already we are introducing automatic database mapping that that means we don't only understand a variable we also can you can just throw us a database there and be translated completely into graphic web or you write a schema and we translate we build up you the database for that schema also we are the first graphical implementation that actually will introduce execution plans which basically make all those data loader stuff that I showed you absolutely because we can infer data loaders from your data structures we can automatically automate optimize your data from your queries and that makes it very powerful so all of this is also free and open source you can track what we are doing online and everything is on github I already pushed the demos I will just do another update in a minute and then you have all the demos all what we done today in this repo if and download it and look at what we actually did today yeah that's it basically

Channel: Pusher

Views: 8,530

Keywords: developer language, language code, libraries, help, tips, forum, meet-up, meet up, learn code, coding education, coding hints and tips, lecture, coding lecture, learn about code, learn a developer language, amazon alexa skills, developer conference, node.js, javascript, backend

Id: q-5MUqLAEFs

Length: 108min 15sec (6495 seconds)

Published: Sun Nov 24 2019