GraphQL Contributor Days | December 2021

hello everybody and uh welcome to graphql contributor days we're really excited uh for you all to be here we're really excited to have this group of people again after so long uh feels like it's been a year it hasn't but it feels that way uh we're really excited to talk about graphql today and all the amazing things that are going on in the community and all the amazing things that you all are doing um before we get started though of course we would love to thank our sponsors uh this dot labs and hazura uh tomma you want to talk a little bit about hasura hey folks glad to be here and glad to be kind of helping host this event again um master is an open source project uh where we're kind of working on providing graphql apis on your data sources uh multiple data courses automatically um so check it out on github um and otherwise uh happy to be here and dive into more conversations around graphql related topics over the coming hours awesome and uh this.labs is a javascript consultancy uh and we love working with our clients to build out their graphql solutions and are really excited to help facilitate this conversation coming up in the graphql ecosystem we have several events uh you can check them all out but uh you can learn more at this.media.com and uh today uh tanmay and myself will be your host uh my name is dustin goodman i'm an engineering manager at the stat labs you want to introduce yourself hey i'm done i'm the founder ceo at tesla yeah and we're we're just really excited to kick this conversation off um uh and we we have just an amazing group of panelists and we wanted to let them introduce themselves uh starting with mark yes uh hey everyone my name is mark i'm joining in from montreal canada today um we've got snow here as you can see already i'm a software engineer at netflix where i work on graphql there lots of federation stuff yeah that's pretty much it uh patrick you're next oh yeah sure um yeah my name is patrick i'm a currently a software engineer not working just quit my job to find something new and i'm joining from london and i'm a maintainer of a python library for photographical i think natalya isn't here quite yet tejas it's stages uh sorry you're you're not the only one uh hi uh tejas here um i work at spotify even still today um but i'm on my way out and at spotify we use graphql um to build out a platform called backstage which is also open source um but the internal part of how we use backstage uses graphql it's spotify and it's a lot of fun to um kind of use the graphical user interface graphic ql to explore you know our service architecture as you can imagine we have a ton of microservices so graphql helps service discovery quite a bit uh you just slide next uh i'm uli i'm a member of a group called the guild we maintain a lot of open source tools in the ecosystem uh graphql code generator graphql tools and schema stitching graphql mesh and many others and recently um new server infrastructure starting with uh graphql helix and envelope and the new graphql yoga that is going to be out soon that's it for me um hi everybody my name is andy i'm working for adolescent um on various graphical topics but i'm also the founder and co-maintainer of graphical java and yeah a general craft contributor over the years hello i'm gabriel from sweden i work at an i.t consultancy and startup incubator called harrison and i also maintain a open source library called the rescript relay which marries together relay which i guess you've all heard of the graphql client from facebook with rescript which is a programming language that compiles to javascript i also run a meetup for relay online which is not very creatively called relay meetup so that's me uh hey guys i'm joey i work at paypal i live in austin texas and i'm on the web platform team leading the graphql efforts for uh supporting frameworks and tools and our most recent uh effort is around a unified graph head paper sluster's not quite here uh hey i'm connor i work at a firm a lot of times when i say that people think it's just a firm like any place but it's a real company it's pretty much that button you see everywhere telling you you can split your payments um i work on the account experience team and right now we're a recently formed team so we're kind of putting our apis together and kind of i'm pushing for a graphql the middle tier right now for us as we combine all these disparate apis that existed beforehand so i find myself in a family position at a company trying to get them to use graphql right yeah that's me i guess i'm right from amsterdam the netherlands uh end of day here i have been working with graphql quite some time i recently joined a company called stepson which is a tool to create a graphql api from any data source and um well yeah i just love graphql creating content throughout it like basically on all uh all different ways so happy to join today everyone uh my name is john calling him from washington dc uh formerly a senior staff software engineer at coursera um and now unemployed um but most recently working on guiding the company through federation kind of pulling up some new stuff with dgs you know doing the whole netflix thing as well as doing some consulting and advising in the graphical space too hey i'm jamie i work in developer relations at graph cms i'm from the uk and i've been spending a few a few a few kind of months working on creating content around graphql i'm excited to be here and talk with you all hey i'm aditya vichandran i live in kansas city i have a consulting company called surya consulting and i basically help teams transition to graphql transition to react react native and cypress i got into graphql a few years ago in a previous startup that i worked for and i really liked it and i also have some courses on graphql on pluralsight so that's pretty much what i do make courses and help teams transition to graphql and react my name is kyle shirati i'm a software engineer at stockx um in the detroit area and we recently have been trying to really scale our graph to handle the load at all times that's been our biggest thing as well is starting to educate other people so we had a nice uh group of people that know a lot and then we're trying to just spread that knowledge out and uh make it easier for everybody else to add on to their stuff through some cool cogen in automation of a lot of processes and tim will be joining us later i believe but it's really exciting thank you everybody for your amazing introductions um for everybody following along at home uh this is roughly the agenda that we're gonna be talking about but it's a free-flowing conversation not necessarily one we're gonna stick to in any particular uh stringent manner um but uh definitely we'll take a little bit of a bio break in a little bit but we'd love to kick off today with some community showcases and updates and i believe uh gabriel and uri both have amazing demos for us uh uri i think you want to get us started uh yeah i don't i don't i didn't prepare them i just wanted to um say that um this week we're gonna launch uh graphql yoga uh graphical yoga used to be uh like a very popular uh backhand um framework for graphql maintained by prisma um and uh with the time you know uh prisma moved to other things and uh they were kind enough to um basically talk to us and see if we have a vision for graphql yoga and where it could be taken on next and in the last few months we actually took uh envelope which is the plug-in system that we wrote for graphql which is a more you can think about it as more kind of like low-level framework for frameworks like uh for example redwood gs uh just recently migrated from apollo using a polar server under the hood to envelope but for people who actually want to start with building graphql software super super quick uh envelope might be so for us we use it just as a regular framework but for some people you know they might might want something more out of the box batteries included so we worked in the recent months to create uh basically graphql yoga 2.0 uh it's still in a branch but this week we're gonna basically release the first alpha and um and it's really batteries included meaning like it includes not only like uh the latest versions of everything uh but also out of the box it includes a support for a different stream um live queries and um and also the latest graphical that supports all of this um and an actual graphical that runs locally and not in the cloud um so um yeah so uh i'm really excited uh for this and i'm gonna announce it in graphic galaxy and and it's still in like in an alpha phase because we want to actually get the community to try it out and tell us um what they want from a battery's included framework and um yeah we're really excited about it we think there's a there's still a space for uh let's say a batteries included framework that actually is being worked on for the long term constantly being up to date uh the the the maintainers are hearing the feedback from the community and actually keep things uh updated so yeah thanks for uh giving me the opportunity to to announce it if people want to want to like uh track uh track it just google graphql yoga and go to the repo later this week master will be updated but until now there's a pr uh that is ready basically to use um thank you very much that's really exciting thanks for sharing um i know gabriel also had a pretty uh cool showcase that he wanted to share with us today yeah let me see if i can share my screen here i think i made it so you could yeah do you see my screen now we do oh excellent okay so a bit of background as i mentioned in my intro i maintain an open source project called rescue relay and rescript is a programming language that compiles to javascript and like some of the main benefits of it is that it's it has a very strong type system and very good type inference and a very ergonomic syntax especially for building ui stuff and good integrations we react so you like you use rescript instead of or in addition to javascript or typescript basically um and the project itself is about using relay together with rescript and i've built a cli that can basically go through your entire risky predator project and automatically remove any fields any selections from your queries and fragments that um you don't reference in code so basically preventing over fetching and it can do this quite well because you can always because rescript has such a strong type system you can always trace back any value inside of your program to an actual position in a graphical um operation so i'm going to demo when i ran this on a project that's kind of mid-sized it has about i think at the time it had about two or three to 250 graphical definitions in it fragments and queries and whatnot um and i'd also like to mention that like since i've been doing this for for quite a while i i like to think i'm very mindful of like removing things that i don't use anymore but it's just it's impossible to to remember to remove everything so this will run the cli let's see here and the cli will go through uh the project and it will automatically remove and you can see that it removed 34 fields from 25 files and i will see the diffs here so it has find a bunch of values that i don't reference anywhere in the code and it can automatically remove it so it's it's basically a way of preventing overfitting and it also comes with a [Music] with a ci action so you can put in your ci and then just make sure that you'll never ever ship anything to production that you don't actually use that's yeah that was it that's awesome thank you i know you mentioned it's uh made for relay is there any possibility of it being accessible through people using apollo um it is i mean the same principle should apply like there's the same there's the same it's the same type system and and whatnot um i just haven't built it for since i used relay myself but it should be possible to build it for that as well adapted to that awesome i was just curious i'm a i use apollo more than i use relay personally and i was like most people do yeah um do you have a github repo that we can all uh look at this that i can share out yes um i can write it in the uh in the chat here bring it up the other thing about the type system is it enables quite a lot of interesting uh tooling so for instance i've built a i have a dedicated extension for rescript3d visco extension which can for instance like autocomplete unselected fields in the values themselves so like you're in rescript code and you're outputting something from an object and you get an autocomplete both for everything you've actually selected uh on that type and then you get another complete list also for the things you haven't selected and when you select anything from that list it automatically inserts it into the operation so kind of like combining those two gets you quite far with like not having to deal with the graphql operations uh directly ask your questions i'm curious i have not used rescript in in any way but i'm curious about the kind of gql less approach where it seems like there's kind of you're vacuuming up these extra fields and ultimately you already know whether or not you're using it could you just do the opposite and generate the graphql query from the usage of the object um i've been trying to experiment a bit with this i think it should be like possible at least in a fairly basic sense but i guess the problem always becomes when you have potential like a potential of lots of code points that you touch so it's hard to know exactly what you're supposed to like from this route of using something figure out everything that's used all the dependencies of it uh without actually over fetching or unfortunate uh but like theoretically something should be able to build with the setup that's there make sense thanks sweet uh cool awesome so i think those are those the updates i think to be um we had a topic towards the end to cover um just i think on the agenda we had like the graphical foundation stuff towards the end but maybe we want to bring that up and if we're doing community updates maybe we just want to go through some of the any of the graphql foundation updates or graphql spec updates that folks want to share uh some of the foundation members are joining a little bit later so we were going to defer those updates until that point if that's all right but we i think we're all most excited about those updates so definitely can't wait for them to join um did he but that being said did anybody else have anything uh kind of uh and i've lost my words now uh community showcase or update related uh even if it is foundation related i had a small uh update to share is that we're kind of building out the we're building out some work on um supporting uh streaming uh uh kind of use cases with uh especially for like larger for like really large um large api responses not just from a um api latency point of view but also like if you want to stream out like uh you know hundreds of thousands or millions of uh entities and you want to kind of gradually get that stream and stuff so we've been seeing that use case fairly often and streaming felt like a good fit uh one of the questions that i had for folks anybody who's implemented either a stream or something like streaming over uh graphql um a have you been using kind of the ad stream uh directive uh we tried to do that but then we decided to start off with using subscriptions and websockets instead because that'd be easier primarily because um i'm not sure how folks are handling back pressure and things like that so because it back pressure becomes hard to handle so we just do it with web sockets um and it's it's easier but i was just wondering on the streaming side have folks had any uh implementations especially at scale what that's look like and a specific technical question around how people are doing back pressure or if they're worrying about that at all i can start off with a small snippet so at paypal we just decided not to use a subscriptions or stream for like we have a chat application for a specific product we're working on and we just basically for that streaming part we use a websocket completely separate from our graphical layer so we haven't decided to put it in curious to hear what are your guys's opinions for putting them in the graphql layer also i mean one good reason um to to put it in your graphql is that uh the data that comes from it you can then merge it into your store um so basically let's say you're you're now getting an update from like a subscription like let's say you have a chat message and a new message then you could propagate it immediately to the store and it will be normalized like with the client or relay or oracle um and then uh it will just you know be handled as any other data that you get throughout your regular api doesn't matter if the update is push or pull i will share i've had some personal difficulties getting subscriptions to work on different hosting services um i'm a big fan of putting graph ul specifically on aws lambda through serverless and uh gateway doesn't support sockets very well especially with the non-persistent connection uh making it a bit difficult to particularly perform at that way so uh that's definitely been a reason i've got opted for a separate websocket in those specific instances but like erie said i would much prefer to be through the graphical area because it does manage my stores and states for me so that's definitely been a challenge i've been looking to solve i haven't found a solution i like that doesn't take weeks of engineering sorry enjoy the question that you had about the in the graphql layer or not i i i i don't think i caught that clearly what did you what was the what was it like using stream defer or a subscription or live or whatever it is so that you're still using the same graphical endpoint and like already said it's going through the same layers so your your whatever store you have connected to graphql um right now it it was hard so we just didn't do it uh and we're using an apollo gateway as well so that's part of the complexity um but yeah if you're curious to see if anyone has solved that um i think dustin you just mentioned the same thing right yeah the way that we had solved it uh because we were on aws we leveraged dynamodb stream uh and were able to publish uh updates through api gateway directly from dynamo but it wasn't it wasn't the easiest of solutions and it took a couple weeks to get it just right um and definitely saw some latency issues with it so it wasn't wasn't my favorite solution but it was a functional one so yeah i'm also curious if anybody else has a solution to this that they've uh explored yeah i think for subscriptions type stuff we've just fallen back to uh having like containers that scale out very quickly but like like improving like that time for a graphical system to like spin up and scale out is seems to be easier with the existing infrastructure to get websockets type stuff working than anything else the other thing i've also seen is some people using some of the work or stuff on cdn's um like i think um i'm not sure if this was a fastly or with uh cloud flare but um you terminate this you do the websocket stuff with the cdn layout with the workers at the cdn layers at the back and these edge edge cloud type layers um and then you don't have to do the socket stuff yourself um and you terminate the socket the persistent socket at the cdn so i'm not i'm not i i haven't had any experience with it but i've uh i've talked to a few folks who've been trying trying something like that so not necessarily for serverless environments because then there's all kinds of other things to think about and we can get very deep into that discussion but um in graphql helix we basically now support [Music] basically we have built-in support for subscriptions using ssc as well and not only web sockets um and overall like we kind of like repealed the whole network mechanisms there um to build on the w3c standard so you know you could basically run that setup in any environment um like uh you know cloud trail workers node um aws lambda and all kinds of other weird places um so and we are investing and when i mentioned yoga like we are making sure that everything there works including subscriptions um in terms of like should you use subscriptions over serverless environments it's so today you can um with these libraries uh but the question is if you want to do that um but um but what i would but in as a general comment uh it's much easier now uh it's uh you know while other frameworks uh dropped support for it uh we're now have you know building support for it uh and also handling the network uh as just how the network is supposed to be and not tangled into the framework itself um so it's also very flexible you can choose whatever whatever transport you're using and yeah you want and things like that so if you haven't if you tried it with other frameworks and you haven't tried it with helix and or envelope and yoga i highly recommend you to try again that sounds that sounds awesome and and and um just to shamelessly plug my other question in again has anybody been thinking about flow control type semantics with subscriptions like i realized that one of the reasons why we didn't have to think about it early on is because um by default the kind of subscriptions that i did today were that we were working on with more live query style subscriptions where you don't need too much explicit flow control because you in a live query you understand the size of the response that you're going to get so you have a sense always of like um the the size the response you're going to get and you're in some control of the frequency with which you're getting those updates uh but if you start streaming stuff um and you're sending like thousands of events or hundreds of thousands of events like is the client ready for handling that does the client have a way of saying please stop um i'm still buffering right the way that like a normal stream flow would have so have haven't if you run into problems like that or uh thought about uh anything that you're using i was checking out the new websocket api that supports streams like websocket streams which has some support for back pressure built in but that's only on chrome but http 2 and sse i have no idea how um back pressure or any kind of flow control would work yeah i i see what you're saying just don't care about the client just nuke it it's okay send it all the events if if if it memory overflows it's the user's device who cares anyway this makes sense this is good yeah and also like what what is like a use case where you're bombarding a specific client with so much information um like i opened up my slack after a really long time and i have like a stream of events that come in and it's like two billion events that suddenly come in because i opened it after a month and my 2g breaks my battery dies my 3g brakes my 4g brakes 5g also like nothing will happen because it's like a billion events after a month right so that's a good one that's what i meant because it depends on kind of how fast the client can consume to render something right like maybe it can't render the list fast enough to to even consume the stream so it's interesting i never got into this uh issue yet but it's very interesting but uh but anyhow i think that was on the that those are kind of some of the streamed effort style uh updates that i want to talk about but otherwise i thought we could uh dive into maybe the first uh uh part of our thing that we kind of traditionally just touch upon a little bit of graphql onboarding graphical learning um any insights that folks would like to share from the last time we chatted for the last maybe six to 12 months on new things that you've been doing that have been kind of helping you accelerate graphql onboarding not just for people who are using graphql for the first time but you know maybe even other stakeholders in your organization like folks on the op side on the infrastructure side uh you know to to kind of get ready to deploy and use graphql internally you know what are you seeing uh as as things that are being successful that uh you know might be useful to kind of share with the with the rest of the folks here and for other folks on the live stream as well so any any uh any thoughts on the graphql onboarding and graphical running settings uh i have a little bit i can share um so we did two things in the last year they've been really really impactful um one we started uh it sounds really dumb but we created a internal wiki for just our graphql servers and how they interact um the amount of questions of people just asking the same thing over and over is is kind of crazy and if you just document it it's very easy for somebody to self learn at their own pace and if they need to go back and look at something they can do it so that's been you know hugely impactful is just doing that um the other thing which has been surprisingly better and better as time goes on is we've created cogeneran creating a subgraph for any of our our stuff so a lot of times if people come to us and you say i don't really know where this fits maybe we should make a new one and immediately it's oh i don't want to stand up a service i don't want to make it ready for load i don't want to i don't want to do any of these things right um so the easiest way to do that is just create a code gen you run one cli command and all of a sudden you have your own subgraph and you can move on um so those two things have been immensely helpful for just either helping train people or people who know what they're doing already just move at a faster pace all right yeah i can jump in as well um one of the things that i've seen over the last sort of six to nine months is a huge increase in content for those new to graphql as well um and it's really great to see there's so much kind of uh so many kind of things being shared on on best practices and uh helping people get on board and using graphql maybe for the first time in the position that i'm in i speak with people on a daily basis who are new to graphql and they're they're looking to kind of consume a graphql api and there's actually a huge amount of great content that is being created by so many people i'm doing doing um some great stuff right i've i've been watching um some of your content and reading some of that which is great and i know the guilds um recently kind of released a tutorial on the how to do uh graphql site which was it went sometime unmaintained or or whatever but it's great to see content being kind of contributed there again um and i think for anyone watching and listening that tutorial covers helix to believe um and building a service that's really cool um and tan my i know that the the the team at historia you really something new with the uh the the docs on onboarding people with with graphql in the different frameworks that's really really cool and yes it's kind of used in his sierra but you know what other better way is there to learn all these different frameworks to just kind of have something with graphql running and working and you can learn about the stuff on the front end and you know there's so much stuff going on in the graphql community if someone is new to it or kind of intrigued by it there's a lot of great content going on um i think i think the guys at uh graph cdn recently took over the podcast that the the prisma team ran so we're gonna see more content being created in different formats which is it's just great for you know the time to be alive they would say that's uh no that's that's that's great thanks thanks for the thanks for the note there as well but uh i think though yeah i think i think the um in terms of kind of like helping people get started with graphql and stuff right i think there's uh uh i i think there's like a lot of work to be done there and um it's sort of kind of organic as well because the community needs to figure out what the community doesn't know in a sense right it's like what are the gaps that we need to plug well we find it and then somebody will have kind of plug that gap and i think that's that's been that's been awesome to see um i think some of the challenges that we've been kind of uh facing are more around kind of uh a lot of kind of like these uh how to operationalize graphql because uh because those people are so far removed from wanting to learn new things often right on the off side they're like nope i don't want to figure out how to modify my log service or my metric service to make this work um so i'm wondering if uh folks have had experiences doing that on like the graphql onboarding for like not the graphql developers or the graphical consumers but other graphql stakeholders also like the ops people for example uh what do you mean by the office people the people running running the actual services yeah like people running maybe other services like i don't know if it's impacted them like even right like anybody who's like uh helping out with the apm side of things or the alerting side of things right like do they are they impacted at all or like uh you know are they impacted by the graphical thing at all like especially if you're running in a security sensitive context and people care about like these sim events right like security events like how are you folks thinking about uh like is that a problem and then like how does that happen yeah i i definitely think it's a problem i think um there's a lot of semantics um http 200 is one of those things where it's just you get these kind of blanket this is a successful failure kind of deal where it just doesn't quite follow the natural http thing that most folks are used to so you know your graphql server can be up and just not work you know particularly well um there's also other security concerns that i we've run into as uh as we've had like security folks looking at the servers so like introspection schemas are one of those things where it's kind of like a low-hanging fruit that you're not thinking about but is you know will leak your entire schema there's multiple ways of getting that um introspection there's a bunch of other introspection fields that you need to block but you also need it in certain cases for tooling so it's like there's this there's like a long checklist of uh after you've got you know you're like congratulations you have your server but like there's a whole long tale of all the little things you got to think about um between uh between you know that server and the next step that um i don't think there's a concise thing like so for example i've been talking a lot with graph cdn and as a cdn they have to um they want to be able to use the schema they want to be able to help uh people build and deploy really good graphical servers and so really they have to think about this advanced checklist of of all those pieces and i realized how many different things there are because there's things like persistent queries like you know lots of things like that and i i don't think that there's a concise resource that that gets to get you through that entire checklist yeah i i agree i also i would add to that that like um it's not only in my opinion just like uh uh an educational uh issue um or a knowledge issue but i think it's also some a bit about how we structure things today where um we can't hook in many of the tools we're using today we can't hook into the into the life cycle of graphql uh it's like we're doing everything before or after um and one thing that really helped us like and that's kind of like the reason we created the envelope and again i'm not trying to plug this thing i think actually this is something that other people from other languages should copy uh or should like look at uh is hooking into the life cycle because then um not only that you can um you know come up with like a resource or explain how to do things but you could actually create like a piece of code that could be reused no matter what tool you're using and then uh that's much easier to share uh like you know some examples about you know of you know you could hook into the introspection phase and you can do all kinds of things that let's say for us specifically in the javascript world like maybe graphql gs is not uh is limiting us in some ways or there are some features that are not existing yet and um and and it's hard to do for whatever reason but it might be easier in i don't know in other languages or other ecosystems i think hooking into that system can actually make it easier you can write like a very like good article about how to do things and then also offer a plugin with you know that makes makes this very easy to use and and and support these things um yeah i'll share some examples of like guides we wrote uh on the envelope website which i think are good examples because um we explain you know like what it means to like i don't know secure the security or graphql api but then also gives you just like just a couple of plugins you can just install and it's kind of like makes the work much easier than like uh i don't know a fancy um conference talk or like a really cool blog post but actually something that you could easily solve um and hopefully like you know that could be a new uh that could also lead to kind of like out of the box experience right like right now the you know the out of the box experience of using graphql is you know everything is open you're not using persistent queries like there's all kinds like defaults are very um insecure um here you could like basically build a default that is by you know it might not be the best for everyone but it's like it's a bit better i would say than the current starting point where people are um getting started with yeah that's definitely something i noticed was like every time from graph girl to the new team which i think this is like my third time is that graphql still hasn't really gone to that point where it's like you're not kind of either choosing something or building something out kind of from scratch and like no one really agrees on what that thing is if you're choosing something like react finally got to create react app right which isn't great but there is something graphql i feel like does not really have that yet especially given that you could make it as as yuri's thing is something that's language agnostic i think you get some some folks get lucky and they choose a thing like graphql java like andy mark and teams done great work on that and so you benefit a lot from all that stuff but if you are in another language because of your backend servers or whatever it is then you just you're stuck behind the times by default and that kind of doesn't set you up for success i'd like to plug in some more community resources and courses so i've noticed a lot of the people who you know i work with the clients they basically have a hard time even at the beginner level with graphql so they they are in the rest environment and a lot of the times they're looking to just you know consume a graphql api um and they have questions about you know which one do we pick is it apollo and things like that um so just um resources that help um get started and at least set up halfway is like you know they could still have some rest parts they don't need to completely put over to graphql these are things that um some clients aren't um you know fully aware of and that's where a lot of questions come from my clients um and a bunch of uh as pluralsight authors also got together and created a bunch of courses all related to consuming graphql uh from the client perspective and these are available on pluralsight and i've also noticed that evs courses on linkedin are also a big hit among newer people who like to get into graphql so we do have a ton of resources some free and some you have to subscribe to but there's tons of content at least from the perspective of consuming graphql you actually brought up a kind of interesting point where some of my clients and some of the people i've been working with are missing kind of the power of just what fields can actually do in a graphql environment we we're stuck in this rest mentality of i can only return the entity and the data around that entity but with graphql itself we're able to return more than that we can return the business logic from the server we can resolve it in whatever way needed for client consumption and i think that's something people forget does anybody have any resources they really like to teach that concept to others i know when i talk and i present on graphql it's definitely part of my conversation but it's not one that i can go and point and be like hey look at look at this thing that somebody put out there that really uh supports this um i sent out a link uh which is basically a whole path of courses on pluralsight and the very first course talks about just generically like what is graphql you know the tech agnostic part of it which is like a big picture and there we touch topics around you know what really is the difference between graphql and rest and you know getting them into the you know mindset of graphql i have a i have a question for i think mark and joey in terms of like some of the other this kind of uh graphql onboarding stuff because i guess it's been like a few years of like graphql kind of penetrating the organization right like what is what has it been like and mark given that i think like you just joined netflix like what was the onboarding for you like to understand what the internal graphql stack is like as an advanced graphql user already right as somebody is already familiar with graphql but like getting to the netflix flavor of like graphql implementation right so what's uh would love to hear from both of you and like what that's been like and any learnings there sure i can start um yeah that's a good question for for somebody who's who's used to a graphql stack somewhere than joining a new company with kind of a different one i think for me one of the biggest differences i was coming from github which was a giant monolithic graph to a giant federated graph at netflix that was kind of the main difference um and kind of like what it entails in terms of observability and how people are building graphs so um i mean the team i joined is just amazing so they were super helpful explaining context around it what they've built great docs were there already and then i think for me what was useful is like thinking of the problems i had before in our team and just asking like how are you solving this and then figuring out how it works at netflix um um but in terms of i think i want to touch on something john was talking about earlier with the 200 okay i think is the in famous example of if you observe a graphql api like the rest way or just like the standard http way you can get into trouble right um and i think that's that's definitely one part we probably are missing a little bit of information about is how do you observe a graphql api and with federation that's even more complex but even a graphql api like a simple mono a graphql api can be quite a black box if you just get started quickly like we have great docs to get started quickly with graphql but then when you deploy to production yeah if you only have a graph that's slash graphql and you get performance for slash graphql and you're looking for 500s on slash graphql it really doesn't give you a full picture so i think and you retouch up on this too is like are there errors happening at the parsing state validating graphql query custom validation rules um is it a server error is it a client error do you have errors as data which you would need to observe the response itself to know if it's happening so there's a ton of stuff happening here i think and we need probably to be build both better docs on what to observe and tools that help you do that and netflix is is great observability in first place i think and i think what a team has built for graphql 2 is is great so um there's one concept i want to talk about i think is super helpful for for graphical observability is just the concept of query signatures and query signature hashes so i think for for rest you know like you want to see what's going on there's a problem you'll see oh like the slashed user's path is having issues with graphql you don't have that out of the box and i think that's the query signature concept that is basically hey this query shape no matter what the variables are is having problems or this certain field is having issues but feel sometimes they don't give you enough context to know if something's going on so i really love the concept of query signatures and being able to pull traces out of a query signature um and that kind of stuff so yeah i think for me observability is a big one and mark the query signatures thing that you're talking about is something that has been is getting implemented or is that like a wish list of like a thing that you imagine exists it's implemented it's important so yeah there's the query signature the query plan as well uh if you're doing federation but query signature is just basically just they sorted without variables and just so that similar queries share the same signature hash so from there you can like build that into all your tooling right like if you have metrics you can tag by query signature if the cardinality is not too high you can go from a query signature to a trace to any sort of things in your tools so it's it's pretty amazing yeah and that sounds that sounds awesome and and and just to double click on that a little bit like is what are the kinds of errors that or like uh insights that you would get from query signatures that are um uh that you would like that would be harder to get from just like a field level analysis right other things interesting things that you've seen in practice like i imagine things where you have like certain dependencies between fields that cause a certain kind of weird behavior right but i'm wondering like in practice what kinds of like things you've seen yeah i think yeah you i think you touch on it like the dependencies between fields are often like almost everybody's using data loader for example which already kind of like muddies the water in terms of relationship between fields um or even if you're crafting like sql queries based on a shape of queries for example that will depend on even even the ordering could be tricky and in a different world in the streaming world yeah there's a lot of things that can happen and i think query signatures are just like such a close mapping to what we had as a resource before an end point right they usually help out a lot awesome awesome that's very interesting so i'm probably not gonna sound as as uh put together because we basically um we're kind of feeling things out as we go here at paypal and we thought that we could get away with apollo studio and like splunk signal effects the way we had it and that didn't pan out exactly so now we're kind of in this like okay what do we do now how do we solve this and the reason why we gotten this is because we were we were leveraging apollo studio for that visibility of 200 errors because they you know they they have i think a concept like you're talking about operation signatures and grouping things and figuring out um which operations are failing um things like that we thought that was great but two things happened as we started to like lean into that was that it didn't scale to like federation um as soon as we had a bunch of subgraphs on the same graph in studio we realized that it doesn't actually you can't like say i'm this subgraph owner and i only care about this subgraph it like looks at the gateway as an entire graph and does the metrics that way so i will be pushing on studio to kind of solve that subgraph story the other part is that and they made a good point that it's like studio can only see like response shapes and things like that so it can't do a whole lot so you're still gonna have to supplement that with some sort of like tools inside the resolvers where you know more is happening than studio can possibly understand so we're kind of filling that story out now like what kind of plugins and logging do we have to use and where do we put that it's most likely going to end up all all in splunk and signal effects like that's probably going to be our primary place for observability and we'll have to like like i said add a bunch of tools and some standards around how you deal with errors and how you log them but that's where we are headed oh and another interesting thing i just want to throw out there is not just 200s are hard to watch but as soon as people start saying and i think i see it on the side you're sasha for president so as soon as people start putting errors or or whatever you want to call them inside of their schema and they also want to understand when those happen that's another problem like okay so um there is something that we're working with apollo on on what we're working with i'm asking for and they're probably gonna make is this response shape um tracking where you can say like i care about these things and i want to be alerted on different shapes of responses so that gets kind of interesting one thing that we've done recently uh with a client of ours that used apollo federation was we basically took the um the federation execution from apollo source code um and actually replaced the apollo gateway with envelope so envelope just was like a drop-in replacement for the apollo gateway managing uh the federated services and that just gave us more hooks into what's happening on the gateway um so we could write all kinds of like custom plugins to start um just looking deeper into what's happening and then sending it to their you know whatever tracing system they're using and login systems they're using um we just start starting to do it so but it looks exciting uh i mean it's uh um maybe we they could also later build in the same hooks yeah and i'm really interested in using a lot of this stuff area that you're making and i would like to kind of champion the whole the guild and apollo can play together because i'm not like super cool with these these two sided things so yeah i'd love to blend a little more i wanted to mention by the way um at the beginning like people ask for like really cool uh resources and stuff i think uh uh graphql wtf uh is like one of the coolest things that happened recently in the community uh from jamie uh it's really cool like the videos are i think it's like the best uh more most concise and um efficient knowledge videos i ever seen um came across it was just about to mention it that's thanks so yeah that's yeah so maybe maybe the domain name wasn't the best [Laughter] but it sticks out yeah so just google uh graphql wtf it's really cool i highly recommend it yeah thanks i think one of the reasons why i started that was literally um at the time a few months ago i was kind of thinking what can i read to kind of get started that's kind of just concise there was a lot of kind of written documentation but video tutorials were quite old um you know there was a lot of kind of video tutorials on youtube showing people how to get started with yoga the the other version so it's great that there's a new version i can't wait to create a video on that um but a lot of the content in video format was kind of outdated or it was too long so um and while that long content was was really great if you had the time what i kind of decided to do was just kind of create short concise videos that show people maybe how to do one thing so reason why i'm here today really is just to absorb the knowledge of all of you and try and go away and learn and and dive deep on some of those topics and hopefully share that with people um who who don't know um do that as well so um yeah thanks for the shower but uh i'm looking forward to all the content that everyone creates awesome so we were talking about um graphical learning experiences and we used to kind of talk a lot about other other related topics as well uh one of the one of the items we had next was talking about um just kind of diving a little bit deep and just doing a little bit of a round the table update on uh what folks are seeing with you know authorization uh at the kind of graphql layer and generally kind of sort of alongside it as well um so we can uh we can move into that topic that sounds cool dustin does that does that line up with the agenda like our bank yeah that works out great awesome so yeah we'd love to hear uh how folks are uh have been approaching authorization security if uh you know any anything has changed over uh the last few months and years on on kind of internal approaches things have become kind of more problematic or things that become simpler by going down a particular route um i i think like on i just kicked that off with an update on kind of how we're thinking about it i think we've been been seeing a fair amount of success with um model level authorization so the idea is like whatever though the authorization rules are all specified with the model and then as the resolvers um try to fetch uh or try to resolve parts of those models the authorization rules then kind of kick in and compose automatically so that uh it doesn't matter if you do like query users or query comments dot user um the same authorization policies get applied and there's an opportunity for doing a fair amount of static analysis in uh deduplicating kind of repeated authorization rules and stuff like that which makes you know the data fetch and the check a lot a lot a lot faster um so so there's some interesting things there uh but that's kind of been working like that's i think that's been kind of the that's been a very interesting pattern that we've been following that's been working well uh but that's that's on the authorization any any uh anything that you folks have been looking at when it comes to authorization and uh approaches or questions that you folks have yeah tom i'm curious um from what i've heard from your talks you know you provide the data sources as well so doing the authorization with the data models makes a ton of sense um does this fit in well with like people who already have like a rest or a grbc layer and they're trying to make which is another common way of you know where graphql pops up is i've got all these different systems and i want to put a layer on top of it yeah um i'm curious about that because that's what we're doing so i'd like to hear more about authorization strategies when you're doing that versus yeah here's my schema give me my data sources that's great but we don't have that yeah that makes sense i think what we've done for um services when we're bringing kind of models that have been specified by services but not that are not kind of serviced by uh some kind of data model or physical model underneath right where you can't really do any of the query optimization type stuff we do more of the um the field level kind of the schema filtering right where uh you don't get where you have visibility on certain fields but that's the that's the policy that we attach at a model level um we're not like it's not feasible to attach uh any kind of business logic style authorization policy at that if if the if the data is coming in from a service right that's been that's been hard to think about the only kind of idea that we've played around with a little bit that i've seen also in industry not in the graphql space but like in the data space has been um that you effectively have to kind of um materialize that service internally like you have to fetch that data and then operate on that data entity to determine if that entity is accessible or not which is way too much work to the graphql layer it seems better like to push that work into the grpc service of the rest service itself so um i i think so far like what we've been seeing is the most convenient cut is that the authorization policy for what entity is accessible or not is just kind of pushed into uh the service itself like the grpc or the rest or the soap service that's servicing the model um it's been hard to do that at the graphql layer anything that's not like field level security uh meaning this field is visible or not right has been hard that's been the impression that i have any any i would love to hear from other folks on if there's any interesting graphql authorization stuff that's been happening for services i'm curious joey um when you say like you have existing res and grpc services and now is are you talking about like offering alongside res and jrpc a graphql api or like moving towards a graphical api is that what you're talking about no i i just a lot of time my stocks are like um you know you can i think they're my i'm more familiar with the absent because i haven't used lesser yet um sorry tommy but i would like to but basically yeah you develop your scheme i think and and your models or your rules go with the databases but there's a much more clean like paradigm there it's just like my data and my my fetching layer essentially and it's easier to work with that way but if you just have like you know a bunch of rest services that have been here for 20 years or grpc or whatever and now you're saying hey i want this to be in a graph so you can have unified access how do you translate that stuff like what are people doing like right now i'll just say so you guys know we're using really simple auth policy directives that go on objects or fields right objects just cascade really really simple stuff but they don't do anything very complicated right now they just i mean that we have the concept of scopes and application context permissions between different apps and whether or not you're logged in but it's not like um i don't know we don't do a lot of like role based like cascading roles and stuff like that yeah i think things like scopes and things relate to apis in general make a lot of sense as like annotations on types and i think what time i said is key like the fetching users or comments users should result in the same like authorization results and if you're doing perf like field level authorization only you're gonna make a mistake eventually um and i think that's why maybe like people say sometimes like hey graphql is super hard to to protect because you can like access types from different places but i think if you concentrate your efforts on protecting types rather than fields and use field like security as an exception that secures you a lot from this but what i'm curious about is authorizations uh like um i don't know a user cannot add a comment to this pull request because it's not part of it the team that created the pull request like complex rules like these um and they usually make more sense i'm of the opinion they make more sense like deeper in your domain logic or your service layers but one thing i'm really curious about is that it's kind of intention of your the main layer being user aware or viewer aware should i say so and in my experience i've rarely seen code that's already set up this way where where every function that exists in your domain accepts kind of a viewer um and so that's it's really nice to be able to push that authorization logic deeper but often the existing apis of a big code base i've noticed don't necessarily support these viewer aware apis i'm curious if anyone has been successful pushing that deeper into their stack or you're more doing like resolver level authorization checks so a previous company we were able to solve this pretty pretty well and this was back in uh 2018 so still pretty early graph fuel days um but what we did is because you have the contacts available at every resolver level we had an additional we called it a data filtering object but just another entity we kind of put in between the end result of a resolver and the data fetch layer and everything had to pipe through there and so we were able to do all of our user role based filtering at that level and in the case of fields we were also able to handle some of that behavior so it would strip out anything that your user role didn't have access to as well as handling general like you don't have access to this entire entity object it's the cleanest approach i've seen so far and i've continued to adopt it on future projects as a result uh but it's one of i'm sure many options that are out there for that one are you declaring then the sort of restrictions in the schema or is this in the resolver that's doing the filtering and validation um so we actually separated out the logic a little further so the resolvers relied on a business logic layer case and that business logic layer accepted the entity scope and this was also true in the uh previous iteration which was a rest api so it was already kind of baked into the entire system and we were able to just translate it's like hey here's the same user we had to change how the context was read from just because the rest implementation and the graphical implementation were ever so slightly different uh but that small difference it was just handled at the business logic layer though and so then all resolvers benefited from that abstraction layer which was i think really helpful in the overall migration um but then i've seen other systems where that wasn't inherently baked in and so adding we had to add another layer on top of the business logic layer strictly for authorization and i think that just that's where it gets messy so it kind of like mark said if it's not inherently baked into a system already it's exponentially harder to implement later but worth doing in my opinion the reason i asked we went down a rabbit hole of attempting to declare these things in the graphical schema um i think it was an interesting thought process because it's useful to be able to map those uh say authorization errors to domain things or you know just be able to see it in one place but it definitely got a little hairy um but yeah it was under the hood applying something similar to what you're talking about in terms of um different role checks and things like that based on the context but uh i'm not sure if it was worth the extra work versus just skipping it and going straight to the resolver yeah i i think i mean 10 mi kind of alluded to this but trying to keep that logic out of the resolver i think's worth it because you don't want to overload the responsibility necessarily like i i've seen some resolver schemas get out of control i mean we're talking one model and it's already a 4 000 5000 line file of resolvers and their functionality because abstraction got a little crazy um that's just been my experience though i i think i think although a lot of new patterns are kind of against adding more modular layers to things i think it's really important to still do that as the ecosystem grows especially in a large enterprise application yeah that makes sense one one like this stuff that we started like thinking about was um you know we i i'm with the same opinion that like it's always good to push it into the domain layer because you know there there's like knowledge of you know i think authorization always tied to the to the business logic and it's much easier to manage it there the knowledge is there but what we tried to think of was like even though you're doing that could you still let's say take that logic that is being written by the people who know you know those rules the best but then also use that um that logic inside uh the graphical execution um like so you could very fail early for example um so um i mean um we have some new things there like recently we also like um the matik the guy who wrote graphql shield um joined the guild and uh and then we got actually another group uh that worked on a new uh and on a new library that's called graphql of z and we're going to publish their blog post soon um we're thinking that that might give um i mean it was still i don't know like there's still it's a question there like what's the best approach i think there's a but i think it might be cool if we could kind of like um enjoy both having the logic somewhere um uh close to the business logic but then still fail early at the gateway um so yeah maybe i'll share later like a blog post might be related to them this library is pretty cool it's very similar to an approach that we were doing around um being able to declare all of these rules and how to combine them and stuff like that do you know if the types themselves get changed um as a result of using these directives like the code generated type uh it's a good question i think that i i we haven't looked into that yet but it's a really good question it's worth looking that's kind of the what we've been uh doing john in the sense that like the types do change but it's we we call it scope based schemas or role-based schema so the idea is that for a particular application scope you're gonna get a schema um but the name of like the user type that you get for a scope called user and the user type that you get for scope called something else right like manager or admin it might be called the user type but it might be different but it's tied to a particular scope and so the assumption is that any application or any client is using the api as a scope right within a particular app scope right um so they're getting kind of the types that are code gen for what their type should be right um this fits in like well the schema visibility part of the authorization discussion right but not necessarily with the authorization authorization part of like what data entities can you access and what data entities can you operate on that part is still like closer to the like deeper in the layout basically right it's closest to the model because it just needs to be there i guess um but but yeah i think schema visibility has been the only thing that's been trackable to specify at the outer-ish graphql layout right and and it still feels okay um as soon as you as soon as we try to think about like putting in more authorization rules at a graphql schema level but not at like a model level right even resolver level is too detailed and graphically level is far too detailed uh like it's too verbose it's it's like it becomes too much so schema visibility has been easier but anything more than that has not been nice to put on the authorization site in practice i'm curious about folks have done the sort of mass like gigantic monolithic graph and um and then cut out those what you're talking about these sub the subset graphs uh related to the role and executed things against that is it i guess that's what you're talking about somewhere yeah that's that's what i was that's what i was mentioning i think i think joey was mentioning last time they've been doing something similar right they've been trying out like this schema visibility thing joey have you have you folks been implementing that how's that been going sorry about what exactly like basically the idea that like you have subsets of the schema that are visible to certain scopes like you don't get to see the entire graph depending on the role or the scope or that you're in yeah i mean there's a couple things going on there there's there was our concept of like bffs on the graph that's a whole migration strategy like something completely different um and that and for that those chunks of graphs are definitely locked down to a particular client through a scope and that off policy director i was talking about and then there's the visibility portion of it which is uh essentially going to be using apollo contracts so you have to authorize or sorry authenticate before you can uh even make a request to the graph so we require a bearer token and based on that authentication we understand who you are and we understand which graph you should be seeing um so we'll essentially send you a copy of the graph that we want you to see based on tags so we'll tag our graph up and say like this is the merchant graph or this belongs to this and uh we'll assign um tags to groups of people this is vaporware right now and as i speak so uh that's what we are hoping will will happen um once all the contract work is done and i just want to throw in there that i like this introspection topic i know it gets crazy um i i don't know my gut feeling is that introspection shouldn't be a security issue because you know these are queries that you're that's like your api that's like saying ah you can't see this rest route it's like no people know the restaurant exists you just can't access it right so i don't know i'm kind of of that manner and if you have these contracts and you basically say like i know who you are and i'm only giving you a graph that you can access then what's the harm in letting them introspect that because that's something that only they can play with anyways so for sure like we don't have introspection turned on because our security team freaks out and says this is the rule we read it on a blog and uh for me i'm just like okay well that's great but if you change this one sentence of it enables the hacker to i had a really nice snippet basically they wrote a sentence that said this allows the hacker to see all of the schema and do some uh malicious things i said replace hacker with developer and malicious with you know like i forgot but the sentence read really well like this provides the developer a way to see your schema and use it completely so yeah yeah makes sense but i think it depends on like how mature the security side of like the tooling is right like if like you have like really good like things in place to let people like suppose if you have like for example like good persistent queries and allow this thing in place right then it doesn't matter like go take my schema party with it like who cares right ultimately you're not gonna do anything because i'm not gonna let you do anything in production right so um yeah so there's like everything in graphql it's very contextual what are you doing what's your setup um uh are you a public apr you're not a public api you know everything matters are you generating schema from your database and not really thinking about it too hard like yeah i would say everything goes into whether or not you should allow introspection yeah but i don't think a blanket statement just turn it off in production i just don't like that that's not sitting well with me it kind of defeats the purpose of like this whole idea of having like a schema it seems like a really powerful thing that if you did it right it can be awesome and people can just play ground on to your stuff yeah i know you can avoid having playground but that's not the point yeah makes sense well i think what we're going to do is take a quick five minute bio break give people a chance i know we go for a while and give everybody opportunity to coffee uh and and uh and then i think immediately after that we can dive into favorite topics of the day always uh the everything everything related to microservices federation what's the updates there what is the wish list there we've been we've been kind of touching upon some of those topics already there uh but would be useful to kind of get an update and like just uh updating thoughts also from from folks around the table so um i think maybe we can touch on that right after coffee slash vibrate yeah we'll return in about five minutes you guys need some of that apollo summon music yeah we've uh it's uh it's it's it's like i think it's going to be like a feature that zoom at phone like whenever you're in the waiting whenever you're in a waiting room or in like a breakout room they'll have like a little thing that lets you like some chill house whatever the genre whatever i'm always dancing in those those summer breaks yeah i don't know music i don't know if you've used uh slack huddles yet but uh if you're the only one it starts doing that i i thought my spotify went off quite a number of times it's like hmm what's happening and then i realized it was slack being slack and then it muted me and then i was really upset i used huddles but i don't go in there by myself so that never triggered i i get in there and then if somebody doesn't join with like 30 seconds 45 seconds of you joining they turn on the elevator music and auto mute you it's an interesting feature yeah on the particularly dark days you can just start a huddle with yourself and then just you know listen to the music as it comes those are my mondays so monday will be my self huddle days i'll call them office hours and no one will come actually really great because then you can put it on your calendar and uh you know joey's working having you have so many office hours these days you guys don't know how truly weird i am at one time i had a meeting and it was all dark in the room and people came in and there was like whale sounds playing like and i was just like guys welcome to the meeting i was i was a scrum master at the time they don't let me be scrum master anymore at that one time joey did it and that was it well i think we're gonna pick back up it's been about five minutes maybe a little longer give people a minute um i wanted to shift now into the amazing world of federation and microservices and all the things that graphiel can do in that area uh i don't know if roy's back yet but i know he particularly had a topic uh surrounding this but if not does anybody have any uh any oh there he is sorry i didn't mean to rush you back okay um so roy uh sent a question in earlier uh or wanted to talk about federation being supported by graphql servers to make it uh possible out of the box is that generally the topic uh well something i wanted to know about was federation how people are doing it i mean it's step 7 we have an approach apollo has an approach i believe azura also has an approach uh so i was also thinking uh graphical mesh can do some sort of federation bring services together uh so i was also thinking like what should be should it be like a default way to do federation generally a lot of people are using i think the approach that we've been recently trying out that so we do it at two levels right one is kind of the service to service federation and one is kind of the more data source to data source federation right it's like i have two different data sources and they have like physical models or logical models right like and i want to combine across them um and so we use a slightly different execution mechanism for that but like the way it works is that you bring in data source one you bring in data source two you specify like a logical model on data source one you specify a logical model on data source two and the logical models might come from like five different tables or like have some computed fields or whatever right and then you say that there's a relationship or a join condition uh that you specify as a part of the graphql server's configuration and then hustler is able to take a graphical query and create the right query plan and do like temporary tables and predicate push down like do a bunch of things that makes that cross domain cross data join very effective um when we do like the service to service stuff the graph cryptographical stuff we follow a similar kind of semantic model where um at the graphql servers configuration you configure saying that type one from this graphql service and type two from this graphql service have a connection um so then all the resolvers get updated and stuff right like that just happens this is not very efficient like as efficient as possible from a query perspective because we don't control the data sources but it's uh it's pretty much i think the best that you can do like with good query planning and data loading and like memoizing and like all of the usual techniques um but but the specification of like what you're joining type things is done at the graphql server configuration level so that's the difference that that that so far seems to be interesting for us like we haven't uh expected that the services will publish the joins like the relationships if you know what i mean like service one says i'm related to service 2 service 2 says i'm related to service 1. that that relationship is not happening at the service level it's happening at the graphql level if i'm making sense but that's been like how we've been thinking about it so far would love to hear kind of how that's evolved for other folks yeah yeah it makes sense so what i what i dislike of some implementations is that underlying services need to be ready for federation well i would be saying like it's really nice if every schema can be federated with any schema on the planet basically exactly exactly right but then i think you'd have to do that configuration at the entry point right at the graphql entry point rather than at the service yes you would say this is my server um there are tons of other services and from my server i connect other things exactly exactly yeah something they would always have schema a and b that you want to connect through schema c yeah that that that's that's what it sounds like but uh um uh i'm just gonna pick on like uh mark and joey again because i think they've been doing some like interesting microservices federation stuff right like what does that mean like folks uh in in terms of kind of where this where the like asking the service owners to publish the relationships and stuff like that like how is that evolved and like what what are your thoughts then i don't want to go after mark again so i'm just going to jump in uh we so we we do apollo federation and right now we are not federating anything so our router is a glorifier our gateway is a glorified router but we have our first use case where we've got you know two graphs that want to play with each other so that's exciting um i will say that as far as like coming coming together on like a unified federation spec that that would be great um but let's keep fighting it out because even within apollo they've now switched from federation one to federation two which solved a bunch of problems that we didn't like about the first one so i think there's a lot of iteration that needs to happen here so like yeah let's all keep duking it out in our own corners or whatever um but i'll have more when we start doing it right now federation is still kind of this like concept that we're like ah it's gonna happen soon funny i just want to say joey has been overly humble here having seen a few things they've been working on a few sneak peeks um yeah they're working on pretty amazing things so uh and i think actually we're working on pretty similar things so um as some of you may know i was once pretty critical of federation or at least careful of a federation especially for people starting from a monolithic architecture not not going too graphql for federation uh but using federation if you are in a microservice context in the first place um i think it's beautiful um if you have the observability for it and the tooling for it so yeah um netflix uses uh federation it it is based on apollo federation spec um the difference is we do have our own uh gateway um built in kotlin but it is fairly standard apollo federation yeah i'm not sure what else to talk about but it's been working pretty great especially with the observability stuff i talked a bit earlier oh i will mention one related thing um we did something i guess cool which was there's a problem with federation one in that i don't know if this happens in any of the other implementations of federation but if you try to like have two different value types or two different enums that are different from each other it like blows up and what that means is that if you have to change an enum for example and you have maybe 50 subgraphs you have to change that enum and all 50 subgraphs if it's present um they call it schema shuffling right so you got to do this lock step thing or it just seems like it sucks federation 2 is supposed to solve that i'm not going to like pitch that right now but in federation 1 the way we solved that was we added a subgraph registry i use a lot of what i like to call gorilla warfare there's not a ton of resources on my team so i just do things that i think can get done one of those things it's not pretty but i just threw a github repo on top of apollo studio so effectively the way that you manage your subgraph registry you know where all your subreddits are and what routes they're on and what schemas are responsible for um is that you i took away all controls from the rover command line that apollo gives you nobody can use that for this particular graph only the ci can do it that's in the github repo you just make prs and as you make a pr it'll do a check um in that pr you can affect multiple subgraphs so i can make a pr that updates the enum in all 50 sub graphs and and that kind of thing so schema shuffling is not a problem if you have a layer in front of it so yeah that's pretty cool and also you can revert things that's another big deal we can revert changes and see who did things does that mean you have a monorepo for these schemas for the subgraphs even though the services are not in that same repo absolutely yeah there's a there's just a github repo it's called subgraph registering it doesn't do anything except for have a config of the sub graphs and then the actual schema files referenced um yeah we also built in some notification system there and you know yeah it's cool though how have you maybe solved some of the front-end consumption through things like we use cogen a lot and how have you kind of solved that particular problem with some of the spiraling effects of federation as you kind of mentioned um sorry this question for me like how do we generate queries for clients based on our yeah or like you know as the idea of federation is we have the the single point for anybody to consume our graphical api and it stitches everything together for us at least that's my understanding of federation uh and i'm simplifying because i know we have some people in the audience who aren't following as well um but how how do you provide that to the client consumers do you just say hey uh the the front front-facing federated graph that you've produced is now your source of truth or do you let people code in from all of the different microservices yeah no they have uh consumers have no concept of federation right it's just this is api.paypal.com fql that's the graph um for all intents and purposes they have no clue what's going on underneath so they'll take that super graph schema and do what you normally do with the ground so essentially at that point they don't know that makes sense and then i i know we were talking about joins and uh i forgot whose opinion it was but join should exist in the kind of super layer uh so to speak uh how with all with like enums being maybe a layer down in some cases how would you handle handle that like have we seen have we has anybody experienced that general question out there what do you mean exactly so i mean the question just to just to add to that or like maybe maybe paraphrase that right i think the where do the dependencies or the relationships between type score do the services publish that themselves or does that happen in the central graph earlier like what has worked and what has been better and i think like uh uh mark given that you've been kind of using that with the federation style i think would love to hear your thoughts and then woody of course we've been doing a bunch of work uh in this space also would love to hear from like how that organization of where these relationships exist go or does it not matter because everything is in a mono schema repo right for what joey was saying like there's a mono repo that has the whole schema so it doesn't matter like where the relationship is written down or expressed um so for us everything most things are in their own service so there's no motor repo um and they registered themselves to a schema registry that usually happens during a build step ci we have tooling to register against that registry validates a lot of things it can lint and make sure it will compose as one big graph correctly um yeah it'll do everything basically that you're sure that it's going to work when a client queries a super graph um and where these dependencies are defined we're using the apple federation specs so um it's implicit in some ways i think with the next federation 2 you can kind of you can share types without necessarily explicitly saying like hey this is a dependency you're kind of like all working together but there's also using directives um to kind of instruct the gateway about those dependencies and making sure the gateway is able to build a query plan but that's kind of the beauty of federation is where you don't you don't need to be overly explicit about all the dependencies and you trust that the gateway has enough metadata about the subgraphs to compose them and being able to build query plans and magic isn't that query plan right like like databases that's the future federation is like building the best query plans uh we can which is a very complex subject actually i mean we talked about this subject a lot here but one thing that i was always interested in i don't know if you could share it here but like i was always interested in facebook's implementation like um you know like if we're talking about graphql at scale and um you know handling crazy amounts of uh like huge graphical schemas and uh with crazy performance i don't know if there's you know something that can come close to this and i wonder you know uh how they are doing it uh yeah i don't know if you can share or not leave them um yeah i can talk about that and hey thanks for uh having me here i joined a little bit late my name is lee i'm a co-creator of graphql and the director of the graphql foundation um and spent a long time at facebook of course where i first started graphql with a couple other of my colleagues um i'm hearing all this conversation about foundation and marc andre mentioned that he used to be kind of uh cautious about foundation or maybe even to the to the point of not recommending it um i'm still there for what it's worth i i think i think foundation federation brings a ton of value but at a lot of costs and schema registries query plans like there's there's just a lot of additional stuff that you have to manage facebook doesn't have to manage any of that they don't use federation um instead they use a model that we're actually now exploring at robin hood as well uh which is the you have a single definition of your schema that exists at your gateway facebook's architecture is a little bit different from what we would consider sort of like a modern api architecture they have uh their quote monolith but really their monolith is a stateless uh gateway um it's just capable of doing a lot more compute they do server side rendering there in addition to api processing but that's basically what it is all of the definition of graphql all of the schema all of everything that a stateless tier with localized caching so that when you need to load any data what happens is you load out to any subservice that might own that data and uh and cache it locally in in the case that you need to touch it more than one time that's where the data loader pattern came from was managing exactly that kind of access model i think the modern version today is probably best exemplified by by what twitter has built which is using some kind of modern communication model between services and then leveraging that to build out your your graphql model so um i'm pretty sure twitter uses jrpc that's that's what we're exploring at robinhood as well servers to service communication always happens over grpc it's it's far more efficient far more safe than graphql for server-to-server communication i will continue not to recommend graphql for server-to-server communication and recommend grpc in its place but graphql is dramatically better for over the wire from your services and gateways down to your clients and that that's what it was designed for um so the the architecture that is i think one step evolved from the facebook architecture is um facebook uses thrift internally which is basically jrpc but having some kind of grpc for service to service and then accumulate all of that together at your gateway tier where you either manually or automatically define your your graphql schema that has a number of interesting properties one is the source of truth of changes over over time for your schema is source control there's no like registry that you have to keep track of or side logs or uh or anything like that where you're trying to keep track of how each services have changed its its data models and how that aggregates um there's no complexity there like the complexity is just git log like that's how you figure out what's changed over time um that is incredibly powerful when you mirror that with a mono repo and even if you don't even if you have sort of like a side loaded module uh repo for your clients uh in facebook's case they have a monorepo at robinhood we've also moved to a monorepo and that allows us to say you know given this directory which houses our api definitions if anything in that directory changes run the code generators and it runs code generators for client-side consumption and it's the exact same model that you might expect if you're familiar with jrpc where you change your drpc definition and when you do that you simultaneously change your producers and your consumers of the grpc data that's a super compelling model and being able to leverage git as your source of truth for for change and triggers uh is really really nice in terms of how it interoperates with a bunch of other ecosystem tools that sort of treat source control as the as the truth and and don't need to know anything about graphql let alone apis or even web traffic they just know about files on a computer changing and and that's a really nice sort of bottom layer proof it also has this nice property where downstream services don't need to worry at all about api traffic all they need to know about is their um their own sort of internal uh under the hood api contract it's not about web apis but purely about like data and data out to that service um and to the degree that jrpc is a good model for that there are others of course then you can have service to service communication outside of the gateway as well so this is another thing that we've explored multiple paths for and what i think is interesting is both paths end up being valuable this was true at facebook i'm finding it true at robin hood as well one path being direct service to service communication you have service a service b and a gateway that ties them together with graphql service a might need to talk directly to service speed and it wants to do that directly over grpc that's completely valid or you have a or b or server c that needs to talk to a or b or some combination of the two and it actually wants to consume that through graphql and so having a pathway or hopefully where it doesn't need to sort of like exit your vpn and pop back into it again but you can sort of talk directly to the gateway and then say hey gateway actually have this complicated query that's graphql-based and i would like to load this larger amount of information i'm willing to pay the minor graphql overhead in order to do that is also super useful when you're not entirely sure what what service a particular piece of data comes from nor do you care and you want to buy that um that abstraction layer in place where you know if service a migrates to service b server c never even needs to know about that because it only ever went through the gateway in the first place and that remains super valuable um so that's that's i think the evolution of facebook's architecture which is monolith all graphql definition happens at that monolith which is some combination of automatically generated queries plus our schema plus a little bit of the manually generated schema which is layered directly on top of in facebook's case thrift which has a programming model interface on top of it or jrpc lee just a quick question there in terms of like how this is organized because in terms of like the developer who's working on a feature for the api right um do you expect are the teams kind of getting neatly isolated that like i have the service a team which is publishing in thrift or grpc and then i have a graphql team which maintains the graphql api contracts and how the mapping is done from the graphql schema into what services are being called is actually consciously maintained by a bunch of humans right at the graphql layer or is that piece also automated or is the same developer kind of working also on service a and then also doing a little bit of their work to help you know bring it into the overall graphql schema like what combination works well um that's a great question and the answer is it's actually a little bit of all depending on what your intent is and what you can get away with in terms of automation so um there's a lot of stuff you can get away with automation where you describe say you want to um anything that happens over this jr so it's over graphql you just describe how you translate jrpc to graphql and then any change downstream to graphql or to grpc as soon as you change the grpc definition you change the consumer by changing the consumer you also change the graphql interface and you have to be intentional like there's code review right like you can see what the producers and consumers look like when you do that change so your unit test run and everything like that but you're not writing additional manual graphql specific code you just in the same way that if you change an upstream thing and you automatically run your code generators to change say like swift uh like type definitions you wouldn't want an api uh author to go write ios code but you would like for their changes to have downstream effects that you could automate if you want and you'd want to run unit tests so i think that's the base but then there's this um this understanding that there's a meaningful difference between a services capabilities and what surface area of that set of capabilities you want to expose over your api and you either need to get into the business of writing sort of allow lists and blacklists on the fields and your your grpc fields and then trusting that those will hold over time or you might actually want to have some derivation between you know grpc prefers communicating in this structure but in graphql you want to use something else maybe in grpc you have something string based but you actually want to hard coded as an enum when it gets the gateway or vice versa maybe it's an enum at the jrpc layer but actually you expect that genome to evolve quite a lot and you don't want to turn clients and it's it's actually more ergonomic to describe that as a string or maybe there's some other kind of drive data in place whatever it is there's some intentional designed piece to the graphql layer so in terms of the operating model what this what this looks like in a mono repo this is a lot less painful because it's all one code base and really what you're talking about is directory a for your service versus directory graphql for your gateway and all of your developers just kind of know where the code lives and it's not a big deal if your code is developed across multiple repositories then it's certainly a little bit of extra weight because every developer needs to also have the graphical gateway sort of checked out so they can like first i do my service change and then i go to the graphql and i and i also add some capability there which could require two commits for an amount of repo it might just be one um but then it's it's an intentional choice where you that last piece is is shared so um in both facebook's case and at my team at robin hood um we have a team that owns that so it's not a community-owned thing there's there's a team that owns it and that team is accountable for the health of the graphql service itself um if there are a new common api definition so if there's like a generalized enum that gets used across the whole surface area they might own those things um general patterns you know logging visibility all of those kind of concerns would be owned by a single team and but then there's a an open contribution model where you know okay we own the the graphql service but then there's you know a subdirectory or a sub piece of the schema that's very clearly about this one vertical of the business or product and like clearly that is owned by some other team they come in and they contribute to that over time uh and that i found was a very viable working model at facebook it's well aligned to facebook's culture which is if you need to get something done you just figure out what directory the code lives in to do that and then you just get it done um robinhood actually has a very diametrically opposed culture where there's a strong sense of ownership and every single service is well owned and every piece of code is well owned by a particular set of humans but even in that model i found that it works quite well because you still like still even though you have the top level is clearly owned by our api team and then like each piece below it is clearly owned by the teams who contributed that piece of schema um but it's true that then you end up with one service that has a large amount of people contributing to it but i found that as long as you have good tooling in place that tends to work pretty well question for you lee is that layer i'm curious about the resolvers so you've got this grpc layer um i guess there's a fair amount of standards that maybe you want to follow for for being able to code gen the schema that that accesses those but are you writing resolvers as like directives or something or is there actual code that's running at the graphql layer that's then calling the grpc layer um it's all code based so um in neither my old facebook team nor the team at robinson do they write the schema using the schema definition language they write it in code which has the benefits of having the full power of the programming language at your heels facebook of course used hack at robinhood we use python both languages have the ability to do annotations which is a hugely valuable tool which is gives you a similar kind of expressiveness as directives especially if you want to leverage code generation so uh for example our python type definitions read structurally very similar to a schema definition and where you have you know class whatever which is the name of the type you're describing a bunch of fields that are essentially stubs for um for the resolver functions and then the decorators that describe the how that ties to some piece of auto-generated architecture and then like you run some cogen and then it fills in the blanks for you and you're good but you don't have to if you if you take that direct decorator off then you can just write a regular function and that function can do derived logic you can do something else and whatever you want and then that way as people are writing the code they can kind of change like what elements are going to be automatically generated we can do that entirely where i just say like this type is generated by that jrpc definition and then it's done or i can expand that out and do it on a perfume you can expand it out even further and go all the way down to writing resolver code um at which point i probably there's there's another abstraction place which is you you kind of don't want to write directly against a grpc consumer those tend to be kind of low level and fiddly to work with you want to have a little bit of a layer between the two especially if you want to do localized caching and or like a put data loader in place and rather than directly writing data loaders as part of resolvers we tend to auto generate those um as part of our jrpc consumers so everyone has to look everyone has to learn a shared language to be able to contribute on this graph there's no polyglot ecosystem in this model right yes there's like the one language of truth um for us that's python plus some my pi types which is done interesting um we actually had a prototype version of this before that was built in kotlin which i think kotlin is an awesome language and i'm grumpy that we didn't go with kotlin but the rest of our company uses python and so that's what we were the most we have a bunch of go and rust and other things too but yeah it it does require that you end up with one coded service with a language that you expect the vast majority of your your teams to be able to contribute to would you then recommend for for ecosystems that have to be polyglot like paypal's never gonna say that you have to choose a language you know it's java or node right now um would you recommend then i'm just curious federation for those companies or do you think there's even still a better solution than that um well to be clear i'm not saying that federation is is non-viable or bad like clearly all of y'all are using or many of you are using it and finding a ton of value from it so that's great i'm i'm describing an alternate model here which is um you know the the graphql all lives at the gateway and it's designed rather than generated from um from federation but you might choose federation for a bunch of reasons and i think that could be one um maybe wouldn't be the top of my list i think uh uh like disparate organizational structure might be the primary one especially if you um i think especially for for intuit intuit has a lot of business wings that basically never talk to each other so it's not even about um whether the languages are similar but like is there good communication streams across everybody who contributes to that schema together maybe that's the reason and you really want to give autonomy to each of those teams um at facebook team autonomy was an anti-pattern we our culture expressly um did not like that because we we wanted to build the organization that recognized that the product surface was one thing perceived by a customer so we wanted to treat it that way on the software level uh at robin hood we're a little bit in between you know we of course have pushed things towards a monorepo we have a strong ownership model but there's still a sense that teams need to interoperate together in order to get something holistic done but if you really value team autonomy then i think federation is a fantastic tool to manage that for polyglot environments i can see it going multiple ways i'm i have high expectations of of engineers i expect a good engineer can learn multiple languages so the idea that like hey my my gateway's written in kotlin but like all of my engineers are java it's like get them to learn kotlin you know and it's like if they really like you you you know you that ends up being a brick wall and you're like i just i can't convince my uh the folks across the company to agree to contribute code in any one particular language then then sure federation can be a way out um but i i don't know that makes me that makes me a little bit grumpy a little bit sad that the reason you would go to federation is you can't teach your engineers new things or can't get them to learn a new thing um as opposed to a lot of the other i think like much more valuable reasons that you would choose federation yeah talk about this a bit uh if you don't mind uh i think like from our perspective from the graphql communities perspective we often like only see graphql itself and kind of ignore the underlying concerns but when it comes to like big companies like we talk about facebook netflix there is a ton of work that went into the culture the organization structure the tooling how they operate already that makes a huge difference like i wouldn't go work at facebook tomorrow and be like you know what um teams need to be more independent they need to be able to use their own language and we're going to use federation like or at least i would need to have a very very very good reason to propose that same thing that i wouldn't go to netflix and be like you have all these services that's over we're we're going to build this one big monolith and everybody's going to contribute to the scheme out there so i think we often forget about that in conversations like you want to graphql is often not that huge of a deal compared to everything that's been built like forking the language making sure her model it is able to work making sure your source control system is able to work at that scale like there's a huge amount of work that's unrelated to graphql that you can't just like change like this so yeah i love that point and um it reminds me of uh conway's law right your your um your organization uh tends to produce software that's shaped like that organization and i think conway's laws is written in terms of a of a like non-designed or an unintentional result it's like you you intentionally design your organization structure and then like whoopsie like your software now looks like your organization structure but i i think like if you understand conway's law then you can leverage it as a tool so it basically says like the you know the river flows this way and you can swim downstream or you can swim upstream and swimming upstream is really really hard and streaming downstream is like really easy and when we're talking about these architectural decisions uh the the net result is actually like very similar right like to the joey to your point earlier clients don't see any of this like they just see your company.com api slash graphql or wherever you host this thing and like check it out they get a holistic api that's what our front front-end developers want in the first place like fantastic however it is that we arrive at that conclusion it doesn't matter that much and so we should choose the one that is swimming downstream for your organization rather than swimming upstream so if if your organization is structured in a way that truly values autonomy um values polyglot environment then like pushing everyone to like force them to contribute to a shared monorepo of where your api definitions look like that's going upstream like it's not gonna work but like similarly so if if the team really values interoperation between teams and uh and high touch on the api surface layer um and like strong connection bonds between individual service teams and the front end engineers that consume them then federation just feels like extra stuff to work around and would be pushed back against and you can sense my bias i tend to gravitate towards teams with the culture of the latter but i absolutely appreciate that that's not the scene the thing that we see everywhere i love that point more country i do i do want to call out also um one of the pain points we're having with federation since we're using it is that you can federate like technically but if you don't federate uh from a point of view of like how you approach the schema so schema design review process if that's not federated properly you're in trouble and we're experiencing some of this pain right now like great technically federation's solved everyone can be their own stack and whatever like let's all join the same graph but the moment you start to join the same graph there needs to be some sort of federated review process and that's kind of like what we're trying to wait through right now is like how do we do this so that it's not just the wild west and people are doing whatever they want and then it that doesn't turn out that great so that's our greatest challenge right now federation there's i have an interesting story here as well which um i know facebook is is unique in many cases but this might actually parallel the experience many people will go through you're in an organization you already have services running you already have a product and you're trying to figure out how will i introduce graphql so you get an organization buy-in people like graphql seems really cool okay now like how do you approach this problem that joey just described where okay everyone's kind of into it but they might not know how a well-designed scrapbook api might look like and you want to make sure that what you end up with is high quality so what we did is we of course we started with a very small set of folks essentially the people who built graphql and we designed i think news feed at the times api and a pretty small surface area and uh we basically said every new thing that's going to come into this um is going to have to go through us we're going to you know heavy design review process and we did that for a while basically to make sure that you know if you needed to represent a date that we ended up with like five different ways to represent dates like we just would push it towards one and we we eventually sort of built a library of case law and precedence and we could point to stuff and eventually we ended up with like a bunch of really good design principles that we liked and also we had all these people who kept coming to us with you know hey i want to change this piece of the api what do you think and we also had these people from across the company that um had gone on this journey with us and so you know if we're the supreme court of making api decisions then these were the you know the case lawyers um and at one point we said okay case lawyers you're now the federal judiciary bench or whatever you're you you are now the the folks who can take this settled case law and bring it back to your parts of the organization and in that sense we we actually really did federate design review um we called it friends of graphql and uh you and you know these folks were the design reviewers for anyone in their part of the organization so we had someone on the photos team and someone on the messages team and so on and so forth for a bunch of these surface area and as their groups kept working on graphql they would um you know try to make sure that locally they're making good design decisions and if they weren't sure they could always sort of bring it back up to us and we would talk about you know particularly challenging things um and that ended up scaling well there ended up being sort of two layers to that where we had um in each area you know one person who was sort of the the person who'd been around for a long time doing this but then they would also sort of delegate out other sub-pieces so you'd have you know someone within photos and then another person who would own you know specifically the albums uh part kind of making up this example uh but you you end up with this like you know two-layer thing that ends up with a large number of design reviewers that all sort of point back to the same set of case law and there's a path for you know flagging issues and raising that to one group so you still have one sort of authoritative group that can make tough calls about holistic design decisions while still having like a broad set of people who can help locally make good decisions um and then everyone there had the same set of working principles knowing that you know hey we're empowered to make local decisions of what's right for our product but at the same time we appreciate all this comes together as one api and a front-end engineer is going to see one api and they're going to think weird if there's like many i don't think the end result was perfect but the end result was like very workable and changes happened pretty quickly and didn't get blocked on our centralized team let's get it here that's what that's what we're doing uh we have domain architects and we're building out like a training process because they're all rest architects now but they're going to need to be graphical architects and they're the ones that will sign off on their domains so that's hopefully that'll work the same way love it i have also a not a question like hypothetical question about um you said a bit about like that you recommend you don't recommend we would recommend that say grpc for service to service um i wonder um you know part of the cool thing about graphql is that because you know the intent of the of the consumer so in the query of the consumer you could evolve the apis faster uh or easier right you don't need to make a breaking change and uh you basically can say okay so this specific field uh hasn't been used for a while um but you know and in an end point you can't do that right yeah you just you get the whole endpoint i wonder if um you know do you see that also being useful for service to service communication like and how could we see some ways where that we could get these benefits also over let's say grpc or i don't know service to service communication in general that's a really good question um it reminds me you know another thing that graphql gets you that jrpc does not is sub selection you know with jrpc there's one shape of data you get the whole thing which for service to service communication is actually the right trade-off because bandwidth tends not to be your bottleneck it's compute and sub-selecting costs compute um and uh you know having the same shape does not and you get some cash benefits along the way um but your your point here is a good one that you know how do you know you can deprecate a field uh but especially if generating the data for that deprecated field is costly like how do you know when the right time is to actually kill that off grpc has a model for this through their field deprecation strategy basically you know you you mark a field as deprecated in the in the definition itself and then when you do producers you produce a consumer which has that deprecation information flagged that says like hey that the producer wants to get rid of this you should be aware here's the migratory path like you can add comments there in fact graphql's deprecation function is largely inspired by grpcs um that's that's where the inspiration for that came from um that's only half of what you're asking though the other is like well how do what about like stuff that i don't know it's like you're not aggressively trying to deprecate it it would be nice to just kind of keep things tidy and clean if no one's ever consuming that in the first place there i think it's a little bit of a mixed bag and it depends a lot on your um your source architecture so with with graphql especially with a public graphql api you have no idea what people are are doing aside from your logs so it's incredibly valuable to be able to uh break down each query and get a and get a real sense of who's hitting what even if you do have strong source control and every consumer of your api is in your source control um especially if you're shipping mobile clients you might have old clients in the wild right like ship ship software ship software you can never take it back and those folks might be still issuing queries for for deprecated fields and so having a real source of truth of who's querying what is is super useful uh for service to service slightly different set of problems exist you you tend not to have shift software um you tend to have control over all of your consumers not always the case but i'd say like 99 of the time your your grpc consumers exist within your like service vpn which means you can go look at source code and the really nice thing about grpc is it is entirely predicated on code generation you you write a gopc file as a source of truth and you co-generate in both directions um and so you can then like do whatever you want with that um and so you know if i were if i were being charged with like hey i want to go clean up and i got a bunch of these old fields that are like kicking around and all my drpc definitions i want to figure out which ones i can go delete um i would add to my code generator logging on the consumption side i would say like okay i produce this stuff and then i consume it and by consuming it's like a bunch of getters um and every time i hit a getter i want to you know increment a value in a counter and then i can essentially get the same result but i'm doing it on the consumer side rather than on the producer so the producer doesn't know it's just yielding values but the consumer does know and that logging is where i would get that um there's a little bit overhead there but there's also a little bit of overhead for adding graphql field logging so like i think it just shifts it so i think you can get that if you know where to put it but um yeah it's a slightly different shape from graphql it's for sure yeah i mean i'm asking because one thing so i thought about it a lot i never got like to discuss it with a lot of people so it's pretty cool that you're here and just like one thing that we experimented with and we're doing now is that we basically let's say we have we have a bunch of services that are written in grpc uh so we generated we generate sdks for them um but then we actually also do the conversion uh to graphql on those sdks so then like you could actually let's say you get an sdk where you could query graphql on the sdk on the consumer level you log those uh you both log those um uh queries and convert them into grpc so um then it's kind of like you you could play around when you have also this option you could like play around with the different um like because because it's those trade-offs right like because if you many people we started that because we saw a lot of people actually putting graphql on the uh provider level because of those benefits but then you know like you said caching is harder and also the data manipulation work is being done on the provider which is not necessarily the right architectural choice um yeah but but also we weren't like we did it and then we weren't really like super experts on grpc so uh maybe we felt like there's something like that already in grpc or we're just i mean like we're doing something really weird here or yeah i don't know i think that's part of what i like about the grpc ecosystem is that you can actually bend it to do a lot of things it's uh if you can't tell jrpc and and a lot of the things like it were part of the inspiration for after all graphql thrift proto buff captain proto um i'm a big fan of all those layers and borrowed from borrowed many concepts for them for graphical speaking of ideas that uh influence graphql uh lee would you like to lead us uh through a little exploration of what's new with graphql with the foundation any foundation updates themselves um sure yeah there's i think a handful of super interesting things happening right now um i'll start with foundation things and then we can get to what's new and with the the technical side of the foundation because we have the two halves two interesting things happening on the foundation side one is as as many of y'all know we have a working group that contribute changes to the graphql spec itself that's super healthy that is a community driven thing so anyone who has an idea can show up to those working group meetings um they bring their proposal they talk through them they get advice and suggestions they move their uh their proposals along and eventually we add them and there's a bunch of very interesting and super powerful stuff in the works there um that group is overseen by our technical steering committee and our technical steering committee was put in place about a year ago for the first time by me which meant that i just picked a bunch of folks um the intent is for the that technical steering committee to essentially run itself in the long run i'm trying to get myself out of the business of handpicking these things and deciding things by myself so we have an elections process going and we had a bunch of people nominate themselves to be members of our technical steering committee and um that nomination process just closed about a week ago and uh we're about to run a vote so um that's one thing i think is pretty interesting especially for many of the folks who are paying attention to the development of graphql itself um so that's one thing that's going on that's pretty cool another thing that is is really awesome is you know part of the we started the graphql foundation for many reasons one of it was that we wanted to actively encourage community-driven development of the core of graphql and one way we can do that is by financing it and we have a bunch of members and sponsors of the graphql foundation we've built a modest budget and now we're taking a very significant part of that budget and distributing it back out to the community via grant program so if you go to graphql.org today and um i think you scroll all the way down to the bottom it's in the footer but there's a link there that's called the community grant program trying to remember the exact url if someone finds it go ahead and post it in the chat um but you'll find there there's uh we just opened it up about a week ago for the first round of submissions um you can write about what it is you're going to do that'll help the graphql community and apply for one of these grants we'll probably uh accept a half dozen to a dozen per quarter we're planning on running this every quarter and each grant is going to be somewhere between a thousand and ten thousand dollars so um nothing to shake a stick at uh also not um enough to live on so don't quit your day job and go apply to graphql community grants but hopefully meaningful enough to buy people's time to really invest in making the community better and the technical process better itself so i'm super excited about those things on the technical side we just released a cut of the the graphql spec that's a fairly minor thing in terms of its impact on everyone who's using graphql but this is actually a really valuable thing for the set of folks who contribute to this it's a bookmark at the end of a very long chapter of the journey from graphql being a facebook project to graphql being a community owned project um our previous spectrum was that's part of the reason why it took this long is we i now know way more about open source law than i care to and we had to kind of run a process to make sure anyone who's ever contributed to the spec agrees to a new thing that basically says they're we're not going to sue each other for using graphql which means this actually is really important there's a bunch of companies out there that explicitly say that they can't use graphql because they have some concerns that at some point in the future they could be sued because some arbitrary contributor to graphql might decide to go do so uh that that can't be the case anymore um so that was one thing that we fixed uh the contribution model itself was improved um the the tool that we used to deploy the spec was improved and then of course there's been a ton of super fascinating changes to the spec itself um i wrote a blog post um about this uh about a month ago that documented a bunch of the changes in there there's numerous um but it was actually have even accelerated since then i would say if you if you look at the a handful of our recent working group meetings and just kind of pay attention to what's going on you might get the sense that a lot of things are changing and uh that's true if you look at the spec cut and see like what's happened in the previous two and a halfish years years more is happening now than happened in the previous two and a half years so i'm actually even more excited about what's coming next which includes nullability operators streaming results input unions and many other very significant changes that's my stream of consciousness big update but i'm happy to dig into stuff and talk about it and ask answer questions quick clarification was uh the title of that blog post graphql 2021 that's the one fantastic i will share with the community should do it the opposite of community grants where we can pay for certain things let's get those input unions i mean that's actually um kind of part of the intent of that i know um so for for those who uh who don't pay close attention to the working group um might be surprised to learn just how much work it is to make a change we you know uh some of the some of the changes that have landed in the last year or so um we're in process for multiple years and uh you know our intent is not to make every change take multiple years but um it's not uncommon to take at least a few months even for something that might seem like a small change which for people who are familiar with contributing to open source software feels glacially slow because they're used to oh i found something with an open source project i care about i'll just write a pull request and send it in someone will go review my pull request emerge it and the new major version will come out and then i'll be able to use that thing that might happen over the course of a few days or maybe even a few weeks and not the speed that we use um if you compare that to say the pep process for python or the javascript ecma rfcs that's about the speed that we run at right where those those proposals show up they get talked about for a really long time we want to make sure every single change we add to graphql meets an extremely high bar for uh usefulness for quality basically anytime we we start with the assumption that if you change graphql you have produced cost every single person who uses graphql now needs to make sure their tools didn't break that their existing schemas didn't break um if there's a new thing they got to go teach all their colleagues that new thing there's a lot of cost with even a very minor change and we need to make sure that the value for such changes meets that cost at least if not exceeds it and ideally exceeds it um and that there's no like edge cases and corner cases where we go oh whoops and then like we've got to go change your mind and go fix that or a patch release like we don't do patch releases we we get it right when we land it um and that takes time and because it takes time the people who are the champions for these changes uh spend a lot of time on this stuff um luckily there's a lot of companies out there um you know big props to to those companies who support their engineers for for owning those changes and so we have folks who show up who say that they use their you know company sponsored time to work on these things for 20 or 30 of their engineering hours and they you know work with us every in every monthly meeting over the course of an entire year and and then ship something and that's fantastic others don't necessarily have the luxury of that and they need to borrow time away from work or maybe they um do contracting or something like that and they're literally borrowing potentially paid time away from themselves when they do that um and that's the intent of the contributors grant we want to make sure that people who are who are really benefiting this community by building extremely powerful upgrades to the core of graphql itself aren't just financially compensated but are made sure that they are having to give up some make sure that we um provide some flexibility uh for the folks who do that great work lee i got a question from our viewers uh this comes from ilia and they're asking could you talk a little bit about the history of directives in the spec it really looks like it was added as an afterthought even though since even ast is very different sorry for butchering that yeah um ifilia has has follow-ups um let me know uh in case in case you dropped a piece of the context i'm happy to give some of the context there um i'll give a a brief history so graphql i would say went through three major phases of design uh phase one was 2012 through 2015 and i would say that would be uh you know we'll call it like uh graphql v1 or in facebook graphql or something like that which was uh the very first version of graphql that we designed um within the company and out of the gates i would say we did a pretty reasonable reasonable job most of our design decisions from 2012 have lasted all the way to today nearly 10 years later but in that three years from 2012 to 2015 is when we were that's the time period where we were doing that uh design review and federating that out and a lot of stuff happened really fast uh we did not have graphql mutations um we we did not have uh interfaces or unions um we did not have a lot of things we didn't have uh named arguments um there's there's just like a ton of stuff that we did not have we didn't even have fragments so if you wanted to repeat some piece of query you literally had to copy and paste it um and over the course of those three years we patched in features right like our our customer base for graphql were other engineers at the company and so we knew that if we made a mistake we would just fix it we'd change it again and so our change threshold was relatively low and there was a lot of exploration really quickly by 2015 we had something that didn't make a ton of sense um there was a lot of design inconsistencies within it it had kind of gelled to a uh a state where it was hard to make changes anymore um and people largely viewed graphql as complicated and especially the query language is complicated um so in 2015 this was right around the time where the the relay project was also getting some traction actually the relay team wanted to open source what they had built but it made no sense to open source relay if graphql was not also open source um at the time i think we really undervalued just how important graphql could be and thought open sourcing would maybe be not the not the best use of our time um luckily please thank the relay team they convinced us that we should open source graphql which is what we did but i looked at what we had and said you know this is a little bit of a mess if we open source what we have uh i don't think this will work and then also like what the heck would that even mean are we gonna go ship a php library like most people don't use php um so it's probably gonna be largely ignored or treated as like oh graphql that's some like php thing right it's like that not where i wanted it to end up um so that was when we we started a redesign we'll call it graphql d2 or open source graphql as opposed to facebook rfql and kind of like stripped it all the way back to its its bare essentials and put it back together again and uh this is actually where our rsc process came from because when i i started i led this process um i didn't do it alone i i was doing a lot of the design work but um all of my graphql team were the editors so i would write proposals to say like here's how we should do this we did it this way before old here's how we're going to do it new and i would get a ton of feedback on that i'd iterate i would say two-thirds of my proposals got killed and one-third survived and the result is the the graph pill we all know and love um the accumulation of those proposals that were accepted by my teammates was a pretty different version of graphql syntactically similar in many ways but um very very different set of features it actually took facebook nearly two years to convert from old school facebook graphql to open source style graphql um they're there now but it took a little bit uh directive getting back to the question um directives was one of those proposals so um directives was not part of the original 2012 version nor did it appear anytime between 2012 and 2015. largely because the um directives shows up in two places they show up in the query language itself and then they also show up in the schema definition language so schema definition language actually wasn't even not even part of the original open sourcing of graphql just the query language was we added that the following year um but within the query itself this was a solution that i proposed um of a very long tail we had a bunch of experimental features for our internal version of graphql facebook graphql with varying degrees of usage and i would say the most important of those that got a lot of usage was skip and include so you could decide that a field should be skipped or explicitly included with a boolean variable that you could pass up from the client and the way that that used to be described was as an argument um that seemed really weird to me that like the same arguments that you pass in as data into your your resolver uh are is syntactically identical to this like special case argument called skip or include um and then there were a handful of other experiments um like um uh streaming actually even at that point we had an experiment with streaming where stream was like a weird annotation in the form of an argument and um we had a couple other things that um that showed up as like syntactical additions my concern was that anytime we wanted to try one of these experiments when we had to muck around with the syntax itself but that put a very high bar on running the experiment and so directives was a way to say this is a this is a way out you don't have to change the syntax at all and you can experiment with a bunch of stuff so in the sense that it feels a little bit tacked on it's kind of true it was like it didn't survive for the first three years of graphql's existence it was added on as part of the open sourcing effort but it was designed it was intentional it was to say here's the handful of things that we're already doing um that should have an explicit syntax but rather that syntax being the skip and include syntax like it should be a a um a door that allows you to uh open into a wide world of of user-provided um capability uh when we added the schema definition language uh we made sure to to bring directives over to there as well and i'm happy we did because that's how federation works that's how a lot of code generation tooling works um that's you know a ton of people leverage directives for exactly the reason that it was originally designed which is to add additional um information and capability beyond what graphql itself describes ilya had a follow-up uh for example list ast node for schema and directives are very very different are there any plans of fixing this in the future i realize that's very very hard varies were in all caps a list ast is different from a directive ast um you might have to follow with me later i don't know if i if i understand the question completely understood ilya we encourage you to reach out to lee on twitter conversation never stops here of course uh and this as always is a great space to get access to you all for a little bit and ask questions and hear what's going on in the community and uh i think on behalf of the community i would like to thank you all for coming today and sharing all your amazing insight on all the different things that are happening in the graphql ecosystem and community i know we weren't able to cover everything i'm sure we could sit here all day and talk about it though uh so like always the conversation doesn't end here it continues beyond uh you can find all of us on twitter i'm sure if not uh anybody not on twitter there's also the graphql community discord um i don't have that link available but i will grab it real quick and share it discord.graphql.org discord dot graph fuel dot org done so yeah if you're not part of the conversation not part of the community definitely come join us uh tanmay anything you want to share in closing no i think that was uh that's i think that's a good uh conclusion and uh thank you everyone for joining us for uh the great conversation so far um and uh and and see you again i guess in a few months when we do one of these again with the latest and greatest updates that are happening um i think a few interesting graphql events also happening this week right this graphical galaxy i think that starts that is it starting tomorrow does it start today um i think workshops are tomorrow okay i need to give a workshop tomorrow so yeah awesome cool so uh there's a bunch of graphical stuff happening in the community that all of you should uh tune into as well um but otherwise thanks a lot for being here folks awesome thank you everybody and we will see you all at the next one bye

Channel: This Dot Media

Views: 188

Keywords: graphql javascript, graphql

Id: yPxkcq_uvzU

Length: 148min 40sec (8920 seconds)

Published: Tue Dec 07 2021