An Iterative Approach To Decoupling Drupal Sites in Gatsby / DrupalCon North America 2021

uh so many friendly faces in the uh the chat this is uh iteratively decoupling your existing drupal site with uh gatsby um there's a bitly link in the the session chat and also here on screen to the slides and some resources and uh we'll just dive in because we only have half an hour um so i am uh brian perry i'm a front-end architect at uh bounties i love all things component related and also all things nintendo currently playing uh ori in the blind forest which isn't really a nintendo game it was originally an xbox game but it's great and it's a lot like uh metroid so i think it counts and i'm on twitter at brycomedy and on the web at brian perry.dev and uh how about you introduce yourself matt yeah i'm not seeing your slides update but i'll just uh continue on here um i'm matt i'm gonna be talking about the back end of the front end i'm a lover of all these devops and all things outdoors uh you can often find me hiking i'm really excited it's hiking season uh find me on drupal it crash and on the website vermeer.gov so do you still not see the slides matt i don't are other people i can see them can uh can everybody else in the uh yeah everybody else in the uh the chat can so well matt you've got some time to figure that out uh so uh we uh matt and i both worked together at bounias and um so uh drupal is one of a handful of things uh that we do at bountieus um but we have uh we both primarily work within the drupal practice uh get to work with a bunch of great drupal folks learn so much from them and uh just in general if you're looking uh to do great drupal work we're often hiring and if you're looking to have great drupal work done we'd be happy to get someone in touch with you we also have a sponsored booth as well so uh matt and i are gonna kind of hand off uh kind of shifting between the uh the front of the front end and the back of the front end on this project um but first i'm gonna start out by talking about kind of what went into our decision to decouple the bounties websites specifically in this way where we started small and kind of progressively added to it so the original ask that we had was really just for a couple of key portions of the site the home page and this new co-innovation landing page we had a pretty uh different new design compared to the rest of the site which also implied a bunch of new behaviors and animations that weren't present on the current site and then the rest of the site the idea was that it would really just have a mostly cosmetic overhaul to kind of update global styles and get things in line with the rest of the site and this was definitely a a pretty familiar point for us we considered the possibility of decoupling the site and introducing new front-end technology in the past but had trouble justifying it specifically a full-scale overhaul large-scale decoupling it's tough to balance that with our competing priorities both for the site itself and also um you know compared to client work and you know this was also just us maintaining our own company site and then this uh new design concept kind of put us at a similar crossroads where there were a couple of pages uh or a couple of templates that we really thought would be benefited by introducing a javascript framework and decoupling portions of the site but we wouldn't see quite as much benefit for the rest of the site because it was mainly just a cosmetic overhaul and the intent there was to keep things in their traditional drupal pages um so you know decoupling the site just to ship those two pages really would have drastically delayed things but um you know as you can assume by the fact that we're even giving this talk we did do that so the question is you know kind of what changed this time and things kind of changed for us when we started asking this question like could we actually just start by decoupling only those two pages on our site and you know we didn't really know if it was possible given what we had but we found that once we started looking at things from that perspective and assuming that that was possible it started to open some doors and made the idea of starting to decouple our site possible and yeah as it turns out we in fact could decouple the site um so we uh and decouple the site in that particular way so we initially launched with two pages just those two main pages that were built statically with by gatsby and then all the rest of the site was still rendered by drupal and it laid groundwork that allowed us to now any page on the site we can have rendered primarily either by drupal in twig or gatsby using react so let's talk a little bit about the overall structure of the build and how we approached it so again it's really just our one site but we do have multiple front end technologies here uh drupal is still the cms back end regardless of um you know what front-end technology is being used and we're using gatsby to build a subset of the site statically and um drupal is still handling the vast majority of the pages on the site and then we also have what we're calling our front end globals kind of in the middle which has some shared components and also global styles and then things like design tokens and variables that are used by both systems and then just a little look at the uh project structure we decided to kind of take a mono repo style approach and have all these pieces live in the same repository um so we have you know three main top level directories that front front end globals drupal's doc root and a gatsby directory we just found partially because of the timeline that we were under um you know the fact that this is all one site um it just seemed to be more efficient for us to be able to just manage this in one place and not have to manage separate dependencies and separate prs across multiple repositories and then we kind of already covered off on the front end globals but that's kind of a shared place for assets that are going to be used in each of the different front ends and then just a little bit about uh you know the react and drupal sides specifically so for the react end of things we really focused on trying to build functional components with as little internal state as possible and that made it uh practical for us to use these components in a bunch of different contexts so in gatsby with data sourced from gatsby's graphql api directly within drupal or even cases where we needed to just pass in hard-coded data as well and that also you know made it really useful from a prototyping perspective and we use storybook heavily in this initial phase and that also opened the door for some of our react developers who really weren't drupal developers or interested in ever becoming drupal developers to contribute we had some developers who never had to spin up a drupal environment but were still able to make um you know really important contributions to this project and then on the drupal side of things we ended up creating new content types to represent the decoupled page templates we kept using paragraphs which we were already using heavily on the site i was under the impression that that paragraph data structure really wouldn't lend itself to this sort of decoupled build but we found it uh pretty reasonable to access uh those entity relationships via the graphql api and the gatsby source drupal plugin and it also had the side effect of uh really providing a high level of layout flexibility in that we could reorganize the paragraphs uh in drupal and affect the layout of the page that got built by gatsby and uh because there are still a number of traditionally rendered drupal pages we still have our regular drupal theme and you know as we mentioned before that's pulling in things from the front end globals but also has its own styles and javascript and templates and all that so for some of the more uh back of the front end things we can hand over to matt who hopefully can see the slides by now i can yeah technology is great so yeah i'm gonna talk about hydrating gas feed or how we get data from uh drupal into our static site so the first thing uh next slide you'll hit is the plugin in the mgm world called gas resource triple great plug-in and essentially what it does is it crawls all of these on api and shows all the data exposed by your api into your graphql database allowing you to query it so here we have an example of our configuration um we're able to set the base url via environmental variables that allows us during our build process and you can tell it to point to like a dev environment or proud environment um you'll pass an authentication and then we have this section called disallowed link types so this is something you'll see really often with gatsy search drupal and essentially what it does it tells the plugin to completely ignore a certain json api path so you'll see this really common uh with the file endpoint so you know if you're accessing data over json api and you're accessing your files endpoint you're gonna grab every single file that's exposed in your triple installation you know that's a lot of data you might not want to pull all that in um so we can completely ignore that specific endpoint and the plug-in won't grab any of that data but what we do later on is we can tell the plugin to include specific references when it queries for other objects so you'll see um in our filters section what we're doing is you know for a specific endpoint in this case the node client story we're telling it to include uh various logo fields and what that will do is when json api returns that endpoint it'll also return the files object with that with that response and then all that data will be put into your graphql database we will be able to access it so next now that we have all of our data in graphql um we want to access it and put it into our react components so the way we did that is through a pattern uh you're sometimes referred to as higher order components um we call them providers but essentially what they are is a function that takes in a json api object and returns a pure react component so in this example we have our home page hero we have a couple of fields that we're grabbing off of our paragraph object and then we pass it directly into the react component so that we can have the separation of concerns one thing you'll also notice with this file was that we have a graphql fragment so this we're not going to get too in depth of this but um with graphql one of the things you're able to do is create fragments in other like areas of your application so you don't need to have your entire query in one location so think of it kind of as like creating a subroutine in sql but you know we declare it within these provider files so we can have the query associated with each provider directly in the same file um yeah uh then next so we build out a bunch of those providers and then we essentially have a a function that is responsible for taking in a bunch of paragraphs and turning them into a bunch of providers so you know this is a just a react hook that we created called paragraphs to components you know it takes in a simple object and for each one of those objects it does a lookup of the the nc type or inside of the bundle type of that paragraph and it'll map it to that actual react function that add it to that object for calling later so we're we're handling the mapping between the bundle id and the function name manually in this file we i wasn't really able to find out find a way to do it dynamically later in this file we also have a more complete graphql query that will directly query our node and use all those query fragments to pull the different paragraph types uh with responsive words so that's once we have all that um wired up you know we have our page template where we call that react component so you know we're getting in at this point we're getting in a a data transfer object that represents our entire page and it has you know put the actual paragraph fields on it so on line 14 in this example uh you'll see that we're pulling in a variable that will control whether or not our footer is expanded and then we pull in the actual paragraph field and then we call our react component and all that maps into and we get all of our component providers and then we call each provider and each provider will return to react component which is what ends up building our pages awesome so now that we have our pages built i'm going to talk a little bit about getting those pages out onto production so a little bit uh about what we change with our build process so this site it uses aqua blt to build uh if you're not familiar with blt it's just a build launch tool great set of ci tools that allows you to to manage the apps processes of your drupal site essentially what we did was we just added a step before the aqueo blt build command that will run our um npm build process which will generate the static sites and then we copy the output of that static site directly into our document so that when the artifact deploy command runs it copies that with the artifact and pushes it out to our server yeah okay okay so codes on the server we have our static site how are we serving just the subset of pages uh this is a portion of our hc access file where we're doing that um so we rely heavily on environmental variables that we set as we parse the path so in the first section we're checking for certain static paths and we set a static environmental variable then in the next section we detect we just the general public back path and set a gatsby variable and then the last half of this is where kind of the magic sauce of just directing certain pages happens so the first line um we just grab the request uri and we throw it into a variable and then we check to see if the if that path exists under our public directories so take that variable which is some percent one like for example if you access boundaries.com innovation it'll check to see if onions.com you know public slash cultivation exists and then it'll check if public slash connotation slash index.html exists which is the actual file that gatsby ends up generating for our clean urls that we pass to it and if that file exists then we serve it so pretty straightforward and then later on with our environmental variables we do some fancy stuff with catching so you know if it's a gatsby static path we tell it to cash for a year if it's a less static path and we only want to catch it for about 15 minutes and that allowed for like a deployment to prevent errors from being seen on the site but uh using environmental variables like in this context really helped us to keep the access clean and you know make it clear like what we were trying to accomplish i did a whole bunch of other stuff on the site but uh we're not gonna be able to copy uh had a lot of fun with the buildbooks module so if you ever use that module before the buildbook module essentially gives you a button that you can use to trigger a remote endpoint so we ended up implementing a magic button for our content team where they on the production site they you know update content they want to see a new static build they push a magic button and it triggers uh a build hook so it triggers our bit bucket pipeline to to start a new build we'll push it out to aquia and then we can later deploy it and then uh also really to that we did um set up uh gatsby live preview using gatsby cloud um for a variety of reasons i feel like that hasn't become a critical part of our workflow yet we did have a few hiccups getting it set up but but also you know in the meantime we had our magic button and you know that might indicate that in a lot of cases the you know being able to uh trigger their own build that they can see in the staging environment is good enough um but i'm hoping that over time that can also become you know a more traditional preview can be part of the workflow um but the the other big takeaway is that uh the patience and understanding that our content and marketing team showed as this process changed really made a huge difference so uh last thing as we kind of wrap up here um so once we reach the point where we introduced the structure and had these two front ends it was interesting to see the places where the lines kind of blurred a little bit and just want to talk about some of the impact of that so this is a screenshot of one of our drupal rendered pages where the header here is a react component and the rest of the page down to the footer is traditional drupal twig templates that power this so we had to solve the an integration problem the other way where we wanted to syndicate our header and footer that are created and react back to drupal so to do this we created a essentially an exportable version of the header and footer components still kind of follows that higher order component pattern that we saw before and then we created a a separate webpack configuration that just used these components as entry points and then bundled uh versions of them into the disk directory in the drupal theme and then to simplify the actual integration on the drupal side we use the component module component not components that one one letter is important here and what the component module does is it allows you to find in a yaml file configuration alongside your javascript that allows you to define a component block so that will allow you to have a block for your decoupled javascript component that you can place in drupal's block ui and you can also do things like define block configuration like we see here so we're specifying the ability to pick between a dark and a light theme so when this block is placed we can pick between the theme and have you know different variations of this component on different sections of the site and the configuration is added to like the wrapping div as data attributes so it's pretty easy to get at them in your component to use that data in this case pass them in as props and then you know we also had the challenge of uh ironically getting the data back into these components because our separate webpack build uh didn't have access to uh gatsby's graphql api so initially we just solved that by hard coding things but as we continue to integrate these components we really wanted to be able to have uh the same data set drive both of these obviously so the the solution that we came up with matt uh put this together based on a post that we found elsewhere on the internet is actually using the cached uh graphql json data um when we're building this component and you know it definitely feels like a little bit of a fragile solution but it's working nicely for us so far and allows us to use this exact same uh you know data in both places and uh this means that the gatsby build needs to run before your drupal build so that cache data exists um but uh we're already doing that so it worked out and then a last thing you know especially as we got past those two pages and started introducing more decoupled pages templates throughout the site a common issue like working with our design team was they were you know creating new designs that had a combination of react components and drupal components and you know it really i feel like it felt like them to kind of an arbitrary as a like an arbitrary divide um which is understandable you know why should they care they have this playground of components why can't they just use them all so that was kind of an unanticipated challenge um and we really the real solution for that is to build out more you know react versions of these drupal components um but thinking about this uh kind of from a wire perspective there's some other options there one thing that we've played around with and really doesn't feel like a long-term solution is is we did put together some ways to syndicate drupal's markup back to gatsby and make that available in the graphql api and we're doing that in some cases and it works but it has some limitations the biggest one being that it's referencing the images on the drupal site and they're not part of the build asset um but the other thing and this is definitely something we you know we didn't know at the start of this and probably not something i think that we'd you know go back and change but this actually is a really good use case for native web web components if we had built these things that we were going to use in multiple contexts as web components we could have used them both in our javascript framework and use those same custom elements in our twig templates um so that's just kind of an interesting concept for the future especially as i uh continue to learn more about web components so you know kind of wrapping this all up what did we learn here um you know i think this iterative approach certainly doesn't eliminate all the challenges of decoupling and as we've all seen here it also introduces some new ones that we had to solve but it really can be an effective way to prove the fit for decoupling for your project or your organization you know it's kind of a low investment low-risk way to figure out if this really is going to provide benefit and for us it really was the difference between decoupling now or decoupling possibly never or just kicking the can down the road so uh i'm happy that we did it and uh you know look forward to the many ways the site's going to continue to evolve uh so yeah a couple other things as we wrap up um there's contribution events throughout the week at drupalcon um you can go to this drupalcontributions.opensocial.site but a specific plug from my perspective um i'm involved in the decoupled menus initiative and there's a mentored contribution event today at uh 2pm and i'm working on a set of web components starting with a menu web component that you know hopefully can be used for a variety of different decoupled drupal projects so if you're interested let's chat we've got a lot of issues that people could work on and we have uh well first of all thanks and thank you as well matt um we have just a few minutes for for questions um and there are some in the uh the q a list so we can just try to go through those um if we run out of time well matt and i will go to the the bounties sponsor booth after this but also you can track us down and hop in or on slack happy to talk about all the stuff so john asked are we using layout builder at all with gatsby and how is that working out we are not uh there is a session later on in drupalcon i forget which day specifically about that problem uh using layoutbuilder with gatsby so check that out um nat asks how would it be possible to use view components instead of react compute components using this graphql setup i think someone actually answered that in the general chat um there's a project gridsum which is essentially like the view parallel to gatsby definitely would have been possible um do you have any opinions on gatsby versus next js um i haven't used next a ton myself but yeah so i mean with my experience with gatsby i think the things that gatsby does really well is the plug-in ecosystem and next doesn't have a lot of that but my understanding is that next has a lot more flexibility as far as you know the combination of of server rendered and dynamic javascript apps and uh you know for things that are a little bit less static can be a really good option uh slides uh have been shared they're shared in the the q a um gareth asked i might be missing the point here but why would you need the decoupling rather than just styling these two pages differently i don't know if matt you want to take a shot at that one uh i mean well i think a lot of it was you know we had this additional react developer that we wanted to be able to work asynchronously with us um i mean i i i think a lot of it just condensed like wanting to to like experiment with this stuff right certainly part of it yeah but also i mean so go to the site and check out these pages it's a little hard to to communicate like in screenshots but there was a lot of animation and interactivity that it really helped us to have the react ecosystem to build i think doing it in drupal while it definitely would have been possible or doing it without decoupling um i feel like it would have taken easily double the time and effort um and then also there is you know the advantage of you know seo advantage performance advantages that we got by statically building those pages with gatsby uh frequency of the gatsby build process right now it's really um on demand it's done with the you know sometimes we schedule releases sometimes it's done with the magic button um we did talk about the concept of having you know an a build that automatically runs at a certain time um we haven't done that yet yeah we haven't really had the use cash for that yet uh but yeah that you know the button's been serving the majority of the needs for our content team yep and then another question expanding on the gatsby providers and how we mapped to paragraphs was there one provider per paragraph bundle yes for the yeah easy one they're multiple they could be multiple react components and that provider would decide between those react components yep uh if you had to rebuild everything again from scratch what would you do differently adopt typescript um that was actually something that came up my personal opinion and this is probably partially driven by lack of experience with typescript but um you know our what we were building at least initially really didn't have a lot of complicated state um and it felt like typescript was going to be overkill uh given that and then also we did have a lot of of people who were new to react and i felt like typescript was going to be a little bit of a barrier in that case uh i don't know if we need to get ushered off the stage at any point but we can try to get these last uh few uh through why use the component module instead of just using a custom block type and a block template you could totally do that uh john um definitely an option um i just think it was it was really nice in the way that everything was kind of nicely packaged together and then it made it really easy to have that sort of block configuration just by sending it in a yaml file and that is you know immediately available as data attributes you'd have to do a little a little bit of plumbing to do the equivalent so you know it's really just kind of a time saver uh was it pretty straightforward to consume paragraphs or did you need to write any adapter code i mean we did have to write you know essentially adapter code the uh you know that that hook to kind of go through the paragraphs like getting the data from the graphql api was actually pretty straightforward but yeah to make it something that was easily repeatable we did have to you know create some some helpers for it what security measures did you use with your json api and gatsby site matt you want to talk to that but for the most part we're just using basic loss uh to secure it you know we didn't need any of this publicly exposed for like real time updates it's all aesthetically built during the build process so we just threw it behind basic auth and then jack asked recently used react components in d8 and it felt weird to not that put not put that in a js behavior um and uh sees that we weren't using behaviors is that really correct um yeah that is also something that i have struggled with so part of the reason that it didn't make sense in my opinion to use behaviors here is because these components were used in two different contexts one of which didn't know about doesn't know about drupal at all really and doesn't know about drupal behaviors um i have in other projects using a more of a kind of progressively decoupled approach where there's just a portion of the page that's mounting react have used drupal behaviors um you know and that was especially in cases where we we wanted to pass data from drupal to the components via javascript so that's the case where it really does make sense um but it does it does feel weird sometimes i agree and i know we went over and sprinted through those questions really fast uh but hopefully it was uh was useful but we'll uh yield the stage uh thanks matt and uh thanks everybody for uh for joining um and yeah it was great to see so many uh familiar faces yeah thank you

Channel: Drupal Association

Views: 264

Rating: 5 out of 5

Keywords: drupalcon, drupal

Id: jcCT47muFi8

Length: 34min 25sec (2065 seconds)

Published: Fri May 28 2021