Using Node.js and React with Drupal CMS at Edutopia

so hello my name is Erica Stennis I'm director of engineering at the George Lucas Educational Foundation a little bit about me my background is in financial services technology but more recently I've been doing things in the public sector including some open government work that led me into some work on things like peer to patent which is an open government project and some help with the White House Office of Science and Technology Policy and from there that led me into some drupal consulting but what I really enjoy most is agile software development which is delivering useful software so I'm going to talk about the work at Edutopia that utopia is the foundation is a non-profit the focus of our foundation is to shine a light on what works in education primarily on k-12 education and we have a small team our website's edutopia.org and our website is a single page application so it's got the front end is node and react and the back end is among other things it's Drupal 8 with JSON API so a little bit about our journey and I'm going to be really talking about case study of things we learned while working on utopia in 2016 we made a decision to try to improve our platform we launched a new rebuild site in January 2018 we had a lot of challenges in doing so but we overcame many of them and went to a few conferences related to this in this you know Drupal con would be an example and we noticed that not a lot of people had examples of the kinds of things we were doing so my goal here is to share some of what we learned and some of the patterns which may be useful to others so some of the challenges we faced include our initial decision to build it what we call decoupled system that has react and node on the front-end and Drupal on the backend and then how to get data out of the API resolving URLs previews media handling and performance so these are some of the challenges we've faced and I'm going to talk about them today so where we started we were in Drupal 7 and we had this choice moving into Drupal 8 which is to go with a path of PHP and more twig and node and react and we didn't have exposure experience with either one of those two we had to choose a path and every path implied doing a lot of learning and so the real question was if we went with going with JavaScript what does that even mean what does it really mean and since this is not a widespread pattern it's actually becoming widespread now but at the time we couldn't see a lot of examples of it doesn't even work so why would we you know for starters why would we use Drupal as a service for us we already had an existing Drupal website we had lots of legacy data we have more than 10,000 articles on a website and we also wanted to go deeper into the front-end universe those are primary reasons for us but there's other good reasons to use Drupal as a service one is you might not want to have a proprietary CMS you might have needs that include typical content management use cases you want to might want to have a baseline that can evolve over a longer period of time and then for us going to the front-end universe we really wanted to tap into the community and the ecosystem there and take advantage of things that are not in Drupal so one question is is using Drupal this way supported yes it is in the Drupal community support has been building up over time they have an initiative in the community called the API first initiative they have integration for JSON API and graph QL that's readily available at this time and you combine that with their solid baseline of content management capabilities and they have a wonderful community there's also on-ramp so an example would be Contenta there's also things related to gatsby so the supports for using Drupal as an API have been building up over the last 24 months so the next issue is how to get data on the back end and at the time we had to choose between Jason API graph QL and this is in 2016 graph QL wasn't quite there at that time it is there now but we chose Jason API in our approach with JSON API JC in the API when you use it with Drupal is verbose there's lots of data that gets returned by it which is all the metadata associated with your content and so the way we use it is we use sparse field sets which is typical JSON API but we also compress the API responses and we proxy them using fastly we're using pm-2 clustering in order to avoid having blocking threads on the API and then we needed something that we call a resolver so what's that so because we have a legacy database of content the URL patterns were not strict and they evolved over time and in front-end like in a react app they are very strict in interpreting URL patterns and routes so we need a solution for that so historically we have a lot of content that falls into the URL URL route slash blog and now our new pattern for the latest content we do is slash article but we needed to handle the old routes we didn't want to change the URLs for thousands of articles and so it's important to think of in the our context the routes our content they're not just a hard baked part of the application so from a friend in perspective the front end when it when the router sees the URL it doesn't know which endpoint to hit it also doesn't know what to return from that endpoint so we needed a solution and so the way we resolve URLs is that we created a custom endpoint that has a parameter which we call URL slug and it determines the proper react component type based on the URL and so it in that resolver it may remap some of the older URL patterns to return the correct component in the front end and so we then wouldn't use JSON API to serialize the object and this allows us to over time have changes to the URL patterns and have the application be completely happy and sort of agnostic to what those patterns are so we take the URL we resolve it through Drupal another big challenge was previews and so in the content editing standpoint editors want to be able to see as they're editing at what the thing will look like and if you have a separate friend from the backend that means the Drupal system has way no way to know what no way to render the front-end properly and also when you're editing you're working on revisions actually not live software and Jason API doesn't directly support revisions so we needed a way to get a specific revision and then configure the front-end to use that revision so the first thing we did this is a typical kind of vanilla Drupal edit form and we implemented a module called workflows that lets you assure that when you are creating a draft the draft gets saved and then in the Edit back-end there's a link here that says previews preview preview of this revision and what that does is that link contains a revision identifier that tells the front-end what to use when the user clicks that they're taken to a preview which is an exact instance of our node react app but it's in a special doc root which is a special installation so the revisions baked into the editor link we have a separate doc root for the preview and we used exactly the same engine to render the preview as we render the front-end and in the Drupal world we're doing something where we override a normalizer and we make a little change to the way Jason API works so that the revision ID is returned so we had to do some customization and that's what the current version of Jason API there's a new version of Jason api for drupal that's coming out that has baked in support for revisions so doing this will be easier and we have a github example if anyone's interested so the next challenge that we had was we needed to have ability to build flexible landing pages so our site is like a media site even though we focused on education on the inside where more media and we need an ability to build landing pages and to allow editors to build complex pages and to be able to deploy those pages with no code no front-end deployment we also wanted the complexity of those pages out of the front-end so these are some examples examples of layouts that we handle are like a list view we have a small grid and a big grid and we have what we call a lead that has a lead article and these are just some examples so the way the experience works for an editor and I'm let me actually show this so this is the edit page for our landing page let me go to see so this is our home page this is the landing page for topics and you'll see things in different layouts here so this is what the backend of that looks like it's really a set of components and each component we call them a query component because the data for the component is pulled dynamically so I can go in and edit a component I can change the heading I can define criteria for how it's for what is pulled and displayed and then I can define pagination if I wish and then there's also a type of component so I can have things in a small grid or a large grid or a list feature and so in this way we have a flexible way to handle the layouts so the next challenge for us was media so we had an existing library of images that are used on our website and we have what we call a master image the master images source image that's used and we had a requirement that we be able to automatically generate crops for that and we do things like set focal points on the image so when it crops the focal point is proper based on the particular crop that's used so what we needed though in the environment where you have a friend that's separate from the back end we need a way to generate the responsive images and then to use those images in our content so the way we're handling that is we're taking advantage of an approach that you can use on AWS called a service image and ler the cloud front CDN we take the master image we push it onto the s3 cloud front CDN redirects to lambda if there's no derivative found which is the image that you need for a particular responsive layout then it generates it on the fly and stores it in s3 and so we use Drupal to manage the responsive image breakpoints if you will but the all the media itself is its separated out in its own micro-service sitting over an s3 so for us I think key question is what's the upside of decoupling and decoupling is you know separating the fern in the backend we wanted a cleaner and better separation of concern so we wanted a higher degree of control over the front-end in the middle tier we wanted to tap into the node and JavaScript community it's wonderful community and powerful we also have noticed that since we separated these two elements the velocity of our changes is much higher in the front-end side of the universe than the backend and that's what we would have expected that's kind of why we went in this direction but it also means that the backend services are much more stable the scope of what's in the Drupal service is much more limited and it serves a very particular function which is a Content service and so that's a smaller footprint which means that we actually spend relatively less time focusing on the back-end experience and developing services and most of our time is spent on the front-end and because of the separation we can do smart things like remember what's happening in the session and be smart about network activity and how we handle network traffic so some examples of that we have spent quite a bit of time doing tuning of vegetto Pia and we started out with before we even started any work or even before we picked our approach we had a performance budget and performance budget was four seconds page load on mobile 3G with a stretch goal to hit sub-second and addy Osmani posted a post two days ago about this same concept if you want to read more about performance budgeting what performance budding is valuable for is when you get a new request that adds to your performance or detracts from your performance it gives you a way to kind of say you know what we have to weigh that request and not automatically give in to every request that you get but some of the things in performance tuning that we're really important to us was or were pay attention to bundle size limit the trips the round trips to the network we use code splitting to load just the code we need we use CSS and J s and part of our journey was we started out with something that's more pure sass than we used aphrodite and we wound up using a motion we mounted up using a motion because it played most nicely with the code splitting and then we also leveraged progressive web apps and so we use a serviceworker we use we follow the lighthouse checklist and so performance has been a key tiebreaker for many of the decisions as we move forward in our development so other things that we have worked on to create perception of speed is that we store the results of API calls in state we do prefetching of API calls under some circumstances we do lazy loading of images and content we do fetching of image when you hover over an image on desktop it'll fetch the content behind it if you hover over a link and then also we spent a lot of time just doing relentless monitoring using web page test and lighthouse dev tools and PageSpeed insights and it's kind of ironic even though we had that four second page load budget even during the project the metrics that they use for page measurement have changed now to first content full paint and so forth so even though we set a goal the target moved even in the middle of the work but we were in a good place so some of the gotchas that we experienced are serviceworkers so we use work box and I noticed about I don't know a week or two they added work box into the create rack react app to version two and our experience has been that work box saves a lot of time and it helps you with building a performant location because it has well-defined patterns for things like caching content and other things we observed is that if you make our app too smart on the client-side then you could have a unexpected negative impact to your SEO and so handling server-side rendering is really important it's really important to make sure that the bits that you want Google to find are found and we also implemented Google amp and in our environment Google amp implies a whole second node application because it's so dissimilar from what we do in our standard application that we needed a whole separate application and we also have been bit by web font performance which is a fairly common thing so but for us one of our biggest fears has been the the iceberg of learning a whole new set of frameworks there's so many things to learn we have a very small team we have to know Express react router reduct web pack and not only that but all the drupal side things not only that but amazon things we also have a new tool chains so coming up to speed on these things has taken a lot of time and it required us to get experienced help on our team as well so we also have been leaning hard into very well-established patterns so this picture gives a example of where we are today so I'm going to go from right to left so at the bottom corner you have Drupal with JSON API and above that you have what we would describe as our media service which is really cloud front with lambda and s3 and then moving towards the middle we have three instances of the front-end application or it includes actually services it's got node react and redux one is what we call production the second is an instance for preview and the third is an instance for amp if you didn't use amp obviously you don't need that third instance and then we have vastly wrapped around the services so when I hit the API I'm actually hitting vastly except the first time around and then we have the browser and the browser is a progressive web app it's got an app shell in it it's got react and redux and we're doing a lot of things with caching and data so so what are the some of the things to expect from a decoupled Drupal you expect to have a workable JSON API service you expect to have solid predictable and fast API performance it needs to fit into the tool chains on the Drupal side the tool change this composer which it does fit into you need some way to handle a preview system and this the support from the community for those things it's pretty solid as we go down here down the list the support isn't quite as good so the preview support as I mentioned that's going to show up in an upcoming release of JSON API to out the resolver is something that you have to work a little harder on to develop and then there's plenty of vendor support for spinning up and hosting Drupal these days but what they do need is better support for onboarding JavaScript developers so I'm just going to give a few more examples so this is our one of our articles is there's actually one of our more complicated articles in terms of the content so when I look at this URL this is an actual URL that's hitting our API so we have a URL slug and this is a slug which defines the content so when I hit this it's going to the backend its first checking fastly if it's nothing and fastly than it goes all the way through and it looks at the URL route it determines based on the route what the type of object in this that should be returned and then it actually returns the appropriate object it's all enough like 30 milliseconds or less so the resolver basically lets us keep the routing system managed in the CMS now we do have routes that are not in the CMS as well so but we find that for the bulk of content that we have that capability to take advantage of some of the CMS features for handling redirects and URL patterns and URL aliases and so forth so looking at the landing page again so an example of a change I can make is I can go in here and I'll just reorder the list and I'm going to edit this one and change the label but Eric let's here say that something's changed well let's try it again let's just try reordering it so I just went in reorder the list on here so this is an example of a piece of content I'm going to go in and put text here Eric was here again I'm going to say that now there's a link that says give me a preview of the system and so that's an example of the loop that editors go in where they could make changes and very instantly see the response the response is all handled in the front-end but they do all the editing in the backend so you would also head here and then lastly the experience is quite snappy and the way we get to that snappy experience is by being smart about how we manage state so when I hit the page the first time it might be a little slower but if I hit them again they're always in state and so moving across the pages don't go to the network it's very fast and snappy and the perception of speed is that it's very very fast even though from the page load standpoint it might be typical or average by having using the power of redux to manage a state we can create the perception or illusion of speed and we also doing lazy loading of images as I scroll it down it lazy loaded this whole section which is related content and there it is so okay so so kind of what's a key message here a key message is that yes you can leverage note and reacts when I describe how we're improving performance I'm really describing how anyone building a note a react app would be tuning that performance it's not special to us we can take advantage of the note and react environment and just use Drupal as a service as a straightforward service JSON API works it's fast you have to fit that into the appropriate architecture like using caching properly but it works and the implementation of JSON API is changing it's a little bit in flux so with each minor release we're watching it closely but it's they're gonna move that into the core of Drupal very soon and then there's graph QL as well so for those who want to use graph you well it's out there available from the perspective of someone using the CMS this is just opening Pandora's box there are so many things that you can do that in the past would be hands-off or hard to get to and so you can take full advantage of whatever the front-end environment is that you want to to have and then Drupal decouple Drupal is a scaled-down Drupal so it means that you know compared to where we were in the past we have many many fewer modules the complexity of it is simpler there's less of a dependency on what one would describe as contributed features and so in this case less is more and less is good so that's pretty much it so thank you everyone [Applause]

Channel: node.js

Views: 1,959

Rating: 4.8620691 out of 5

Keywords:

Id: yl0LGQuXY_Y

Length: 24min 53sec (1493 seconds)

Published: Fri Oct 19 2018