Building a SSR Framework from Scratch

today we're going to be talking to kyle about an introduction to veet and building our own like server-side rendering framework similar to next.js he's got a lot of experience with nextgs and he's even done talks at next conf so i'd love to hear his thoughts on my process and everything cool it's happy to be here sweet so if you want to give any like in a little introduction about yourself before we get started that'd be good uh yeah my name's kyle i've been doing like web dev stuff for a little over five years now i went to university of oregon um but never finished my degree one class away so i think it'll probably stay that way for a long time for for a paycheck i've worked at uh leica the university of oregon a startup called autogravity nike acorns and now the startup currently called air and then even if it's not for a paycheck i do a lot of coding for fun on the side i've done a few contributions to a lot of open source projects and i also work with a non-profit called operation code um and if i'm not coding then i'm probably losing at rocket league or watching arsenal lose nice well that being said uh let's get into this post a little bit um have you ever used veet by chance if not i know all about it though very excited for it okay uh you mentioned before we started the podcast swc which i haven't used uh have you played with that at all do you know much about like the differences between that so i'm watching like the non-javascript bundler space very intently but i haven't used any of them um there's a person that used to work for stripe that's also using a programming language called uh zag zagling and he's trying to make a um bundler and he's showing numbers like a hundred times faster than es build even so just the space that people are in i think once everybody acknowledged like it's okay to not use javascript um you know es builds with go i think swc uses rust and then this other person's compiler that's not done is using zags so everybody's just using languages that aren't javascript and and leveraging it in a way that lets people compile transfiled and bundle just way faster and i'm super excited for it all i don't know that's gonna be really crazy if something is that much faster than even es build because switching this major project that i've already used to it is like insane i don't know how you could get much faster yeah i mean but think to yourself like how you thought it could get much faster you know back in the day too i remember when i first started it was like gulp scripts and grunt scripts so things have changed so before we get too far into this post let's talk a little bit about veet itself so not only is it super fast and uses es build under the hood but lately in the in the most recent version they released legacy browser support and the way they're doing it i think is really smart and it should have been the way things have been done for years it will auto generate a legacy bundle for you as you can see down here and it will only load that in those legacy browsers whereas right now even on this really big app i have it work we have all these polyfills for ie11 that are just included and even if you're in modern chrome you download all this extra so i'm excited to see this and hopefully start using it yeah that's really cool i wonder how for example like next also has a bunch of native polyfills that you don't have to add and i wonder how they load them yeah i'm not sure exactly how next does it but i think if they're just using the babel preset emv i want to say it's just all in one bundle yeah we could want to look into it someday sweet um so let's see i guess we can start talking through the actual blog post now so this will just give you a little approach here as like what i was going for so in next.js you can do get server side props and you can load stuff from the server and it gets injected into your component so would you say kyle this is pretty nex-like in this example cool and then i wanted to take it a little step further and instead of having like next has api routes i wanted to just get my data right here in line it's practical for these basic examples but for more advanced things you would need like query parameters and those sorts of things that you get with a regular api route um so to get started some of the things we're going to be talking about is uh server side rendering which we already mentioned a couple times i guess kyle you actually got me into nextgs originally and you were huge about server side rendering so i don't know if you want to give a quick little recap as to what it actually is um yeah i mean the recap would just be that you know before before lands of react or even just a lot of uh javascript usage the fact existed that you need to get data-driven content to a user on the browser and it used to be and still for a lot of people is today um that you get it by making requests to a server and their server returns to html so that would be literally server side rendering because the idea is your server does the work and spits out the document as a downloadable response um and yeah when i talked about it with you i think my favorite thing for for next.js is specifically that you don't get pigeonholed into one or the other you get to choose on a per page basis you know how you want to render things which kind of always was the common problem with like react apps when i worked at companies at some point something would get popular enough and you'd get a seo person or somebody would just know off the top of their head like hey we're not we're not statically pre-rendering you know some of our marketing content that's dinging us on our marketing um and so right then and there right that's that's kind of where one super simple example of where like next and server-side rendering helps you um be you're able to choose these few pages to statically pre-render and not worry about necessarily others that might not provide seo benefit yeah that's true uh so let's get into the actual code base here so to get started making this we're going to go through a step by step on a commit basis and if you want to learn the full process for how i went through this you can actually read the blog post so to get started i had index.html which is really common in really any react app but with v specifically you want to have a little html tag here to replace it with your app html that gets server rendered and then you need to have like your entry point that gets loaded on the client and then down below you can see all the packages used for this project i tried to use as few as possible so there's prisma there's the react refresh plugin which handles hot reloading as you edit stuff which is really awesome and i've found it much better than webpack's different tool set that is there for react other than that using react using express to actually do the server stuff and then it has ts node just to make file changes quick and they get brought in quickly and then v itself and typescript so not too much here and the last file being the veetconfig that tells it you want to use react refresh and i inject the jsx import for react in here as well uh any any questions so far kyle or is this all pretty simple yeah no questions at all cool so let's go back to the commits here and let's go to the server side rendering step so before we go over it i'm going to check it out so i'm going to come over here into my terminal checking it out starting it so that we can actually run it as we talk about it so for right now i'm making an entry point into the app it gets this page prop and that page prop is just an object with a component and whatever props you might pass to it so if we go back to the post that would be in this case i'm passing a name from git server side props that will get passed into the component when it renders in the code you just saw so then down here we have this page loader which i don't want to get too in depth into but veet has a few things that it exposes to you to do server side rendering such as like transforming the index.html and you'll see these comments were brought from the veet docs and essentially it transforms your html it will inject the react refresh stuff so you don't have to worry about it as you're editing components to just live update on the page and you just had to call this one method so it's really powerful what is done for you with just a few lines of code and then down here we're loading our entry point and we're loading our actual page to call the git server side profs method and inject that into our component so down here this is actually the function that gets called every time you load a page it calls our page loader which was just above that we just talked about and then it renders that react component to a string and then it comes down here and replaces that app.html comment from our last commit with the app's html and it also injects the props that we mentioned before from the git server side props into the page for client-side mounting that's pretty much everything we needed to do eviet also has this cool ssr stack trace thing which will let you know where you have problems in your code without you having to like deduce it and figure it out from the render 2 string call which i found really helpful other than that this is the next part which i would love to talk to kyle about a little bit or he can make more clear for everyone this just takes like a url and then gets a file path so if i go to slash test which let's go right here right now i load it up slash test it then converts it to like hey in the pages folder it's slash test.tsx and kyle can speak to this a little bit about how in nextgs there's like a lot more powerful functionality than that in this little example right now you you just have a regular path name and it goes to a component but yeah kyle i don't know if you want to talk about some of the other things you can do with the url and mapping those to files and next yeah i mean framework specific i'm sure you could get the same feature set um you know you can with next a file name may have square brackets around it which might make it um an optional square bracket with triple dots in front of the variable name in in the file name so if you might go to the code for me um that you were just at yeah exactly so that would be what's called an optional catch-all route it basically will catch so in this example you have it at pages so whatever route you don't have covered so in this example you have index and test covered every other request to your website will go through this optional catch-all route it gets a little more um complicated or nuanced for example if you do an optional catch-all within a folder because then for example the optional catch-all helps you get the index as well as other requests to that page and then another param another possibility might be instead of doing triple dots you can just have a variable within square brackets which would basically be a parameterized uh url um and that's kind of going back to where optional catch all helps out is because you can have a parameterized url and also an optional for the case like for example with blog right you might have slash blog which is the listing of all your blogs so you can choose to express that as pages blog.tsx or pages blog as a folder that has an index file and a parameterized file or you can just do the blog folder with an optional catch-all and that will be the file for all of your blog requests and you can split from there and all these things are possible to make but i just like pointing out as we go and kyle feel free to jump in like just how much work it is to really make like a full framework this is like just to get your like ideas flowing and just to see oh wow all of this stuff is like happening when you don't even have to think about it when you use something else so and i just want to make a statement i realized i made a mistake i've been calling it optional those are just catch-alls optional catch-all is when you have two pairs of square brackets around the variable and then yeah it's not just down in the parameterized routing but something that's interesting in your code that's actually like an optional aspect of the config and next some people like trailing slashes so you'd have to plan for that too yeah that too and yeah super basic right now so uh at this step like we have server side rendering so i can load the test page and i can load the home page and it works it's just the same as if you just had static html files uh you'll notice that right here we have a console log like hello and if i inspect the page and i try to click there's nothing happening because all we're doing is server side rendering so everything you saw right here was just to get it to where you load a path it grabs that file and its server renders it and that's it there's no client-side interaction yet so now with that let's look at the next commit which is a very small one i was just adding and then we can just check it out here to see it all i did was add a good server-side props that injects a message and so if we run that again come back over here it says hello from server before it was just right here now you can actually see this is from the server which isn't that intuitive yet because we're not doing anything special here that requires server side logic so that with that let's keep going this is the next big part which is trying to do client-side routing and this could be a little confusing especially as we go through this so the first thing is and actually kyle i'm i'm curious if you know how this works exactly next yes because i i don't i haven't looked into it uh in my example i have this routes array and this would be like auto generated by the framework so anytime you add a new file into the pages folder it would like automatically inject into this file another route component like another route object in here with the path and a dynamic import statement i know that yeah actually i guess i'm just going to leave that off to you do you know how next does this like actually no i haven't looked into it cool maybe something we can look at so i i wrote this like three different times three different ways before i wrote this post uh one of the things that i didn't want to do in the beginning i just server rendered like slash test and returned it and then like server rendered the home page and returned it which is great it's just loading that one component and nothing else but the problem is if you have like two different pages and you want to client-side route between them which is really powerful right like we all know that if you just like click a link to the about page and it's instant you don't have a full server load like that's really nice but how do you do that without including all that extra stuff in your code maybe you have like a whole bunch of orders an about page a blog all of these things you don't want them all you don't want all of those components in the bundle on every page you just want what's actually needed for that one page so the way i came up with this was having this little routes array so if you click on the home link and you're on the test page it can call the dynamic import statement so it will load this code at the point in time when someone clicks it and then it actually updates the page to display that component yeah it might be something similar um and the reason i guess that is because of the fact that they do pre-fetching and i you know there's tons of reasons why you'd want to do pre-fetching besides a dynamic import but the fact that it's default behavior implies to me that you know it's done in a similar way that we're like there's a dynamic import that occurs on the request um when i talk about prefetching i talk about there's both a manual function called prefetch that you can pass a path from their use router hook or you can just use the link component and you'll automatically prefetch the href of a link um when it slides into the viewport both of those i believe dynamically import the pages sorry i didn't mean to cut you off but speaking of that it made me uh want to jump ahead a little bit so i made a link component and we'll get into this more in a minute but it calls this navigate method that the library exposes it'd be very easy to make this custom hook expose a prefetch method so instead of navigating maybe when you hover so you could add like an on mouse over to this a tag and you could prefetch it similar to what kyle's talking yeah absolutely okay so back to where we're at so i have this routes thing and you can just imagine the moment and this doesn't happen in the framework yet either this is just another thing you would have to do if you wanted to build something full like a full-on framework so let's say the moment i add like order it's hard for me to type over this mic order.tsx you could imagine if i had the server running which i do the moment that that happens this array gets automatically updated it sees that a new page is in the pages folder and it adds it here just these two lines and so then what i did next was i made a context in react and this context just stores the active page so if you're on slash test it's currently slash test and the component there so it's this object and then a function to set the active page and that will be used when you client-side link between stuff so then let's look at our actual hook down here it's called use move it and it just returns a navigate method and it might look a little complicated if you've never seen this before but what it's doing is quite simple it's doing a promise.all for two things in order to navigate to some page we need its data and we need the component so we loop over the routes which are right above here and we find one where the path matches whatever path they're trying to go to so if we're trying to go to slash test it finds the one right here and grabs its component and then it also calls git server data which is this little function i made right here it hits an endpoint on our express server and it just gives it like hey here's the path to the component call get server-side props from that file and return us the json so once those two things are done then it sets the active page so this is yet another spot where i'd love to hear kyle's thoughts because this could be done a lot of different ways even just those few things i mentioned maybe get the server data takes too long or maybe you want to immediately load the page before the server data but then you would have a loading spinner there's just so many different things to really decide on right right here like what do you think kyle you're muted yeah so i asked what you're thinking about this type of stuff any comments yeah so that's why it really depends on that's why that's why what makes it so great is being able to decide on a per page basis because what i would say is like you want to do it this way if you have a get static props method because you're telling your framework i need you to go get data from the server in order for you to render this page um there's other things that you can do in next which you could build into to move it you know there's the implicit um statically prerendering of pages that don't have any data fetching methods so you could integrate into your framework the idea that if there's no static if there's no data fetching methods like get server side props from the framework it's going to try to pre-build it anyways for you automatically um so i think in all cases you know you would call get get server-side props if that method existed um and then at the very least you're still going to be calling that uh v transform method uh regardless of what the page file is cool so to continue now um then we set the active page and then we call history.pushdate which is just the built-in browser api and this is another little gotcha some browsers do things differently with the history api react router the team behind it they actually made their own little history abstraction which anyone could use just to smooth out some of those inconsistencies so that's yet another thing that you kind of have to be aware of and one thing we haven't even talked about yet is error handling there's no error handling here at all like what happens if for some reason that page can't be loaded or the server data response fails and has like an error we aren't really handling those and we don't offer any way for them to handle them so many things to think about now when we go back into our entry before it was just rendering create element with whatever page now we just have a context around it because we want to let the user possibly change the page from anywhere in the react tree so context is the best way you can call navigate from anywhere and it will always work so that's the point of that and then we added our hydrate method so this makes it so we can actually click stuff on the front end and i'll check this commit out in a second and show you but when the page loads on the client side it checks okay in that routes array where's the active route and then it waits for that component to be loaded and once that component is loaded dynamically then we can hydrate react again to keep the bundle really simple this bundle just has this little bit of code and a reference to all the pages every time the page loads it checks okay what component do we need to load loads it with another network request and then hydrates react then there's just a few typescript types here and then updating the page to like have a little a link so let's let's pull this commit down and take a look and so this is actually what i expected i didn't actually have the link here yet so the next commit is the actual one we should go over and check out so the link component is in this commit and all it does is pull in our context method called navigate and it prevents the default when you do an anchor tag click and what's cool about this is that if you don't have javascript enabled you still see the html you click it it does a full server page load and it works just just fine obviously your js interactions won't work but links and everything still work which is another key point to why people are so big on server-side rendering it's just less opportunity for things to be broken and then down here this is the other thing we reference that data endpoint that actually gets our get server-side props whenever they navigate between pages this is all it is it uses the same method we use earlier to get the page and we're just sending back the props also we just have to replace data in front of the url since we're passing data slash whatever path we just want to remove data and call the page loader like normal so then down here i registered that data route and then over here use the link so let's pull down this commit and take a look hopefully this one works all right so now we have hello from server and if i click it hello actually gets logged now we're hydrating everything and then let's take a look if we clicked go to test it works i could refresh it it works i click go home it works and it pulled in the stuff from the server and then what happens if i click the back button so it went back because the browser told it to but we didn't actually write the code for this this was another thing i left out of the article uh so i'm curious kyle do you know how we would implement this i just learned because i've never really worked with the history apis like an hour before this call and i'm going to try to live code a little bit that is needed i mean i'm i'm guessing there's i mean if it it still would all be client-side navigation so you'd want to use one of the history methods i i don't know if my i'm only aware of the abstraction that react router provides so you could use like go negative one or back yeah so part of the problem with this is that like it's a method that i hadn't used before because we have to actually listen for the browser click and so what that is it's actually really easy you have to use on pop state and so let's let's take a look here the first thing we need to do is we need to pull out this navigate method because we need it to be called when the browser back button is clicked so i'm going to move this over here and i'm going to make another function here i'm going to export it and just like that and then i'm also going to tell it that set active page is now passed into it so set active page all it does is it just takes a page and it needs to return void and so that should all work just the same except this needs to have the two okay that's actually really cool so i've been using github copilot and i haven't gotten to use it too much yet and i'm pretty happy with what it just suggested that was cool uh kyle i'm curious have you used github copilot much yet i have access but we've been given the thumbs down at work so i haven't quite given it a good go we're allowed to use it on the one our one open source repo but i haven't been in there lately so okay uh so all i did was abstract this out and just made it take these two things so what that lets us do is we come over here and i'm gonna do this uh i guess the correct way i want to put a use effect and then inside of the use effect i'm going to say window dot on pop state and whenever it pops i'm going to call navigate that was close to the window location path name and let me import navigate should pop up and then i also have to give it set active page and i only noticed that thanks to typescript and so then this doesn't need to change we just want this effect to run when the page first loads and all we're doing is registering this callback and so this is another thing it might not be correct to do this you might need to like add an event listener because this might overwrite something else on the page that's using this i don't know why anything else would be but for the purpose of this tutorial it works fine so let's go back and see if it works hopefully it does because it'd be very embarrassing if it didn't hit the back button and it went back so exactly what i expected to happen so just do that again i go to the test page hit the browse or back button and let's clear the network tab so we can see what happens here go home and it's calling the data for the home route before it goes to it and it's getting hello from server it already has the the home component so it's not making another request for that and then it sets it active let's see now with that being with that being said we're getting close to being done um the next part that i added was prisma and so uh kyle have you used prisma at all by chance no but also very aware of it and like asura for example would love to get a chance one day cool yeah it's it's really awesome use it on a few different projects now uh and i just thought it was a really great addition to this type of tutorial it was a lot of stuff we've covered but prisma is really simple to throw in there and so let's let's kind of go over that commit so whenever you add prisma and you run prisma init it throws in an emv file with a database url obviously i'm committing my emv file just for the purpose of this tutorial and it's just using dummy credentials for docker but you never want to commit emv files otherwise then all i had to do was import the prismaclient and my page loader initialize prisma and then pass that in to get server side props so now just by doing that little bit of code check back out stuff oh okay never mind it switched come back over to the index charisma is now available on all of my git server side props on every page that i would make and then next let's see so then the next thing to talk about is prisma has the schema file which lets you define your database stuff and now recently it works with so if you want to use something nosql you don't have to use postgres or mysql anymore with prisma so that's something cool to mention but for the purpose of this tutorial i added a message model which stores a bunch of messages and i just store an id that auto increments and some text then i also added a little seeder just to throw in welcome from prisma to the homepage updated my homepage to actually call prisma so let's take a look at when we run npm start again and see it say hello from prisma and then we'll do a little bit more with prisma before we finish so it says welcome from prisma and now another really useful thing that you can do with prisma is sometimes you are testing stuff and some data is like set a certain way and you don't know why or you want to change a row but you don't want to like actually add that api route because maybe you haven't gotten to it yet prisma has this really awesome gui tool called prisma studio and i found it to be really useful because in the past i'd use something like sql pro it opened up on the wrong page i've used something like sql pro there's like a bunch of different ones like db for all these different things but some of them don't work with the database you're using so maybe you're using postgres you need one client another one you need a different client prisma kind of takes care of it all for you with this nice little web-based editor for simple things it's really nice you can't do complicated stuff in here that i know of really crazy queries but it gets the job done so if i want to add a new record here like let's say welcome from podcast and i save it we can go here but i'm not running the server anymore load it back up and it still says welcome from prisma so that's because by default when we say find first right here it's finding it from the beginning so prisma's type safe client is super awesome so you'll see we get all this auto completion help so order by we want to order by the id and we want to say where it's descending so if we save that come back over here now it says welcome from podcast so we're loading our latest messages from the database and going to the studio we're able to just inspect all of our messages really quickly so it's pretty powerful i love all of the stuff prism is doing every release there's something cool just to point out again one of my favorite parts about it is the type safety so if i want to say like if i want to order by name descending well there is no name and this might seem obvious right now but maybe you're changing your data layer and you're adding a new field or removing one you can immediately know in all of your code like oh that little spot right there no longer works and it won't even compile so it's super helpful uh so kyle any any questions thoughts about what i've gone over no questions it's yeah i mean it's just it's a great you know intro into what it's like to begin thinking about like an ssr framework as you mentioned like there's just so much ground to cover i mean yeah even on that on that last bit um you mentioned when you were just as an example of something else that you know we could do to add to the feature set you mentioned that for example the home component was already dynamically imported so it didn't need to load again once you navigated back and forward in the history state in the same vein you know if you set up like automatic static regeneration uh static pre-rendering you know that could be a static document that doesn't actually rely on data fetching so the whole document um is cache for when you go back it doesn't have to re-download that's a good point yet another awesome thing and same thing with all the the stale wall revalidate stuff that's becoming so popular and everything that would be good to add cool well but that being said i want to thank you for being on here and going over it with me appreciate your time no problem great work

Channel: frontendblog

Views: 332

Keywords:

Id: rVBakvM6QX8

Length: 31min 16sec (1876 seconds)

Published: Mon Aug 02 2021