Building Infinitely Scalable Applications with Next.js and Turborepo (Alec Chernicki)

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
Hey there, my name's Alec and by day I'm an architect supporting the teams that build Disney and Hulu, and by night I build monorepo tools. Today we're here to talk about scale. And when we think about scale, we often think about how many users are applications can serve at once. But there's another side to scale, and that's scaling your code base as your business and teams grow. Today we're going to talk about how you can go from one application to an entire ecosystem of applications and packages using Next.js and Turborepo. First, I want to cover two key concepts that we're going to be with, and that's applications and packages. When we think about applications, we might think about the build artifact that we deploy, but in our case, I'm going to be using the word application to refer to the directory that our application lives in rather than what our users are going to see. When we think about packages, We might think, hey, that's how we share code, but our application directory has a package.json and we're never going to publish it to a registry. So I want to challenge the common assumption that packages are just for sharing code and give you another way to think about them. The best packages are really narrow in scope, maybe something like a resizable text area or a utility that does schema validation. The principle that packages should really only handle one or two things is what allows us to compose them together to build full, rich applications. And we already do this every day with React. We're going to apply the exact same composition principles that we used to build React components every day and apply them to packages. In our case, we're going to think of packages as a way to separate groups of logic within our application. Now, with that out of the way, let's walk through the scaling journey step by step. We're going to explore three phases of application scale, the challenges that you might encounter when you reach those points in scale and some examples about how to address them. Let's get started. You may start with something like create-next-app or a Vercel demo application. You'll launch your MVP and start adding more and more features. Eventually your code base is going to grow and you need some way of grouping your logic within your application. You might create folders called UI or components to hold Shad CN components or data to store server actions or fetching functions or even utilities to just store general helper functions. These folders help group or categorize portions of logic within our application that can then be composed together to build pages and layouts. As our application grows files within these folders can kind of become a grab bag. Eventually, it becomes difficult to know which files or functions should be used in what contexts and where the separation of concerns really is. The mental overhead for working with these files is going to increase with every new feature added and every new team member added to the project. So let's take our first step at modularizing our code base to prevent some of this frustration. Let's reuse the exact same naming conventions that we used previously and break it down by the feature or context that the logic is written for. This is going to make it a lot easier to traverse your directory structure and find just the bit of logic that you need in order to compose your next feature. It's also going to add a little bit more separation of concerns to hopefully slow down spaghetti code and code bloat. This can and should be done incrementally and you really should only do this when you start to feel that your original root level folders are getting a little bit messy. Now your application's going to need to grow quite a bit more before you've run into any additional friction, but at a certain point you'll encounter really common growing pains. Your code base might get so big that common tasks like build, test, and lint just take too long. And there might be so many files to navigate within your code base that spaghetti code starts creeping up on you. Now, these problems are super common and they're probably a sign that you're doing really well, but if you don't address them early, it'll cause frustration among your developers and slow down your iteration velocity. Let's solve this with packages. Because we have limited time today, I'm going to show you the finished product, but if you have any questions about how any of this works, please feel free to reach out to me on Twitter or X or whatever it's called at alecchernicki, and I'd be happy to answer any questions you might have. Let's get started. Here, I set up a really minimal monorepo using pnpm. In the packages directory. I have a feature checkout package whose source directory looks exactly like the structure that we set up previously in our application. And that's because it is. I just copy pasted the files from our application into this package. Here you can see two React components that are rendered on the right in our Next.js application. But the crazy thing is that if I update one of these components, you'll see that I get the same exact hot reloading experience as if I built these React components directly into the application. This pattern is called the Internal Package pattern and there's some great documentation for it on the Turborepo website. Internal Packages gives us the ability to create stronger boundaries between bits of application logic while giving us the exact same developer experience of working on one application. This pattern gives us one huge advantage, especially when it comes to scale, and that's the ability to distribute your tasks across your dependency graph. Here's what I mean. When you're one application task time is going to grow linearly with the amount of code that you contribute to that application. But with Turborepo, we can parallelize the tasks across the dependency graph. This means that tasks are no longer limited by the size of your application, but by the size of your largest package. It's important to note that you're not getting all of this for free. You're effectively trading scalability for ongoing maintenance of dependencies and boilerplate configuration files for things like TypeScript and ESLint. It's also important to think of packages as an extension of your application. This means things like keeping versions of dependencies like React and TypeScript in sync with your feature packages and your application. Down the line, you might want to create a new marketing application. You might want to build a new feature that checks if that user's authenticated and redirects them to your product if they are. Using feature packages, you can reuse existing logic that you had already built like authentication logic in an entirely new application in business context. Feature packages not only allows you to create a separation of concerns within your codebase, but also offers you a business agility that's impossible to achieve when you put all of your logic into a single application. There's one last point of scale that I want to cover, and that's when you have multiple teams working across many applications and packages in an entire ecosystem. If you try to put all of your developers into a single monolith, you might start to run into some issues. Teams might want to start to use different versions of dependencies in their packages and applications, and there's totally valid reasons for doing so. Or you might start to encounter really long merge queue times just to merge a PR into main. Or you might start to see that there's completely different product lifecycles for applications in the same monorepo. To combat this, you can become a multi-monorepo species where every team works in a different monorepo designed around the business vertical that they work in. Teams can then build and publish the packages within their monorepos that they feel would be most valuable to teams in the ecosystem. You'll be able to get the same developer experience by adding a dev script for all of your packages. This will watch for file changes on disk, output the final build result, and your Next.js application will reload with the updated package. There are some common questions that may come up if you do choose to adopt the feature package pattern. One of them is: how big should my packages be? Your package can be one file or 100 files. Some packages just take more logic to work as long as they stay narrow in scope and they only handle one or two things. You should be good to go. Folks also ask if they can move from an Internal Package to a built and published package earlier in their scaling journey. If teams look at your codebase and see a little bit of it that they want to be able to reuse in their own applications, that means that the built in published package probably would have a narrow scope defined. In this case, it's totally okay to share it. The danger comes when you try to publish a package that's doing too much and you have to backtrack on the API and refactor it and communicate with external teams and manage dependency versions and it can kind of become a nightmare. But if you feel like you have that narrow scope early on, go for it. The last question that I always get is when is a monorepo too big? The mantra that's always helped me personally is what changes together stays together. If you start to see that there's conflicting interests within the same monorepo, then it might be time to break it up. That's up for you and your team to decide it based on the unique challenges that you're facing in that moment. Speaking of unique challenges, I just wanted to mention that the patterns that I presented today are just one way to break your code base down. I've seen folks take the exact same feature package structure that we used earlier and break it down even further into separate packages for each of the features concerns. There are a million different permutations and ways to break your codebase down. What matters is the way that you organize your codebase works for you. Next.js and Turborepo have added incredible monorepo support over the last couple of years and there's so much more opportunity ahead. I just wanted to thank Vercel for having me and if you have any questions at all, you can reach out to me on Twitter/X at alecchernicki and I'd be happy to compare notes. Thank you so much and have a good one.
Info
Channel: Vercel
Views: 3,429
Rating: undefined out of 5
Keywords:
Id: Bq3kOftoRqE
Channel Id: undefined
Length: 9min 43sec (583 seconds)
Published: Fri Nov 03 2023
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.