Understanding an OPEN source codebase LIVE

hi everyone welcome to a new video today we are going through an open source project pretty well funded open source project trying to understand the code base trying to understand the general structure that's followed specifically in mern stack open source code bases and the motivation behind this is that we all open open source projects and then we're basically very overwhelmed by looking at the code base so I just wanted to share the general structure of a mern stack code base and if this video does well then we can do it for other sort of code bases and Tech Stacks as well I have one request I feel this video will be slightly slow and the reason is I want to keep it extremely beginner friendly so if you feel this was really slow let me know in the comments just basically reply with this was good this was faster this was slow and then we'll tweak the pace based on that let's get right into the video the project we're discussing today is cal.com a few of you have been able to set it up a few of you have started to contribute to it as well um but a bunch of you are basically overwhelmed looking at it um I was going through the code Base today so my previous contribution I didn't really go through the code base much I was able to set it up but I didn't really have to dive deep into it today I actually dived deeper into the code base and the structure is very similar to a lot of other Mern stack applications um and I've seen almost every good mern stack project it's divided into some some very specific modules and sub modules which is what they have done as well so let's start by understanding what these module sub modules look like for most mern-stack open source projects what each and every module does and then we will dive deeper into a few modules for cal.com and I'll show you the code and what they've done now they've implemented things then you can go from there and figure out issues cool, let's get right into it so most mern stack projects if this is your root folder they're basically broken down into two very big folders where you will spend most of your time the first one is called the apps folder and the second one is called the packages folder so, if you open the project if you look at cal.com you will see a bunch of folders at the root level but the important ones or like the ones there where you will spend most of your time are these two let's understand what these have inside them packages is a bunch of small packages that either use each other so one package could use another package or more specifically your final apps use packages let's understand a few examples of what this is one very common one is called UI this contains all of your react components the basic react component is what your input box looks like what a button looks like and the standard styling that you've used in your project using any sort of standard styling Library like MUi or Tailwind so effectively if you look at the package.json here you will either see Tailwind or you'll see mui or some other styling framework and you'll see a bunch of components so if you're a front-end developer if you're a pure front-end developer this is where you're spending most of your time writing components and fixing them this is like almost nil State here they're like very nicely decoupled components that you've written and the end team uses it either here in apps or someplace else but you as the front-end developer if you're like a pure designer turned front-end developer this is where you'll spend most of your time the second one is called either the store or it's called recoil sometimes basically has a bunch of your hooks and mutations to manage the state of your application now what is the state of your application if you look at UI it basically has a bunch of components which means a button an input box but these could be like highly variable in applications for example for example if I go to my LinkedIn you see a bunch of things all around you see a bunch of posts by people you see the notification number here is 88 so anything that's specific to me or my view of LinkedIn is the state which here is 88 a number 16 another number a bunch of messages and my feed but you and I both see my network the same icon the same home button only the notification number is different so these numbers these posts all these things are called what's called like a state and you usually store all the logic to create State variables and fetch them in something called store slash recoil if you've heard of libraries like Redux or Zustand stand they basically are Frameworks that let you manage State and recoil is another one that's very popular these days and that's what's used in cal.com and generally in other sort of popular code bases that's the one we're learning in my course as well because most open source code bases I've seen recoil is the one that's used most popularly cool what does it do it helps you manage in store state moving on to the third one this could be uh your config so it contains variables Global variables like the api's URLs that you might have to hit a bunch of your own front-end URLs uh Secrets things like this Now the secrets are actually stored at the root level in maybe a root level .env but uh them being exposed as variables is done here so you don't really spend too much time here you define it once and then whenever you need a new config variable you are right here a bunch of auxiliary tsconfig or Eslint folders which contain your Global linting standards what is linting how your code looks how many tab spaces should it have and like automatically making your code cleaner by running a command all the configs for it and the other one PS config is basically every sort of package here should have an independent uh sort of TS config a lot of times the react TS config looks very different from a TS config that's pure backend and hence you put all of them in a single folder again something you won't touch very often then there's one folder that's fairly common it's called common and this contains a bunch of either types and this contains uh so common contains a bunch of your types anything that's globally needed by basically all of these guys and a few things inside your apps is stored inside common and we'll quickly see what that has what that means in cal.com then there's something called types which is basically your Global types in typescript a lot of sort of shape of your data and like props that are being sent all around is stored in types and there's finally DB or sometimes it's called Prisma depending on what sort of orm you're using but this basically contains a bunch of high level functions that get you interact with your database so that you're not writing SQL queries directly in your application you have a high level function and you just call it directly inside your apps or inside your UI cool that is more or less a decent chunk of packages now what are these These are independent packages some packages might use in other packages for example most cases UI uses types and UI uses common because uh common contains a bunch of your things that are used by other modules so it's like basically bound to be used by almost everyone and types your UI might need because these types sort of might be needed here in the UI and in your final app and hence they need to be put inside a separate module so the whole goal of keeping things here like segregating them into modules or packages is that you have a very nice sort of abstraction of small independent modules that each and every a big your final app will be using and a bunch of times packages use each other cool that's one side packages what's the other side apps, this is your final user facing application basically uh the final website of cal.com will probably be in a folder called either web or sometimes it's called next if next is a front-end framework that you're using so effectively next is never really defined here or never really used here UI purely has react unless you're doing navigation here which I think is like not the best thing in the world if you want to do navigation any react or any next or like front-end framework specific thing should probably be done here at the top and a bunch of other popular things that happens inside apps are um you have your docs the final docs that you probably have to publish somewhere in case you're building something that needs user-facing docs one other very popular end user application is Storybook which is it basically contains a bunch of your uis that the world you can see use It's Your Design system for the user the world to see and a lot of times you also open source your components so all these things that are present here all these things that developers write or like front-end developers more specifically right can bunch of times be used by people all around the world so one very basic example might be stripe if stripe has a design system a lot of times you need like their button to appear across your website and so open sourcing Your Design system or like specifically the components is good and where does storybook help you storybook helps you visualize these see these just the components without any app specific logic what does that mean your website will basically contain a bunch of app specific logic and I'll show this so it'll be a little more clearer but you have to navigate through a bunch of pages to reach let's say a button that you want to see but in storybook you can simply navigate to the button and then see it cool let's see these in action I think I've talked a lot here so what is step two after you've done everything and you understand these basic constructs you need to open the code base so let's look at the code base uh maybe before I do that a bunch of other folders that you don't really need to know about and you don't won't really spend too much time on if you're a pure front-end developer are devops or sometimes called Scripts or Ops which basically contains a bunch of your scripts that actually deploy the thing on the internet sometimes contains your infrastructure as code files basically basically describing your final deployment as to what your instances look like describing your infrastructure as code or specifically like scripts that let you deploy your code from GitHub to the main repo also if you're on GitHub most probably there'll be a DOT GitHub folder that contains all of your workflows for testing linting and maybe even deploying cool that's the general structure of a mern stack code base at the high level there are apps there are packages there's a DOT EnV file there's a devops folder and then dot GitHub inside you have a bunch of modules here these are the common ones there are a bunch of small modules actually very application specific we'll see this very soon in cal.com and then your final apps are usually storybook the final website if you're building a mobile app then the mobile app could also be here if it's in react native and your docs which are like public docs you have for people cool cool let's go into packages and see if whatever I said is actually true what all do we see here we see app store CLI which seems like a command line interface that SEO I'm assuming create your own app or like bootstraps an app for you I could totally be wrong I'm not diving into this one I'll dive into a different one app store is probably it contains the front-end components for a bunch of like very specific I think when you're building a dynamic component basically it sounds like they allow other people to create apps I mean I know that for example discord, Zoom can integrate directly with cal.com and when you are trying to do this you probably want to allow them to bring their front ends inside cal.com which might be what this does I don't know I haven't dived deeper into it atoms sounds like a state although I could be wrong here and it seems like this has Tailwind so it seems like it might be a bunch of front-end components there's like a single component here no idea what that is moving on config probably contains your config variables your theme what Global colors exist this still seems fairly understandable moving on uh core this sounds like common which contains bunch of your things that will be used all around um event.ts for example contains a bunch of functions that can be that will probably use for uh events and why is this good or why is this cool because this is in your central core folder which can be used by all the other packages here and also by the main thing uh your apps cool and just exposes day.js now you might ask why have they put this inside a folder if all this does is expose another npm Library the answer might be in the future they want to move to a different library and they want to keep the overall high level API of this thing same okay let's make this slightly simpler if you look at index.js it exports something called dayjs which is a global variable they might in the future want to move away from dayjs to a different library and when they do that they can simply just make their changes here basically replace this day.js by I don't know dayjs2 or some other library and that way their high level API that's being used by everyone else Remains the Same they internally are able to change the library even though that won't probably happen here because their folder itself is named dayjs if this was named date time or like something more generic then this is a decent way for you to expose external apis which it might feel like oh why are they creating a new module here they could have simply just used dayjs all across the reason to do it is just to create another abstraction across the external Library so if you want to move from One external library to another you can simply change the high level functions you don't have to change the high level functions that your library is exposing the internal functions can be changed and so that whoever is using it you don't have to make different function calls anymore hope that was clear moving on uh emails so this is the one that I will teach today so why because it's like very I sort of need this at work so I just was very curious as to how they've implemented emails and this also introduces some new react Concepts that might be interesting or intriguing to a few people cool cal.com sends a lot of emails emails don't look ugly they don't look like simple plain text whenever you're sending these emails you sort of basically the way to make them look pretty is that you're actually sending HTML and CSS and you're not really sending text which is why a lot of times when you see promotional emails they look really pretty it's basically HTML CSS JavaScript under the hood but again we all know it's very hard to write HTML CSS and JavaScript which is why react was introduced so it makes it easy to write components so the problem is uh that was react was basically built for websites uh it's sort of a framework that's sort of powerful to create websites and uh you know be used in a place where you need single page applications for example if I'm on LinkedIn without refreshing the page I can navigate from one place to another cool but you don't really need this in an email email simply needs basic HTML CSS & JavaScript but we as developers are very comfortable writing react so this Library lets you write uh email components using react but let's use server side render them now what is server side rendering it's basically a way for you to um write code in react but at least for the first render not send complete the complete react bundle but simply send the thing rendered into HTML CSS & JavaScript what does it mean it basically means when you are on a normal website and you ask a server that's running some react code it basically returns you a very big JavaScript bundle that does a lot of things client-side so on this client you figure out what to put where and that's how your for example LinkedIn page gets slowly rendered but when you're on an email screen you don't really need a bunch of navigation you don't really need buttons to be clicked to change something directly here on the email which is why you simply ask the react server for HTML CSS and Js for this final looking page so what this thing returns you is a very nice HTML tag inside which you get body inside which you get a bunch of styles for the buttons you don't get you also do get this big JavaScript chunk that you're getting here when you're not server side rendering but the initial load is really fast and initially you are getting the HTML that you're getting is what straight up gets rendered versus uh no it's almost no HTML coming in JavaScript coming in figuring out client side what to render this was too confusing read a little about server side rendering and why it's used and why it's very specifically important in this use case and there are a bunch of ways to do it um one the one that they've used basically is using something called react Dom so if you look at it um emails and this is the one that I'm diving deeper into in packages um so like we're going to take 10 minutes here and understand what this does it exposes a high level function called render email so let's look at that let's see what it does Source render email what does it do it simply gets a component from the templates folder so dot slash templates basically contains the templates for each and every email these are all the emails that you could send for example the attendee location change email if someone changes their location uh I'm assuming this is the location of the meeting uh you send them an email um disabled app email feedback email so these are like the emails that you could send in if you look at them these are react components that sort of render the final email but as I said you need to render them server side and simply send HTML CSS and JavaScript to the end user how do they do it if you look at the high level render email function it uses a library called react Dom server which basically does the same thing I explained which is South Side rendering it renders the thing server side and sends HTML CSS & JavaScript which is why if you look at the high level render email function this is the only thing that emails this module exports so this is something important now modules are supposed to these packages that you see are supposed to expose something that other places can use and this one specifically just exports one function called render email where does it export it if you look at index.ts this is where it's exported it also exports a bunch of other things but the important thing it exports the thing that it exports as the default is render email and since the templates here are cal.com specific this can sort of only be used inside cal.com right now but in the future like this is a very nice they've built sort of a module which internally might seem like something really small most of the logic is basically here of server side rendering something and sending it back but it's it can be very useful and Powerful for people like me for example I am trying to now build something similar If This Were slightly more generic if it would allow me to write my own components here I could simply have used this versus uh like writing this from scratch so this is the gist of what emails does and if you want to test it they have written something here if you go to the very bottom it says you can use an API endpoint to preview the email HTML that's there's already one here feel free to change it so next we'll try to understand what that means before that let me quickly take you through the other packages um there's Prisma this is like the orm that they use which is lets you do a bunch of DB calls with our without writing SQL queries trpc is a technology you should read about it's also something they use it gives you very nice Types on both the front end and the back end lib from what I saw contains it's sort of like common contains a bunch of your things that are used by a lot of other libraries I could be wrong and then a bunch of other things UI is the other important one it contains your actual components so the thing that I was telling you initially UI contains if you're a front-end developer this is where you'll spend most of your time and you'll write the button logic or like right the link icon button or like if there's a new button that's introduced by the design team this is where you introduce it and you sort of call it a night then the main team the people who are sort of full stack developers are like more than front-end slightly more then frontend add it to to where to apps so if you look at the apps folder uh there are four things here which I've explained a storybook is the one that will show you components let me just take you through that really quickly if we go to cal.com so I've cloned the repo locally I already ran yarn install so all the dependencies have been installed so what if what do you have to do if you want to do the same thing locally first clone the repo then run your install once you've done it go to apps slash storybook and then run yarn Dev this will actually show you what Storybook like what's its power and like why it's used in the industry why it's like one of the many end user apps that you create for your end users to well in this case just visualize your components without going too deep inside the application versus if they have a very nice storybook component I can just simply go here and look for the sign up page for example they have a button space which has every variation of button that exists for cal.com both in like light mode and dark mode so this gives you like the theme for your for cal.com for example in this case and also you actually simply just go to the stories for example if I go to button.stories.mdx it will basically show me what code was written to render such a button what code was written to render such a button so if I'm ever using the design system to let's say build an application that looks similar to cal.com and since they've open source their components I can do it and if I want to know okay how do I get this icon exactly I can simply go to buttons Dot stories.mdx and find that specific story that's rendering this specific button so you can see all your components without actually having to navigate the cal.com main application you can also see the code for it if you just search for .stories.tsx so if i go to alert.stories.tsx for example .mdx then I can see all the alert stories so basically whatever you see here is effectively code that's written in some .stories.mdx file most probably the front-end developer who's writing components will also write the stories so that the other person who's the full stack developer or like the person who's actually maintaining state of the application can simply look at what they've done and all the variations of the card and can simply integrate it so their life becomes very easy and if you're the front and pure front-end developer the designer you can simply create the component and if you're a really good one you should also just create the stories which is as simple as defining a alert.stories.mdx or your component name.stories.mdx right next to your component so this is your alert.tsx component you're a front-end developer you wrote this you can simply right next to it Define another file called alert Dot stories.mdx and it will automatically appear here whatever sort of stories you have written and the other full stack developer can simply use your code so it's like their life is very easy cool that's stories and the other one is the main application that I wanted to show which I guess you guys have already seen I don't really need to show it but if you run yarn dx in the root folder or if you you'll actually have to run yarn dx here because this brings up the database the other things you need you can't simply turn on front end and hope everything will work so if you're going to CD apps slash uh whatever web and running yarn Dev and hoping everything would work it unfortunately won't everything works if you run yarn dx and the root folder it starts the database it creates a bunch of dummy users for you it starts the front end starts the backend everything cool and I also have to Monkey patch a few things here I don't know if there's something up with my machine or uh their code is broken but there's like the seed.TS file was giving me a bunch of errors and this is the file that actually seeds the data basically puts in users and like a bunch of events uh and it's like it's giving me some errors and I just got around it by adding a bunch of tricad statements so if ever something was breaking I would just return right there because I don't really need the data right now I just wanted to show you guys how to how to start cal.com and why I wanted to show you that was because I wanted to show you how email templates can be tested how you can see an email template right now the way to do that is if the app is running which seems like it is if you might have remembered if we go here I think this is still starting so we'll wait for it until then if you remember emails packages email here if you look at it at the very bottom they said there's already one here so let's go to apps web pages API and email.ts most probably this file uses the email package so let's go to apps web API what was it pages API and email.TS if I go here this is the sort of important thing to spend some time on this is what server side rendering looks like if you look at the Handler here it's a basic HTTP Handler so this is not even frontend this is a pure backend call that comes and when it comes you look at the very bottom you set a bunch of headers you sent the status code for the request I don't know why this is for the request you should probably be for the response the important thing is here the rest.write render email the function that we exposed where in the cal.com/emails package can be used here and I simply give you the component I want rendered and the props that I want to send it send to it why is this cool this is cool because now we have someone making a backend call but you're returning them server site rendered HTML which is like sort of the thing that you need to do when you're sending emails cool, refresh it still doesn't work which is not the best thing in the world because I wanted to show you guys what this looks like in action basically I wanted to go to /email/ um sorry /API/email this would have ideally things working oh it does for you okay cool so /API/email shows me this verification email and why is this cool because when I'm sending someone an email I can simply give them an iframe to this URL and then this is what they will see in the email this is how you see pretty emails in your mail whenever someone like sends you promotional emails from I don't know make my trip or whatever how do I know this you can quickly test this in HTML and CSS in JavaScript if I write a very basic HTML file which is this URL which in the end will be like something deployed on the internet you see something like this let me remove hello world and then maybe give it some height and run and suddenly broke again anyways but whatever you see here is what you see here whenever this comes back up you'll see server side rendered HTML so what did we learn from all of this um we learned if there's ever a bug in emails where do you have to go you go to cal.com you go to packages if someone has created an issue that says add a new email template you go here and this is where you work and create a template what are these things small things here these are all packages where are these used either packages use each other but more importantly packages are used by the high level apps and the most important app thecal.com whatever you see here is stored here inside the web this web not wrong is a next JS application which again uses a bunch of the modules the core components are defined in UI not here so this guy uses that and whenever you're sort of working with most uh sort of repositories even if you go to backpack for example you will see there are packages and there apps which in our case may be is called something else ah everything is in packages for us yes we don't have apps or like backpack doesn't have apps but like app extension is the same equivalent of web here ideally you should have apps right ideally backpack should also have apps as a separate folder but it's fine so similar to what's web here is app extension here and app extension uses a bunch of these if you look at this there's like DB there is eslint there is react common there is recoil which is like the store there is themes so again bunch of small modules which are in the end what power the app extension I mean again in cal.com what powers the final web application is all of these small packages why to decouple it because it's easier to develop easy to write code you get you guys get that cool that's all for the video was a long one let me know let me know if this was fast this was slow and then we can increase the pace dive deeper make an actual contribution or if you like this high level okay I understand the code base now I can take it from here let me know that as well I'll see you guys in the next one bye bye

Channel: Harkirat Singh

Views: 61,792

Keywords:

Id: _X7_v0h_q3A

Length: 29min 39sec (1779 seconds)

Published: Sun Jul 23 2023