Complete Full Stack Developer Roadmap! (2023 Edition)

hi everyone and welcome to the full stack roadmap this is going to be a long video but a very important one if you're thinking of becoming an open source contributor if you're thinking of cracking Google summary of code this year or generally if you're thinking of contributing to open source companies or getting into remote jobs or even if you're thinking of cracking a few startups India or abroad full stack development is an extremely underrated skill it is a skill that's used all across the industry doesn't matter if you're doing a basic back-end job a front-end job if you're working even in machine learning AI or web 3 there are certain components of your job which require building a front-end or a back end or a basic system that people can use to interact with your more important bigger system for example in the web 3 world if you have a smart contract out there most probably there will be a front end that will interact with it and most probably you might also have a centralized backend where you're storing your user data what this means is full stack development is a skill that can be used all across and is usually the first step when you're trying to become a software developer it also gives you an idea around how websites work how mobile apps work and how in the real world applications are built as always in my roadmaps this video will have an assignment in the end and the people who will be able to do the assignment will be able to join the internal Discord group that we have which is a group of extremely competent and self-dependent software developers who do assignments every time I give them where we Talk Amongst ourselves and if people have questions they ask each other it's a very elite group of people and whenever I come across any remote opportunity I tend to refer people from that closed group because I know these are people who are very obedient confident and also smart at doing things and figuring things out on their own this video will be broken down into various sections the whole full stack development series would be a series of three videos this is the first one in which we'll be tackling backend development I'll quickly share why the second one will be tackling front-end development and the third one we'll be doing some Advanced frontend and backend so that we start from the basics and we move to an advanced concept after the third video you should basically be able to Target as many open source companies as you want because most companies have stacks similar to what we'll be discussing in this video the first part of the video as usual will be peaking interest in this video in backend development what back-end development is what a back-end developer does on a day-to-day basis what are the systems that you build what is the jargon that's used in the industry that might confuse you and what you have to do on a day-to-day basis when your responsibility is that of a backend developer in the second part of the video we'll actually be coding stuff making the back end of a website and deploying it somewhere so that you guys can interact with it in the end I'll be open sourcing a repo which will have some portion of what we'll build today and a bunch of tests you can Fork that repo complete the assignment make sure all the tests pass and then feel free to create a merge request I will not be able to merge all of them but I will merge one at the very end and that might give you your first open source contribution more importantly though after this video you should be able to at least understand a bunch of Open Source repositories and a bunch of Open Source programs that are out there and slowly and steadily I think by the end of the third video you should have enough knowledge or at least know what to do when you are trying to tackle an open source code base after the third video I will be picking up open source projects and contributing to them one by one I wanted to start contributing to open source projects right away but I realized there's maybe a skill set that's missing from people and people need to understand full stack development before they can start contributing completely so this series a three-part video is going to help you achieve that and once you do that we'll be contributing to actual open source projects out there full stack development is all i did for the first three to four years of my job my first job being a Goldman Sachs engineer I worked there and had 30 LPA package my second job being freelancing on upwork and I made 30 lakhs on upwork in around three months here's the video if you haven't checked that out and then I got my first 100K offer which was also pure front-end and back-end development which means full stack after my third job I went a little bit into deep Tech and a little bit into web 3 and that's what I do now but the point here is with full stack development you have enough skills to crack at least 100K offer which is a good starting point for you from India and from there you can decide whether you want to stick to full stack development or build some more advanced things get into more Niche fields and decide what feed you want to go to but the first step is always full stack development and that's what we'll be tackling in this series with all of that context let's get right into the video and understand the jargon that's used in the industry when doing back-end development so we're at the board now and I'll try to explain five Concepts that are very important or jargon that's used in the industry that might overwhelm you if this is the first time you're trying back in development I'll try to explain it in very simple fashion and then in the next part of the video we'll actually be coding it so the very first jargon and you guys might have heard about this if you're doing computer networks back in college is called communication protocols before I get into it can someone tell me in the comments what communication Protocols are five four three two one okay I hope you got it but communication Protocols are basically a way and I'm going to teach this from the very top because this deserves a course on its own on computer networks um but the knowledge that you need for full stack development I'll try to stick to that so communication protocols or protocols in general the word basically means this is a protocol that needs to be followed for doing some task for example um when you're in a military I'm assuming when you're in the Army there are certain protocols uh when someone let's say attacks a building or whenever there is a some sort of an attack somewhere the military has to follow a certain protocol that's where the word comes from and more specifically communication protocols usually mean a way or a protocol for systems to follow when they want to talk to each other so back in 1970s 1960s people had these computers but they couldn't really talk to each other eventually people realized if connect them with wires they can actually talk to each other and if you think about it deeply full stack development or more specifically back-end what we're discussing in this video is nothing but computers talking to each other I'll explain this better a little bit more but before we get there systems need to talk to each other for anything to happen for example when you're on uh what's a famous website today Instagram .com when you go to this website and let's say you have you're at the login page when you put your username and password here and press the submit button a request goes out to an Instagram backend server hence your machine is effectively communicating with an Instagram machine somewhere out in the world these wires are there's a very big web of wires all across you might see a wire come out of your modem this wire goes to another modem and then from there a bunch of routers are connected how through which you basically talk to machines these machines have something called as addresses but we won't get into that just yet what you need to know is machines need to talk to each other and there needs to be a specific defined way around how they should talk to each other else this machine will send try to send something to this machine but this machine won't understand it so a protocol effectively is a way for machines to talk to each other it's a defined set of rules that machines need to follow when they're talking to each other now there are like more Concepts here for example their application Level protocols and then there are more lower level protocols I think that's the right trajectory but we will be only discussing two types in this video let me clean this up the two types of protocols we're discussing are called transport layer protocols and application lab protocols there are a bunch of other protocols in between but we'll only be discussing these two because these are sort of important the two famous transport layer Protocols are TCP and UDP and application layer includes a lot of protocols important one for us being HTTP which stands for hypertext transfer protocol not that it matters these are two protocols that machines use to talk to each other this you should Google and understand what this is from the very high level TCP was built so that when a machine tries to talk to another machine if there is ever a network discrepancy here the machine will retry and keep sending the messages until it is able to create a connection but UDP will just throw messages or packets at the other machine and if the other machine does not accept it there's basically no acknowledgment needed from the other server to tell the original server that I've received this message so UDP is usually used in things like live streaming video when you're receiving video you do when you're doing a zoom call even if you miss a few packets from the other side it doesn't matter there's no point for you to re-ask for those packets because if you re-ask for packet one by the time it comes back that time has gone it's really back in time and you really don't want those frames that get missed so when you're talking to someone on a video call or if you are watching the cricket match let's say usually a UDP based protocol is used so that even if packets get missed you don't worry about them you just skip them there's a small black frame which you don't worry about and the video keeps going from there but in more important things for example the Instagram example that we just discussed um whether you have an input box and you click the submit button this request needs to go via TCP because if there is a network disconnection you want to retry this request um this is an important request only if this request reaches the server and responds with something will you log in so this needs to be based on TCP these are lower level protocols and usually when you're writing a full stack app or a backend app you don't have to worry about them unless you are building a video streaming application application layer protocols as the name says is what an application developer which is you when you're building a backend develop application needs to worry about so this is basically the most important thing here you need to understand what HTTP is you actually don't even need to understand it you just need to know this is a communication protocol that browsers use to interact with servers uh there are other protocols that are also used but for the scope of this video only the first video we'll be focusing on HTTP what will happen in the end is we will be building an HTTP server very similar to Instagram and we'll try to send some requests from our browser to this HTTP server so you'll be able to see that request actually go out you'll be able to see that this is an HTTP request and will be able to see on the server or when we're writing the code you'll realize that what we're writing is an HTTP server we won't be writing an HTTP server from scratch whenever you are using a backend language most of the times an HTTP server is a very popular use case so there are Frameworks or libraries out there that let you create HTTP servers in a few lines of code we will be using one such library but if you want this is a great assignment this is the first assignment that I did when I joined a lab back in college I wrote an HTTP server from scratch in C++, take it as an assignment although if you're a complete beginner I I would suggest you first complete this video and then try to create an HTTP server in the language of your choice with that let's move to the second thing after communication protocols the second important thing is databases now what is a database I think most of you might have heard of things like MYSQL postgres or mongodb maybe something like Firebase as well these are all what are called databases they're usually represented like this in application diagrams and what they are are basically if you remember we have the front end which sends a request to the backend now the back end is like a process running on a server and these servers are sometimes small sometimes big these servers can go down and hence you don't want to store data here what is data if you're accessing instagram.com then it's your user data your username password everything is sort of stored somewhere which usually is a database and there are popular data databases that basically make it easier for you to query and write data to a database these are the ones that are usually used what we'll be using in this video would be postgres and the purpose of this thing this database is to store data there are ways to scale these very independent to how you scale an HTTP server but effectively this is your browser this is an HTTP server what we discussed in the last part and this is a database all three things are very important to build a full stack application these are usually enough to build a simple application which is what we'll be building in this video but eventually when you're building bigger systems you need some more you need some more Jazz which I'll discuss in this video but we want code we will be coding this in the third part of the series but I still discuss it so up until this point we understand browsers we understand HTTP servers and we understand databases again we'll be coding this very soon but before we do that let me teach a few more things so that you have a basic overview of how an application like Instagram would work the fourth thing that I wanted to discuss here are messaging buses or um what are called Pub Subs so up until this point to build a very basic application for example an application that has basic authentication you can hit an HTTP server which will talk to a database verify that the username and password is Right basically what it'll do is it will ask the database maybe for the user that has this username and their password and then it will match the password that you've sent to the password that it got from the database and if that's true it will return your response that says hey everything looks good else it will return your response that things are bad please retry but um more complicated applications for example in Instagram once you have logged in maybe you need to send an alert to the email or the phone number of this user that says hey you logged in from New Delhi India are you sure this is you if not then please click here so this is a common use case when you're trying to log into a system from a new location and to build this usually these things are not enough the three things we have discussed until this point they can be used but a better thing to use here are asynchronous communication systems like message buses and Pub Subs so let's try to understand what they are um here you send a request to the HTTP server which logged you in now the HTTP server needs to send a request to basically your email and your phone number maybe and these are sort of um how do I say this these are processes that you worry about but you don't worry about them immediately you might have noticed when you're doing a Google pay transaction you see the transaction feedback immediately on the app but you get a message from your bank in like five to six seconds the reason for that is these are not important events like they're important but they're not immediately important logging the user in is immediately important that's the feedback they need to see here on the browser but the email can reach them in like five to ten seconds so you offload these tasks that are sort of important need to reach the end user but maybe in a while to usually other backend processes in your system and these backend processes are also machines that you have somewhere on the internet and now these machines need to talk not to the browser but to your backend service let me clean this up a little so your front end sent a request to your backend and your backend now basically needs to send another request to your email server which has the responsibility of emailing the other user and your phone number server which has a responsibility of text messaging them why do you not do the same things here because this is a backend server that's needed for immediate requests um it needs to handle a bunch of other requests if you you could do that here but that's what's called a monolith architecture and as the name suggests it feels bad or sounds weird what we're doing here is a micro Services based architecture which means there are different services that do different tasks and specifically tasks like sending an email and sending a phone number which are very asynchronous tasks can happen in their own time usually makes sense to be separate processes or separate services so This pack-end Server no needs to tell the email server to send an email and the phone number server to send to call another phone number problem is if you make these HTTP requests just like this one is you could make them HTTP requests but HTTP requests require a response right then and there the backend server will basically send this request and wait for the response from the email server and sending an email is a tough task it's a longer task you have to maybe hit another external service which will send the email and then wait for acknowledgment and same for phone number so effectively you're blocking the backend server for these tasks to complete which is bad as I said earlier these are not tasks that need to happen immediately so what you can do is rather than directly talking to the HTTP server and then another downside here is if your email server is ever down the backend server will try to send requests and it's down this will get missed and this message will never need reach the user because your email service was down when your backend try to talk to it this is where very nice internal communication systems like messaging buses are introduced as the name suggests a messaging bus is like a messaging queue where you have a server which contains what looks like a queue and your backend server can simply put a message in this queue and the email server and phone number server can keep picking up messages from this queue why is this better from a direct HTTP call as I mentioned earlier if this is done or this is down the HTTP request will fail and it's gone forever but if you put this thing inside the messaging queue even the email server is down right now it will eventually come back up and when it does it will pick these messages and send out the email as I said email doesn't need to happen immediately it can happen in like a minute two minutes just to let the user know that someone has logged in so even if your email server is down for whatever reason whenever it comes back up it can tick the message from here now you might ask what if the messaging server is down if the messaging server is down then you're pretty much * you have to make sure that this is either reliable or when this goes down the backend server basically either invalidates this request or stores in the database that hey this message got missed and then maybe sends it in the future but usually the idea is that these things Auto scale and these things are transitory can go up and down can even go down forever and then come back in an hour but the messaging bus needs to be reliable if it's not reliable then yeah you're pretty much * and you'll miss out events there are other ways to optimize this further but this is a good enough system for you to understand how asynchronous tasks happen at the back end this is one asynchronous task there are a bunch of other things that could happen uh when systems try to communicate with each other and usually on the back end when you're sending an HTTP request a lot of things happen things like logging things like authentication a lot of times your backend server is not doing authentication you have an auth server sitting in front of the backend server um and you sometimes even have a load balancer so effectively there are a lot of backend services that are running even though in this video we'll only be getting a single HTTP server eventually in the third video we'll create these asynchronous systems that will be able to talk to each other in the back end so up until this point we understand HTTP servers communication protocols nhttp request that goes to a server via TCP the server maybe does a query to the database and sends a message to a messaging queue which then forwards it to other services which do their own thing if you have any questions up until this point let me know in the comments and I'll try to respond to it this is a very nice basic architecture for a backend for the back end of an application and from here let's actually start coding this to create a very simple system let me describe what we're building now the longer term assignment that you are building in this video is that we're building leetcode if you guys haven't seen it leetcode is effectively a platform where you can go and solve algorithmic problems the interface looks very similar to any other application you can log in sign up once you sign up you see a bunch of problems once you click on a problem you see the problem statement on the left you see a text area on the right where you can solve the problem and you see a submit button when you click the submit button a request goes to a backend the backend process is whether or not your request is right and then lets you know whether your submission was accepted or rejected this was an application that me and a bunch of friends built back in college it was called code Village it was very similar to a code Chef or code forces where you could conduct contests we won't be conducting contests we'll build a simple website which will show you a bunch of problems and when you click on a problem it will take you to the problems page you can see the problem here the code here and a bunch of buttons here once you click on these buttons it will let you know whether or not your submission was accepted or rejected but this is the first part of the series the series in the first part we're just building the back end and hence what we have to do now is create an HTTP server the thing that we just discussed which will do a few things which in the next video we will connect to the front end so that the front end can send requests to the backend so let's try to understand what like a simple leetcode application looks like this is the landing page you have a login button here and a sign up button here so two functionalities that you need to build in your system are 1. login, 2.sign up uh there may be something else like reset password but let's not do that let's leave that as an assignment so backend needs to do these things and it also needs to when the user is creating it themselves it needs to basically give them roles either a user or an admin why an admin because later on the admin will have a authority to create problems here so another endpoint would be an endpoint to create a new problem this endpoint should basically check and make sure only admins are allowed to create a new problem and finally we will create sort of a dummy endpoint that will accept the input from a user when they click on submit so basically uh submit problem I said this will be a dummy endpoint for this video because when you click on submit this code that you've written goes out to a backend service which basically has to run your code there which is sort of a complicated process and an asynchronous process which we will discuss in the third video if you want to do it yourself I would highly recommend use something like either Redis Que or RabbitMQ to do this yourself you will also have to dockerize stuff I'm leaving it I'm saying a few things and leaving them as an assignment if you want to do it great if not we will get to it in the third video but in this video we'll be doing one two three four four things uh login sign up create a new problem and submit a new problem and I'll be leaving a few assignments for you to try out as well again we're only building the back end not building too much of the front end in this video in the next video we'll build actual front ends this video will build like a basic front-end that lets you do these things with HTML CSS and JavaScript the text stack we're using in this video is the very famous MERN stack why it's very popular used all across and the language we are using is JavaScript for now but eventually we will be moving to type script but for now we're using JavaScript to build a full stack application similar to leetcode in three videos the first video being you're building this backend and a very basic front end if you can do this yourself or if you want to do this yourself chat GPT can help you out the only questions you need to ask chatGPT are how can I create a basic login signup system using the Express framework in node.js and here the second question you need to ask is how can I create a post HTTP request to accept a new problem what are the arguments that you'll need there you will need just the problem statement for now you will need test cases as well but let's ignore that for now just so that we don't worry about the application Level jargon so the creator of the problem needs to submit a title for the problem and the problem statement maybe a test case which is something you see here in leetcode where you have the inputs here the input should look like this and the output should look like this a test case but this is an array this is a string and this is also a string and finally submit a problem is a dummy endpoint which should accept the ID for the problem that the user is trying to submit and the code that they've written here cool hopefully you can do it yourself if not we'll be tackling this in the second part of the video so let's get right into it and let's start coding so before we get into coding I need to share some more things some specifics around the HTTP protocol that I just told you about as I said we in the end need to make an HTTP server that you see here on the right and our browser is what will interact with the HTTP server if you actually look at any website for example Facebook Instagram or even chat GPT these are all running on some HTTP server and returning you some data from an HTTP server just to show you that example if I right click here and click on inspect this chrome developer tool thingy opens up if I go to network this will show me all the network requests that are going out from my browser to chatGPT as I navigate through the application so chat GPT also has an HTTP server and my browser here which is this figure right here is sending it some requests and just to understand the structure of that request let's refresh this page and see what's going out so if I refresh this page as you can see a bunch of these requests went out now some of these are return simple HTML for example the the first request that goes to exactly this URL chat.openei.com/chat actually returns you some HTML and your browser is smart enough to know this is HTML and it renders the page over here but all the other requests that are going out some of these are like script tags things we'll ignore for now but the important ones here are this is a request that goes to chat.openi.com/api/session and most probably it returns me my details for this auth session please ignore the access token here and this will give you access to my ChatGPT account which is bad but as you can see it's showing me the details my email my photo things like that and there are a bunch of other requests for example conversations this is giving me all my existing conversations so I have talked to chatGPT once before this as you can see that conversation is right here the title of that conversation is no history snooping and you can see it right here so effectively the first request goes out and gives you all the HTML to render this page and then there are a bunch of subsequent requests that give you things like hey what was this person's chat history the reason all of them don't come at once is to decrease the response time so that the user gets something if I refresh this page I see something as some initial feedback and then eventually this thing comes up so there are multiple requests that are slowly going out now let's try to understand the structure of such an HTTP request so when you're sending some requests you will send one the URL that you're trying to access basically the URL of the HTTP server there are more complicated things but just for simplification right now you have to identify and give out the URL of the HTTP server when you're going to facebook.com or instagram.com you're basically doing the same facebook.com is the URL here chat.openeai.com chat is the URL and for the other request that I showed you that's getting my conversation the URL is chat.openai.com backend.api slash conversation question mark offset equal to 0 and limit equal to 20. so this whole thing is the URL you change the url these things at the back are called routes so slash back-end Dash API slash conversations is around so chat dot openai.com is the you URL this thing you see here is the route and then offset and limit are something called as query parameters so let's go back to this diagram you have to specify the URL I would actually break this into two parts you have to specify the URL and the route and you have to specify you can specify a bunch of other things the second thing is the type of the request so there are few types of requests important ones for now are get put and post get request you send when you want some data from the server post request you send when you want to send some data to the server and put is when you want to send data but not exactly add a new thing but update a new thing so practically when I want to get these conversations I will send out a get request so if you look at this conversation request as you can see it says get here but if I want to actually add something new there then I will send a post request so if I add a message here I can you tell me something and I press enter the request that will go out will be a post request if there is a request there could be cases where requests don't go but here as you can see this is a post request that went out and if you look at the payload it is showing you the message that I sent which is this thing hi can you tell me something and this that you see here is the payload so there is a URL there's the route there is a type of the request get put and post and then it there are these things on the right which is the payload that you're sending meaning data you want to tell the server hey do something the route will tell the server okay this person wants to do something here but you need to pass in some data as well and data can be classified into various things the first one is headers headers are basically a way for you to tell the server some auxiliary information for example your cookies are usually there your browser information is there the site that you're sending requests from which in this case is this is is there so bunch of auxiliary information which isn't the actual data that the server needs is sent here as you can see there is a request this is the response headers but there is a request headers section that is sending my data to chat.openai.com and these are the request parameters that are going the authorization header is an important one here I am not showing the whole thing but this long string is what I sent to the chat.openai server and they know this is me this is like a cookie if you have heard of it or like some sort of a password so so that so every request that I'm sending out I send this cookie along with it in this case it's not a cookie it's an authorization header but this authorization header is what tells the chat GPT servers this is harkirat and these are like stored somewhere in your browser the first time you log into the browser so when you log into chat.openia.com if I open this in incognito window you see this and when you log in this you send a request to the server a post request to login sending your email and password that I put here if I put my email and password and click on continue I send a post request and the server returns an authorization header which I store in my browser and in any every subsequent requests from this page after I've logged in I send that authorization header we will be implementing this very soon but just for you to understand this authorization header is what very important amongst all the things that you see here cool so these were headers then there is something called query params query params are effectively query parameters which over here is question offset equal to zero and limit equal to zero so this offset equal to zero and limit equal to 0 is a way for me to tell something to the server usually when you're sending out get requests this is the way for you to pass data to the back end there is no easier way to pass the data because if I open this in a new tab here as you can see it says authorization token is missing but uh when I'm sending out this request a response is coming back and I'm sending in that request hey limit is 20 which means I need 20 records at Max from my history here and offset is 0 which means I need the first 20 we'll explain this later on or go to chat GP and understand pagination but that's what's happening here in this request and these things I am sending there are query parameters the next thing is body the body is another way for you to send data so you could either send it via query param services or you could use the body for example if I send new message here hi there I press enter as you can see here the post request that I send has some payload in there the input being hi there or like I have a typo there but you get the idea this data that I'm sending is the body and usually when you're sending post requests you send data like this inside the body this might feel weird to some people this is like a key value pair or a JavaScript object or what's called a Json and you're sending some data here so this is similar to we send data here as well but this was a get request so we sent them in the query parameter and this was a post request so we sent them in the body cool there is also one more thing called cookies which we will discuss later for now this is used for authentication but for now we will use the authorization header for authentication so we should be good to go and we need not worry about cookies just at the moment now you send the request the sending structure looks like this and we will be implementing this very soon on the HTTP server the server needs to respond back and the server also gets to send back some data the important data it sends is response headers so as you saw earlier it can also respond with some headers and these are usually used for sending things like cookies and that's probably the most important use case other than that you don't usually worry about the response headers it can also send your body which could either be text Json or HTML yeah so if I go to chat.openai.com so here what I'm getting back is Json and here the original request that goes out so there are a bunch of requests but the first request that goes out returns me so if I go here you can see when I refresh the first request that goes out returns me HTML so the server could return you various data shapes and usually these are the common ones one more important thing the server returns use status codes these are slightly complicated but the idea is this is a way for your server to let the client know the operation that happened on the server so you might have heard 404 very commonly 404 usually means this resource isn't present on the server every status code means something 200 and 201 means things are good and you got back good response but if something goes wrong on the server for example if you don't have authorization to use a page your backend server should ideally return 403 so that whoever is sending the request knows that 403 so if I search for what does 403 status code stand for it will help me it means forbidden response which means you are not entitled to this resource that you are trying to access when I am trying to send a request here from my browser without the authorization header as you can see the status code I got back was 401 401 and 403 usually both mean that you don't have access to this resource there are a bunch of status codes all mean a lot of things we do not have to worry about them yet but the important ones you need to remember are either tools 100 or 401, 403 or 404 these are the important ones your server should return When you either have a good response to return or a bad response and that's more or less it this bubble basically contains the request life cycle of when you send out an HTTP request and what comes back now let's actually start writing an HTTP server you can write an HTTP server in any language of your choice we are using the MERN stack and hence we will be writing it in node.js some JavaScript prerequisites are good to know for example you should know how to write for Loops while Loops General basic stuff but other than that we will be using libraries to create the HTTP server so the HTTP protocol is like a very famous protocol that you can write on your own but usually in every language they create some very nice libraries that make it very easy for you to use those libraries to create HTTP servers because the use case is very common right you understand an http server needs to accept a route a URL the type of the request and a bunch of these things and you know it needs to respond back with some things so all languages have created abstractions there are very famous libraries out there the one we are working with today is node.js the language and I mean JavaScript the language node just the runtime and express the framework the in in MERN E stands for Express and express is a library that lets you easily create HTTP servers if I go to this page you should probably be able to see a very simple boilerplate code let's see the hello world program it looks something like this as you can see we will code this from scratch but very simply this is the way to create a get route so app.get means we are creating a request Handler a get request Handler on the slash route and we're sending back simple text hello world so let's now start coding and write a simple HTTP server and go from there cool so the first thing you need to do is install node.js there are multiple ways of doing this I am going to ask chatGPT you should to based on your operating system might be slightly difficult on Windows but fairly simple on Mac and a Linux based machine so let me search how to install node.js on Mac I already have node.js so I don't have to do this but I'm still doing this so that I can show you guys you should also understand terminals What terminals are the Mac has a standard terminal if you search for terminal you'll see one here you can install more complicated ones I usually use item these days this is a terminal you can basically run the same commands here if you're on Windows you will have the command prompt that will let you do these things you should ideally move to WSL on Windows and even more preferably use some version of Linux it will make your life much easier if you're starting off seriously contributing or like seriously starting development Linux based machines or even Mac works by fine so I would strongly suggest you do that so here as you can see chat GPT tells us hey you can use the official installer and once the installation is complete open the terminal and verify the installation by typing these two things so this is important for me it's simply Brew install node and that should do it so I'm not going to do it but if I press enter here it should install node I already have it so if I search for node space Dash V it's giving me the node version I have which is 18.1.0 and if I search for npm version that I have it says this what is npm so usually whenever you have a library you have a runtime and a package manager as I told you right now we'll be using Express and express is effectively a package that's deployed on the internet and anyone can use it package managers make your life easy to get these dependencies from the internet and then host them on your repository so if your Repo moves from one server to the other you have like a package file which contains all of your packages and you can install it so package managers are very important and most back-end languages have them for node specifically the package manager is called npm and if I search for npm space dash dash help it'll give me all the commands that I can run here okay we are now getting right into writing our first HTTP server using Express in node.js so if someone tells you create an HTTP server for me in Express or create the back second for a MERN stack application these are the steps you need to follow what is the application that we're building we're building leetcode the back end of leetcode but more specifically in this video we're building the basic authentication system so someone can log in into their profile by sending username and password even sign up first sign up send a username and password once you've signed up you can log in once you log in you get back a token from the server that you can use to re-initiate logins to the server cool so let's start by creating a new directory mkdir Express backend app if I go here the next thing I have to do is call npm init what this will do is create a file if I keep pressing enter here by the end of it I will see a new file here called package.json this is a very important file usually the first input file of your node project it provides node.js the runtime a bunch of information about your project so the key things here are the name of your project the version you are on description the main file that should run in this project a bunch of scripts and you can actually change this let me edit this and add a few more things here let's add start node index.js I just want to show that you can actually change this file it doesn't need to be the same and you can like feel free to edit it and the author and the license so this is a very basic file we will add more things to it but as you might remember the file has this main file written here so let's also create a main file you can do that by calling Vi main.js to open it so feel free to use an editor of your choice I will use webstorm you can use vs code which is very famous and a lot of beginner people use that I am using a webstorm because I'm more comfortable with it so as you can see I have two files package.json and index.js that I just created now let's just copy the boilerplate code here the getting started hello world this was the code let's copy it here paste it here and try to analyze what's happening we at the top are importing this Express project what is expressed as I said it is an HTTP server that a lot of smart people have written that you can reuse and just call a bunch of functions on to create an HTTP server so we import that at the very top we create an instance of that so this is a very Express specific detail will be different for different HTTP servers in Express you create an app object by calling the express function you have to specify a port now one thing I did not tell you here I said that the URL basically looks something like chat.openai.com and then you give a route you can slash should also give a port when apps are usually deployed on the internet they are deployed on specific ports which is why you do not see them here but a lot of times you might have to go to chat.openai.com eight nine nine nine nine zero zero zero this port is also an identifier you can think of it as a part of the URL for now but it is an important detail and when your especially working locally and creating a backend server locally you need to specify a port where your application will run and very soon I'll show you what this is actually used they have also defined a single route here a route being as I said the part of the URL so the route is slash which means if you send us request to slash which if the URL was chat.openai.com it would be something like this the route is slash which means there's basically no route then send hello world the text so if I go to chat.openai.com and the chat.openeai.com looks like an HTTP server that looks like this it will return me back hello world and this line is again and a detail for Express where you have to call app.listen which will actually start your HTTP server up until this point if I just run this much code nothing will happen the HTTP server won't start I've just given it a route again I can give it like multiple routes but for now I just have one and if I write app.listen it will actually start the HTTP server and when it starts it will log this I feel at this point this tutorial isn't very beginner friendly let me change a few things here to make it slightly beginner friendly but you will need to understand JavaScript before you dive into Express is what I feel so if you don't know JavaScript up until this point do a bunch of basic tutorials so you understand JS and then come back to an implementation detail I've said this in the past it's very hard to do two very hard things at the same time first tackle one so if you don't know JavaScript yet first tackle that and then come back here you might be slightly if you've worked with other languages this might feel very weird why are we sending a function as the second argument so that is again a JavaScript specific detail do that first I'm not saying that here but first do that and then come back to the video if you already understand this great let's try to run this file so the file is called index.js and I can just run node index.js when I run it I get an error and the error it says cannot find module Express so I have imported Express at the top but I never really added it to my project as I told you our project can actually contain a bunch of these dependencies and our project needs to know what those are the way to add a dependency like this Express is the first one we'll be adding more the way to add it is running the command npm install and the name of the dependency which here is Express so if I run npm install express it should install Express for me in this project and it should most probably change package.json as well so if you look at package.json now it has a dependencies section and there is Express right here this is a way for you to store the structure of your application or all the dependencies in a single place package.json being that file it also created this package Dash and lock.json file so you won't worry about worry about right now but package.json now has one dependency we will be adding more but I think this should be enough for us to start our project so if I now run node index.js and I go to localhost colon 3000 I see Hello World so okay so basically in 11 lines of code you can write an HTTP server now a lot of things are missing here implementation is pretty much nothing we have not deployed this on the internet and this URL might feel weird to you why are we going to this localhost colon 3000 ish I want to go to helloworldapp.com so when you're running apps locally this is how you develop you usually run apps on localhost you don't always run it on some internet facing domain name like chat.openei.com or facebook.com eventually it's very easy to deploy this on the internet and give it a URL we are not worrying about that right now so for the purpose of this tutorial and generally when you are doing local development your app will be hosted here if I change this port to 3001 and I restart the process by typing node index.js this will no longer return anything and if I go to 3001 it will the route is slash so if you go to the slash route or don't give anything it's very basically the same thing so localhost Chrome 3001 slash gives you this and if I change the route let's call it Route 1 node index.js now 3001 gives you nothing if I go to slash Route 1 it gives me hello world let's add another route copy this code paste it here let's call it Route 2 and let's return hello world from route 2 here let's restart the app and Route 1 returns you hello world and Route 2 returns you hello world from Route 2. so as you can see I can add more and more routes we still have to add structure to it but this is enough to create an HTTP server if I return HTML from here you will actually see a page rendered you want to try that let's try that uh what does chat what is chat uh gpt's route it's the slash chat route so let's do app.get slash chat rest dot send uh HTML I worry this might not work but let's see there you go it did work so I sent some HTML now for people who don't know HTML you should you need to do a tutorial of HTML before you understand this but basically HTML is usually written in HTML files and this might feel weird why are we writing HTML in a Javascript file but bear with me for a while what I did here was I can actually change the color of this as well style equal to color red and let me restart and there you go now the slash chat route is actually returning HTML so your HTTP route can actually return back HTML as you will just see here the localhost called 3001 chat returns bunch of HTML and slash Route 1 if you might remember returns simple text the other important thing to usually send in applications is Json which is basically an object that looks like this let's send this and if I go to Route 2 now as you can see it's a black page because no longer is HTML or text coming back what's coming back is this Json if I go to preview I can actually see and this is usually how data is structured from the back end when you in leetcode ask the backend server for hey give me all my submissions it will most probably return you a Json object like this on chat.openia.com as I showed you there is a request that goes out to get all of these existing conversations and if you look at how the data comes back from there it is a Json object it might feel complicated there are arrays inside the Json but the Json can be a very complicated object can be can have nested children it can have arrays so on and so forth so now we are able to send back data the only thing that's missing here is sending headers from the client we are not yet sending headers we are also not yet sending query params and the backend sending back headers or status codes that's important let's quickly do that so let's say there's a route in the back end of your application let's actually change this to some friendly routes sign up login and uh questions so leetcode has a bunch of questions this endpoint should return those another one should be submissions I have created four stubbed routes here that right now basically do nothing the first one is the route that you should hit when someone tries to sign up the second one is the route it should hit when someone tries to log in third one is the route that you should hit when someone tries to get questions from the back end and in leetcode there are a bunch of questions if you open the lead code first page you'll see a bunch of questions so it should return an array of questions and submissions should taken as an input the question that you're asking the submissions for so if you've made five submissions to a problem let's say two states is the problem your request should send from the client should send out the ID of that problem and your server should return back an array of submissions your existing submissions and maybe those status codes whether they were accepted or not, cool so I am leaving this here I think this has become a very long video already so what we'll do now I will leave a few things here as an assignment the first thing you have to do is add a an object store here what is an object store so as I told you initially there are databases right and usually you store data in databases but that will be a little hard right now we don't want to yet get into SQL databases so for the purpose of the assignment and I'm getting into it right now basically the tutorial is done this is how you create an HTTP server now you have to create an HTTP server that will one let users sign up so when the sign up route is hit the user should be able to send some data also sign up should be post and not get login should be post and not get and these two should be gets because you're trying to post data during sign up and in Gate you're just trying to get data so I'm going to add a bunch of comments here add logic to decode body body should have user email and password basically same thing here once you've done that store email and password as is for now in the users array above only if the user with the given email doesn't exist and then return back 200 status code to the client here in the login endpoint you need to basically add the logic to decode both the things again then check if the user given email exists in the user's array also ensure that the password is the same if the password is the same return 200 status back to the cloud also send back a and a token any random string will do for now if it's not the same then return back 401 status code so write the logic for that and then in the questions array for now hard code the questions here for example questions could look something like this given an array return the maximum of the array there should be more things here other than title description maybe test cases should be an array and should look something like this input 105 and output should be 5. this is one test case you can add more so this is the structure of one question you can add multiple questions you can hard code them for now here um and the slash questions get endpoint should pretty much just return the user all the questions in the questions array and the submission one the get submission should return the user's submissions for this problem they should also be a post to slash submissions which should let the user submit a problem randomly AC or accept or reject the solution store the submission in the submission array above so let's create a submission array you can basically keep the shape of this object to be whatever you want so for example this this AI is so good it basically returned me uh this shape which looks good so you don't have to hard code it here whenever someone signed it should be an empty array initially and then when people post a submission it should get stored in this submissions array and then eventually when people get a submission for their problem they should get it from the same array I can guarantee you chatGPT can write this if you give it if I give it this prompt and I say can you complete this function I'm 100 sure it can complete it so you have your own tutor now I have created this boilerplate code there will be different uh ways to tackle it especially like the submissions array shape here might be different but other than that the users are also might be different but other than that it will be fairly straightforward for you to create these six endpoints which should be enough to create the authentication system for lead code and the problem layer for lead code you can also add a few more routes here if you want leaving as hard to Do's create a route that lets an admin add a new problem ensure that>>>>>> only admins can do that so when you're signing up you basically have to give the user an option to either be an admin or a normal user and admins should be able to change these questions array so for V1 hard code all the questions for v2 let the admins add questions as well and then you have to make sure you're authenticating the admin correctly so you'll have to understand middlewares and a bunch of other things one important detail we are not using databases yet so if you if you start your application and then people create submissions and then fetch their submissions if you turn off your application and turn it back on all the data will be gone because we are not yet persisting data into databases that could also be a decent assignment I will do that in the next exercise the third one actually creating database records and everything but for now feel free to use an in-memory object where you store data and just assume that your application will be alive forever and this should be a decent starting point um for you to create an HTTP server for this specific application now applications change and the whole worry around AI taking your jobs the thing is only this structure needs to be defined by a human and eventually maybe not not even that and then a lot of of writing the code is actually repeatable tasks that actually can be done by an AI should be done by any you should understand them though so that is what I would suggest in this video I think if you just copy this whole thing and paste this whole thing into chat GPT it will give you a response but I would suggest you go step by step and ask it hey what is the right way to sign up hey what's the right way to sign in and send back an authentication tokens what are the authentication tokens that are used in the industry how do I make sure that the authentication user is sent back to the user they store it in the browser and then send back in every subsequent requests how do I make sure when the user clicks on the logout button then the authentication token on their browser gets cleared so these are smart questions to ask and figure out the rest of this yourself if you cannot let me know in the comments I will create a detailed video but I think this is a good point to for me to just throw out my knowledge out there and tell you how things are done HTTP server is written in the industry similar very similar they are better structured into multiple files usually typescript is used and not JavaScript and real databases are used but other than that all HTTP servers usually look like this they have such a shape and only application details are different but this is the right way to create the first step of a MERN application which is the backend and I would suggest you complete this this is on GitHub complete this and create a pull request I might not merge it I cannot merge all the put requests but this will be a great first point for you to start writing code and also understanding how to contribute and submit you to GitHub so in the description there is a link to this project you can simply clone the project and complete it or if you want to start from scratch you can do that as well and push it to your GitHub I am extremely excited in the next video also I will be creating such a prompt or like a structure for the application and expecting you to complete it and create a pull request for it I think this is the best way to learn If I complete the whole thing it's no different than any tutorial out there where you understand how something is done you copy paste the code or like do it with the person who's teaching and then you can't build an application of your own these are good comments I have left I think these are enough for you to understand what needs to be done and hopefully many of you should be able to complete it I think I'm not going too fast but if I am let me know and I can slow the pace and teach a little bit more but the overall idea is most applications everything you see on the internet Instagram Facebook is run by HTTP servers in the backend your browser sends HTTP requests the HTTP request structure looks something like this and this is a way to create HTTP servers in Express or node.js now what are the next steps from here it's deploying it on the internet buying domain names hosting them somewhere but this is the gist of the most difficult part everything else is fairly simple to understand here is where you need to write the code and as you might see here it's actually not too difficult basically in 73 lines you can create the skeleton of your app and I think in around 200 lines you should be able to complete the whole application so all the best hopefully you create a bunch of pull requests I am looking forward to those with that let's end the video I will see you guys in the next one bye

Channel: Harkirat Singh

Views: 135,132

Keywords:

Id: od4hT9abIMQ

Length: 59min 29sec (3569 seconds)

Published: Thu Apr 06 2023