Full HTTP Networking Course – Fetch and REST APIs in JavaScript

Master the HTTP networking protocol in this hands-on course. HTTP is the backbone of the modern web. And this course is a deep dive into all the fundamentals you'll need to know to master web networking. Lane Wagner created this course. Lane is a senior backend engineer and is the creator of Boot.dev, where he has published many interactive courses. Hey, Free Code Campers. Welcome to my latest course on web networking. And specifically the HTTP protocol. In this course, we'll be doing over 80 coding exercises. And at the end, we're going to build a web crawler in JavaScript from scratch. I believe in learning by doing, so we'll really be doubling down on that here. There are actually two ways you can follow along with this course. The first option is to write code on your local machine while you follow along with this video. The second option is to head over to Boot.dev, where I published the interactive version of this course. In fact, I'll actually be teaching this video course by doing the interactive version of the course on camera and explaining each step in detail. There's also a link in the description below to all of the raw code samples in a GitHub repository. It's totally up to you how you want to take this course. I have provided the link to the interactive version below if that's how you want to do it. Just so you know, this course is part of a larger backend career path over on Boot.dev, which is designed to take students from zero to hireable as a backend developer. Just so you know, all of the code we're going to write in this course will be JavaScript. And while I'll be explaining every exercise, I will assume that you have a basic understanding of JavaScript and ES6 syntax. If you don't, that's okay. I'll provide some beginner level resources for JavaScript in the description below. I love getting feedback about all my courses. So if you do have feedback, one of the easiest ways to get in touch with me is on Twitter at wagslane. I'd love to know what you think. Also, if you find that you like the way that I'm explaining these coding concepts, feel free to subscribe to my YouTube channel. It's at Boot.dev. If you get stuck or have questions during the course, I'm going to recommend three different places. The first is the Free Code Camp Forum. The second is the Free Code Camp Discord. And the third is the Boot.dev Discord, where we have a channel dedicated to this course. Again, those links will be in the description below. Before we jump right into the first lesson, I want to take just a second to explain how you can get the most out of this course. I've designed this course to be hands-on and project-based. Tutorial hell is a term you might be familiar with, but it's basically the idea that it's easy to slip into a passive mode when you're consuming educational content. The way to get the maximum value out of this course will be to watch my explanation of the concepts and coding challenges, but then to write the code yourself on your own machine or on Boot.dev. If you're never pausing this video, you probably aren't learning very effectively. I'd encourage you to pause this video before every solution I give so that you have a chance to solve each challenge yourself. Once we finish all of the lessons and challenges, we're going to go build a web crawler in JavaScript from scratch. And we're doing that because it's really important to use what you learn so you don't forget it all. Do not binge watch this video from start to finish. This course is designed to take several days or weeks when done properly. It's really hard to learn a concept after just one explanation. So don't be afraid to rewind the video and re-watch the explanation of a concept that you might still be a little fuzzy on. My name is Layne Wagner. I'm the author of this course and I'll be your instructor every step of the way. Let's talk about HETE, Hyper Text Transfer Protocol. Before we jump into the nitty-gritty details of the protocol and how it works, it's important to understand why we should care. Let's talk about image sharing. So you've got your camera phone and you take a couple of pictures. Now, it's important to remember that 20 years ago, right, in the 90s and early 2000s, it wasn't easy to share digital photos with a friend or family member. It was easy to take photos. We've had digital cameras for a long time, right, but it wasn't easy to share them. These days, if you use an app like Instagram, right, you take your photos and then I on my phone can fairly seamlessly download your photos over the Internet onto my phone. And it's really important to understand that this is really the secret to Instagram success. They made photo sharing really, really easy and they did it just using the Internet. So you may be wondering what the heck does this have to do with HTTP? Well, HTTP is the protocol that was almost certainly used to download those images. HTTP is the most popular protocol for communicating online. You've probably noticed that every time you go to a website, you see that HTTP colon slash slash section of the URL, right? That's because every time you visit a website online, you're using the HTTP protocol to do it. So I've used this word protocol protocol a couple of times and you may be wondering what the heck is a protocol. Sounds like a really complex, really complex thing. It's actually not. Protocols are very simple. Let's jump right into what we mean when we say protocol. Let's imagine that I write a note to you just on a piece of paper that says raise. Raise your hand. I write this down on a piece of paper and I hand it to you. And when you get the piece of paper, you read it and you raise your hand. So the question is, upon reading the piece of paper, how did you know that I wanted you to raise your hand, right? You saw these you saw these symbols on the piece of paper and somehow you were able to understand the hand should be raised. Well, it's because we have a protocol for communicating over text, right? We've decided that these symbols that look like this are A-I-S-E that they form a word, right? And that word means to lift something up and hand. We've decided that these symbols H-A-N-D mean, you know, this thing at the end of my arm. And it's just a set of rules that me as the person communicating and you as the person I'm communicating to have agreed upon. And now that we've agreed upon that set of rules, we can communicate, right? Over this, in this case, the medium of a piece of paper. A protocol on the Internet works in the same way. We decide upon a set of rules, right? And then two computers can both follow those rules and communicate with each other. In the case of computers, instead of sending plain text, right? R-A-I-S-E, we would send kind of strings of ones and zeros. But the idea is the same. This is just information that's sent between two computers over a network and the protocol, right? Hypertext transfer protocol. The protocol is what tells the computers how to parse this information as something useful. Because we have this protocol, this way of parsing the ones and zeros as useful information, I am able to send you a photo over the Internet, right? Again, it's just a bunch of ones and zeros that represent something. But because you and I agree on this protocol or this language, I can send you the image of, for example, your niece's birthday party, right? And your phone is able to parse all that information and display an image. But that's what makes HTTP so powerful. All right, let's jump into the first coding lesson. I'm here on boot.dev. I'm in the Learn HTTP course. This is exercise number one. So it's important to understand that in this course, we'll be interacting with a live API. We'll be writing JavaScript code kind of on the front end that makes requests to the backend server via HTTP. And that server is a Fantasy Quest server. Fantasy Quest is a made-up game. So while this is a real API, we'll be interacting with a server for essentially a fake video game. Give us some context to all these assignments and coding challenges that we'll be completing. Okay, so the assignment says, I wrote the getItemData function for you. It retrieves items from Fantasy Quest servers via HTTP. Okay, so it looks like we have this getItemData function. Looks like it's grabbing some data from this URL. We don't really need to know how the internals work yet, but that's what the function does. And it says, I also wrote a logItems function for you, so logItems. Call it with the items variable. We're getting back from the getItemData call as a parameter. Okay, cool. So up here, where we were supposed to do our work, looks like we're getting the items from the server. And then here, we just need to log those items to the console. Let's go ahead and run that and see what happens. Cool, looks like we got some items. Those look like items to me. Go ahead and submit that. Awesome. So moving along. All right, HTTP requests and responses. So at the heart of HTTP is a simple request-response system. The requesting computer, also known as the client, here we have our client, asks another computer, in this case the server, for some information. So when we make an HTTP request, right, we, acting as the client, make a request to a server. The server processes that request and sends some information back to us in a response. That's the entire HTTP life cycle. It's all based on request-response. It's very synchronous in that way, right? I send a request, then I have to sit and wait till a response comes back, right? And then those words, client and server, it's important to just understand that when we say client, we just mean the computer sending the request. When we say server, we mean the computer that's responding to the request. And that's really all it means. There's not like a special type of computer. It's really just who's doing the sending and who's doing the receiving in any given communication. So we'll talk a little more about the specifics of requests and responses later. For now, it's important to think of it as a simple question-and-answer system. So we're asking questions to the server and we're getting answers, right? In the last assignment, we made a request to the FantasyQuest server for a list of all the items in the game, and we got back a list of items in the game, right? So which comes first, HTTP response or HTTP request? It's going to be the request. So we've talked about how HTTP is very widely used for internet communication. In fact, it powers websites, right? So if we scroll down to this diagram here, the client, which can be your computer, it can be your phone, makes a request to a server. Server responds with a website, right? So this is what happens every time you load your browser and you type in the name of a website, right? boot.dev, precodecamp.org, youtube.com, it doesn't matter. When you enter that URL into your browser, you're making an HTTP request to a server and that server is responding with the data that makes up their website, right? The images, the text, the HTML, the CSS, all of that comes back via an HTTP response. So the question is, website data is requested via which internet communication protocol? And that's going to be HTTP. So looking at this error, unexpected token, doc type is not valid JSON. So we're going to get to JSON later, but basically stands for JavaScript Object Notation and it's one of the most common ways to kind of get raw data from a URL, right? In this case, we're trying to get some item data from this URL, but we're getting back some HTML. That's a problem. According to the assignment, it says, that's because we are not getting a proper HTTP response. After all, we aren't making a request to a valid URL. Why not? Let's see. So get data here, takes the URL as inputs. Okay, up here, we're not, there's no URL getting passed into the get data function, right? It takes one parameter. We're not supplying a URL. So fix the code so that the get data function requests the item URL. Okay, so let's pass that in there. And try that. Awesome, that looks like item data to me. Okay, so we've made some HTTP requests to a couple of different URLs now, but it's really important to understand that URLs are not specific to HTTP. If they were, we wouldn't need to prefix so many URLs with that HTTP colon slash slash. The purpose of the prefix is to tell the, especially the computer that's making the request, which protocol to use, right? We want to use HTTP to make this request. So that's why we are prefixing the URL with that HTTP colon slash slash. Like I said, there's plenty of others, things like HTTPS, Mail2, PSQL. There's a lot of them, but the question here is, HTTP is the only protocol that uses URLs that is false. Let's talk about clients, clients and servers. I mean, the context of HTTP web applications, right? And as a general rule, it's important to understand that generally speaking, clients exist on the front end of an application and servers exist on the backend of an application. The front end of an application is what the user sees. So it's typically a mobile app or a website, right? It's the thing that the user is directly interacting with. The backend is essentially everything else. And in order to demonstrate that, let's take a look at the example of YouTube. So when you want to watch a YouTube video, you load up an app on your phone. It's probably written in Swift or Objective-C or React Native. But the point is it exists locally on your phone and it contains all of the code that is required to kind of render that visual YouTube interface. So that app and your phone, we can consider the front end. Again, because it's that visual interface that you're interacting with directly as the user. However, as you probably know, your phone does not locally store every YouTube video on the planet and it doesn't have every comment stored locally, right? So when you load a video, your phone actually needs to make an HTTP request to YouTube's backend. And again, remember just that by definition, the backend is basically everything that's not on the front end, which is just the stuff on your phone. Okay, so there is a computer on YouTube's backend, right? Probably in some data center somewhere that's going to serve or handle that HTTP request. So now that we have this kind of a computer making a request and a computer processing the request, we have the ability to say who the client is and who the server is. So in this case, your phone is the client because it's the one sending the HTTP request here, right? And the backend computer, the one in the data center, is acting as the server. Cool, now we're starting to see how the client makes requests to the server. What gets a little interesting is let's pretend that this server, all it has is video data locally. But the client actually asked for videos and comments. So what needs to happen now is there's probably some other server within, again, YouTube's backend. And that server has comments on it, right? It stores the comments locally on this server. So what can happen now is this first server that's trying to process this HTTP request, right? That asked for videos and comments. Well, it can sort the video, but it needs to grab some comments. So it's going to send its own HTTP request over to this other server. Say, hey, I need the comments for this video, right? Now, in this interaction, right, if we just look at this interaction over here on the right, in this case, this first computer is actually now acting as a client because it's sending an HTTP request. And this server over here is acting as a server, okay? So this is an important example of when a computer is acting as both a server and a client, and that's totally okay. Okay, so this computer over here grabs the comments that were asked for and it sends them back in an HTTP response. This first server, this server now has all the data it needs to respond to the client, and it can send a response back to the client. Hopefully, this has done a little bit to explain how clients and servers interact with each other and how really, at the end of the day, the word client and the word server only matter within the context of a single HTTP request. If a computer is sending an HTTP request, we consider it the client. If a computer is responding to an HTTP request, we consider it a server. And that's really all that client and server mean. So now we've got a quiz about clients and servers. It looks like the question is, does the example code run on a client or a server? And here's the example code written in JavaScript. And it looks like there's a URL here and a request is being made to that URL. So this has to be a client, right? Because a server doesn't need a URL to make a request to. Server serves the response. This is definitely a client. Okay, another client-server question. This one is, the computer responding to requests at the URL, https, api.boot.dev is a, well, if it's responding to requests, it must be a server. So we're finally ready to start talking about the Fetch API. We've already used it in a couple of the other coding challenges, but we haven't really explained what it is. In a nutshell, the Fetch API is a set of built-in functions and tools that we can use when we're writing JavaScript code in order to make HTTP requests. So an HTTP request has a lot going on with it. There's the URL that we have to send the request to. There's a body, there's some headers. And at the end of the day, it would be a big nightmare if every time we had to write an HTTP request, we needed to write all of that logic from scratch. So the browser, in this case, provides that functionality through a built-in function called Fetch. And that's what we'll be using to make our HTTP requests in this course. Okay, let's look at how we can actually use the Fetch API in code. It all starts with this Fetch function. Now, this is not a function that we've defined in our code. It's a function that's made available to us by the browser. It's a built-in function. The way it works is it takes two inputs. It takes a URL, which is the URL that we want to make a request to, and a settings object. We'll talk more about the different ways we can configure an HTTP request in just a minute. After that, we actually do have to use this await keyword in front of the function call, in front of the fetch call. And that's because when we make a fetch call, we're quite literally sending bits of data, sometimes halfway across the world. So we need our code to pause and wait for that response to come back. It's going to take a while. And the await keyword allows us to do that. We'll go way more in depth about asynchronous programming and kind of what's going on under the hood here. But for now, just know that when you make a fetch request, you do need to await it. What we get back from all of that is the response object. The response object has a bunch of stuff in it that, again, we're going to get to. But for now, we just want kind of the meat of the response. We want the data, right? So we know that this server, in this case, the fantasy quest server that we're working with, responds with JSON data, which stands for JavaScript Object Notation. It's basically data that looks like this, kind of these brackets and colons and property value, right? Key value pairs. It is a JavaScript object, but in text form. So this.json method allows us to take that response body, which again is just text that looks like a JavaScript object, and parse it into an actual JavaScript object in our code so that we can work with it. And again, that method call, that.json method call, that needs to be awaited as well because it's an asynchronous operation. Once we're done with all of that, we have some response data. And that is a JavaScript object. Again, it's just a JavaScript object with key value pairs, right? Things like, you know, armor, five, name, lane. It's just very simple, simply structured JSON data that comes back from the server. So let's get to the assignment, is to fix the bug in the code. The problem is that we aren't waiting for the response to physically come back across the internet connection before continuing with our code. Okay, let's take a look. So it looks like down here, there's a get settings function. Looks like this is setting up the settings for the fetch request. This is a URL that we're gonna be making the fetch request to. It looks like there's some other authentication stuff, some logging. But really, at the end of the day, what we're interested in is this part here. So we're looking for a bug that we need to fix. And again, according to the assignment, it's because we're not waiting for the response to physically come back. So what happens if I just run the code right now? Type error, response.json is not a function. Well, that makes sense to me because this response object here is going to be garbage if we're not waiting for the fetch request to complete. Right, so we're trying to call this.json method on essentially a garbage object. So let's go ahead and await that fetch request. And there we go, we get all of our item data as we would expect. So we've already talked a lot about clients and servers and how clients make requests to servers. Well, this question is, the front end of a website is A and the options are client and server. Well, the front end of a website can't really be a server. I mean, there are certain scenarios in which we could consider it a server, like very specific and rare scenarios. But generally speaking, the front end of a website is going to be a client because it's going to be making requests out to the backend. Okay, this question, a mobile phone is often used as an, and then the answers are HTTP server or HTTP client. Now, again, a phone could be a server. Like there are scenarios in which a phone could respond to an HTTP request, right? A phone is a computer. We could run some server software on it, but that would be very, very strange. Most of the time your phone, when you open it up and use an app on it, it's going to be making HTTP requests out to some backend server. So most often a mobile phone is an HTTP client. So let's talk just a little bit more about web servers. So most applications that you've probably written up until this point kind of start, do a bunch of stuff and then stop, right? Terminate when they're done doing what it is they're supposed to do. Servers are interesting in that they don't stop running, generally speaking, right? You turn on the server and then it just sits there on a computer listening for incoming HTTP requests. As they come in, it handles them, right? It sends responses, but it's just kind of always on. And again, this is important to understand because it's very likely that there's a lot of applications you've written up to this point that don't act that way, right? You run the code, the code exits. You run the code, the code exits, not with servers, right? With a server, you start up the server. And then in order to test your server code, you would actually need to send HTTP requests to it, right? Make sure that it behaves as you would expect given certain HTTP inputs. And then you'd actually need to probably manually stop your server from running because they're not designed to shut down automatically. So the question is, good web servers are listening for requests only when a client is active, only during business hours, or always? And the answer is always. Another question about servers here. So the question is, a server is the blank of an application and the options are backend and frontend. Well, a server isn't interacted with visually, right? So it can't be a frontend. So I'm gonna go with backend on this one. Okay, another question about servers. This time, it's actually a statement. It says, any computer can be a server, but the best servers are special made for serving data. Before I answer this question, I wanna just explain a little bit about kind of how the modern internet works. So generally speaking, new kind of web companies are spinning up their servers in data centers. Data centers owned by companies like Google Cloud Platform, Amazon Web Services, Azure. And the computers in those data centers are hyper optimized to be servers. They're optimized for server side workloads. That said, it's important to understand that realistically you can run server software on pretty much any computer, right? In fact, when we're developing our backend applications, even if we will eventually deploy them in a data center, we're generally developing them on our local machines. On just our work laptops. So the servers work perfectly fine on laptops, but it is important to understand that when the hardware is optimized for server workloads, you'll get more bang for your buck. So the answer here is going to be true. Any computer can be a server, but the best servers are made for serving data. Let's talk about web addresses. So I'm over here on my computer. And I want to communicate with, let's say, some server over here, right? Maybe I want to make a request to that server. Maybe I want to browse a website that's hosted there. Point is, I want to communicate with it. Trouble is, the internet is made up of much more than just my computer and the server, right? We have all these other devices that are connected to the internet. We have tons of other servers that are connected to the internet. So how can I tell the routing mechanisms of the internet that I want to communicate directly with that server? Well, that's where web addresses come in. And more specifically, what we call IP addresses or internet protocol addresses. So what does an IP address look like? Well, it looks something like this. 8.13.156.7, OK? There's four sections separated by periods. And each section can be between 0 and 255. Four numbers, each number is one byte of information. This 8.13.156.7, this is just an example, right? It is a valid IP address, but I don't know if there's anything connected to this IP address. So just keep that in mind. There's many, many possible valid IP addresses. The only rule is that each section has a number between 0 and 255. So whenever you order a package online, you have a unique address for your home. And that's how the e-commerce store, say Amazon, knows how to send the package directly to your door and doesn't get mixed up anywhere else, right? IP addresses work the same way. Every device connected to the internet has its own unique IP address. So in this example from before, where I'm trying to communicate with the server, all I need to know is its unique IP address. And if I know that, I can communicate with it. It is important to understand that this format here, these four numbers separated by periods, this is just one format of an IP address. And it's actually the more popular format. It's called IPv4. But there is a newer format that's being used called IPv6 or IP version 6. And it looks similar, but there's actually way more possible IPv6 addresses than IPv4 address. And in IPv6, each section is separated by a colon, colon rather than a period. We can tell them apart, right? And there's actually one, two, three, four, five, six, seven, eight sections in an IPv6 address. And each section actually has more information in it. So the point is there's way more IPv6 addresses, way more possible IPv6 addresses out there than IPv4 addresses. And that's really important because we're actually running out of IPv4 addresses. There's really not that many combinations of numbers here. And so we're running into a problem where so many people and so many devices are online, and we're running short on these IPv4 addresses. So IPv6 has kind of given us more options. The same principle applies, right? The addresses are unique and uniquely identify machines on the internet, but we just have more kind of available to us. So as you're looking at IP addresses, just know that you'll see them in both formats. So it seems like IP addresses are very useful, right? Just like how if I were to give you my physical address, you could send me a package from anywhere in the world. If I were to give you my IP address, you could contact me over the internet from anywhere in the world. But this is where things start to get problematic, right? If I want to contact Amazon's servers, right? Say I just want to navigate to amazon.com. I want their servers to give me a web page. It's not very helpful if I need to know their IP address, right? I don't have amazon.com's server's IP address memorized. No one does, right? So that's where DNS comes into play. DNS stands for Domain Name System. And one of its main purposes is to map human readable, right? Easily read names like amazon.com to IP addresses. Okay, so now when I want to go to amazon.com, I can type into my computer amazon.com, and the domain name system is responsible for looking up the IP address that's associated with amazon.com and giving it back to me so that I can make that request to the server that I'm looking to get to, right? So this whole system of essentially going out, using IP addresses to talk to computers, that is used. It's just the second step. The first step that our computers kind of do under the hood automatically for us is to resolve a domain name to an IP address. And this has the side benefit of, if amazon wants to change out their IP address, or maybe their IP address kind of just changes because their internet service provider swaps it out on them, they can update this mapping, right? They can update the IP address under the hood without needing to change their domain name. So all the users can still just navigate to amazon.com, but now they're actually going to maybe a different server or a different IP address under the hood. So just to recap, there's essentially two steps every time we want to send an HTTP request to the server on a given domain name. The first step is to resolve DNS. What that means is we're taking the domain name, right, say google.com, and resolving it into an IP address. So we're obtaining an IP address. Step number two is to use the obtained IP address to actually make that request across the internet. Okay, let's talk about how a domain name relates to a URL. So here we've got a full URL. This is actually a webpage on Wikipedia about miniature pigs. And the interesting part is that actually this en.wikipedia.org is the only part of the URL that makes up the domain name, or it is the domain name. The rest of the URL is important, but it's not related to the domain name, and it's not the part that we need in order to look up the IP address of the domain name. All we need is the domain name to look up the associated IP address. Cool. So in this assignment, we're going to be working with the Cloudflare API. So Cloudflare is a tech company. They have this public API, cloudflare-dns.com, that we can use to look up the IP address if we give it a domain. So here you can see we're passing in a domain as a variable, and we'll get to all of this syntax later. Just know that we're essentially calling the Cloudflare API, giving it a domain name, and we're expecting to get back a response with an IP address in it. So the assignment says on line nine, the function fetch IP address is simply logging the entire object we receive from Cloudflare to the console. Run the code to see the structure of the object. So let's go ahead and run that. Okay, so this is the object that's coming back from Cloudflare, again, as a JavaScript object. Our job is to dig into this object to get at the IP address, which just kind of looking through this, there's the IP address, right? It's those four numbers with the dots in between. And then it says, update the code to return just the IP address and remove the logging. Okay, so we don't need to log this, but we do need to return... See, it's going to be resp object. The key that we are interested in, looks like it's going to be... Tell you what, let's do this. Let's go over to... We've never used this site before. Super useful, it's called JSON lint. I'll blow it up just a little bit so you can see. It lints the JSON object, so we can kind of see the structure of the response a little better. Okay, so if this is our top level object, right, resp object, then we want the answer key. So we're going to do resp object dot answer. And then that is an array. So we want the first item in the array, which is going to be this whole object. It's going to be zero, index zero. And then we want that IP address. It's going to be the data key. So, dot data. And let's just run that and see what happens. Found IP address for domain, api.boot.dev. And that's the IP address. Okay, perfect, so it looks like it's working. I do want to just really quickly point out that this is a little dangerous. If we wanted to make our code a little more robust, we could do some checking here to make sure that these fields and the length of this array actually exist so we don't get some sort of runtime error. Cool, let's go ahead and submit this. Great, so to recap, a domain name is just one part of a URL. And it's specifically the part that we use to look up the IP address of a server. And then, of course, we can use that IP address to go contact that web server, maybe to see a web page that is being hosted by that server, for example. So, let's run through an example of how you might deploy your own website to the internet using IP addresses and domain names. So, step number zero, which actually isn't written down here, it's kind of assumed, is that you need a web server software and you need a website that you want to serve, which is basically a collection of HTML and CSS files. Assuming you have that, which, again, is kind of out of the scope of this course, so I'm going to just kind of wave my hands and say you should have that. Assuming you have that, then the next step is to take that software and deploy it on a machine. Typically, what you would do today is deploy it on some cloud platform, right? Something like a Netlify or a GCP or an Azure or an AWS. And when you do that, you will get an IP address of the machine that you deployed to. Again, these large kind of cloud providers would give you the IP address of the machine. It's just say, hey, here's the IP address where we just deployed your code, okay? Once you have that IP address, then you would need to go buy a domain name. Again, you go find a provider of domain names. They're called registrars, something like Amazon, Google, Namecheap. You go buy your domain name, and then, again, using their software, you tell them what your IP address is. So, you know, say you buy xyz.com. You go and you say, hey, I just bought xyz.com. Here's the IP address I want it to point to, and then they go and update the domain name system for you. That's just kind of a high-level overview of how you would go about actually deploying a website to the internet. With all that in mind, the question for this quiz is, which is a human-readable label that converts to an IP address? That would be a domain name. Question here is, why don't we require users to type in IP addresses directly to browse the web? I'll pause for a second if you want to take a look. The tongue-in-cheek thing here is, I absolutely hate when websites auto-play any sort of audio, music, videos, whatever it is. Drives me crazy. I think it's a terrible user experience. So I think the same thing applies to requiring users to type in the IP addresses of websites. It would just be a terrible user experience. It's much easier to remember domain names like boot.dev and amazon.com. So using IP addresses directly while possible, just awful, awful user experience. Okay, so we got another assignment here. Looks like what we'll be doing is parsing a URL string into a URL object. So JavaScript has this built-in API, the URL API, that's really useful for parsing a URL string, like this https://example.com slash example path, and breaking it down into a JavaScript object, which will then essentially have all the parsed sections, so we'll be able to extract things like the domain or the host name or the scheme or whatever. It handles all that parsing logic for us. The assignment is completely get domain name from URL functions, that's this one up here. Given a full URL string, it should return the domain or host name. Okay, so let's take a look at a couple things. So let's just log. Let's just start by logging out the URL that's being given to get domain name from URL, just to take a look at what it looks like. Okay, so that's the URL coming in. And then because we didn't return anything, the test suite is saying the domain name for the URL is undefined, right, because we didn't return anything. So let's use this syntax. And I'm just going to pull up the docs as well. This is the MDN docs for the built-in JavaScript URL API. Looks like we can use this syntax, new URL, and then pass in the URL, so let's go ahead and do that. Do const, const URL object equals new URL. Give it the URL string. Now we should have a URL object here. Now that object should have, let's take a look at the docs again, some properties on it. Yeah, okay. So we've got the hash, the host, the host name, a link, password. So these are all kind of different sections of the URL. We care about the host name. That's what our sort of our assignment said it wants. So we will just need to return URL object dot host name. And looking at this URL that's being passed in, I would expect the host name just to be boot dot dev. That's just the domain name part. Let's run that. And we get the domain name for HTTPS boot dot dev learn learn Python is boot dot dev. That looks right to me. I'm going to go ahead and submit that. We've talked a little bit about DNS. Let's go into just a bit more detail. So DNS or the domain name system is essentially a giant phone book for the internet. Back in the day, if you used a phone book, you could look up your friends by their name and find their phone number. With DNS, we can look up a website by its domain name and find its IP address. Very, very similar system. Question is, how does DNS work under the hood? Well, there's this organization called ICANN, I-C-A-N-N. And it stands for the Internet Corporation for Assigned Names and Numbers, which, as you can probably tell, is a very poignant, very accurate name. ICANN manages the domain name system for the entire internet. And whenever your computer has a domain name that it wants to find the IP address of, it contacts one of ICANN's root name servers. And then from there, it's able to resolve the IP address. And if you're wondering how it knows where to find the root name servers, that's typically built into your computer's networking configuration. So once your computer contacts ICANN, ICANN is able to look up that IP address from their distributed DNS system. It's a very large, large system, right? If you think of DNS as just a phone book, ICANN is basically the publisher of the phone book. They control what goes into the phone book. And in case you're concerned about the inherent power here, it is a not-for-profit organization. So it's not your typical tech company. Okay, the question for this quiz is, DNS is a or an, and the options are IP address, programming language, system that converts domain names to IP addresses, or domain name. I'm going to go with the system that converts domain names to IP addresses. The next question is, what does ICANN do? The options are keep the DNS system up and running or control what you say online. They keep the DNS system up and running. Let's talk about subdomains. So if we look at this kind of root domain here, you'll see there's two parts. There's.dev, which is the TLD or top-level domain. So it could be.dev like we have. It could also be.com,.org,.net, right? There's quite a few different options. And then boot is the domain name. A subdomain actually prefixes a domain name. So for example, our API is hosted separately. So boot.dev hosts our website, what we are on right now, the learning platform. But api.boot.dev actually hosts our API, which is what the website kind of uses to go fetch data, update user records and passwords, all that kind of stuff. The api.boot.dev is essentially our backend. We also have a blog.boot.dev, which is just a separate website that hosts our blog. So subdomains can be used to basically break up the resources hosted on a domain name without having to actually go buy new domain names. We paid for boot.dev, and now we own all of the subdomains that can be prefixed under boot.dev. So let's take a look at the assignment. It says, I've updated the get item data function. That's this function here from before. Now it accepts just a domain as input. It's more convenient this way because it means if the API we're using ever changes its domain, we only need to update one variable. OK, so let's see what the heck we're talking about here. So we've got this get item data function, and it's making a request out to this URL. And it looks like what's happened is we've made the domain portion of the URL a variable, so it can actually be swapped out fairly easily. And that get item data function takes the domain as an input. OK, so the problem is there's a bug. The API is not hosted on boot.dev. It's hosted on the API subdomain. Fix the bug. OK, so when we run the code as it is, we get unexpected token. Doctype is not valid JSON. This looks to me like what's happening is HTTPS colon slash slash boot.dev is hosting a website, right, which starts with this HTML tag here. It's not valid JSON. And what we're looking for here is a JSON object or JavaScript object to come back from the API. So we need to actually hit the API instead. And when we run that, it looks like we're we're properly getting back our item data. So we've already talked about URLs, but we haven't really talked about URIs. You might hear or see URIs. And frankly, sometimes it can be confusing because URI sometimes looks like URL with a lowercase L, but it's actually an I. Anyways, let's let's get into it. So URI stands for Uniform Resource Identifier, and it's essentially a superset of URLs. So if we come down and look at this this Venn diagram, you've got URIs like all possible URIs. And within URIs, a subcategory is URLs. OK, so URL is just type of a URI. Another common type of URI is a URN. And URNs and URIs, they can refer to other things that aren't necessarily accessible via the Internet, things like ISBN numbers of books. So generally speaking, when we're working with the Internet, we're working with URLs. But it's just good to be aware of URIs. You will see them around. And sometimes, for example, you'll be asked to provide a URI. And really, you're just providing a URL again because a URL is a valid URI. OK, enough about that. So the question, all URLs are URIs. That is true. Next question is, all URIs are URLs. That is not true, right? If we scroll back down, take a look at this. All URLs are, in fact, URIs. Not all URIs are URIs. So if you use exercises back, we wrote some code that extracted the host name from a URL string. And now we're going to look at kind of all the different sections of a URL. So take a look at this giant URL here. The reason this URL is so large is I've gone ahead and added every possible section. This is like kind of the most sections a URL can have because every section is present. Typically, sections are optional, so you won't always see kind of all of this information. Let's go over each section and talk about what it means. So the first section is the protocol. It's right here, right? HTTP colon, in this case. Could be a different protocol. We might use something like Postgres for a database connection. But because this URL is accessible via an HTTP call, it's prefixed with the HTTP protocol. Cool. Next up, we got the username and the password. So test user, and then a colon, and then test pass. So the colon is the separator. These are the two different sections for username and password. Again, these are optional. You won't always see usernames and passwords in URLs. I find that I use them a lot, again, to kind of access resources in code. So to do things like, you know, get access to a database or a piece of infrastructure, you'll very rarely see usernames and passwords in a URL when you're just browsing the internet. We typically use usernames and passwords on a website, kind of in the site, via form submission, not in the URL itself. That would be kind of strange user experience to type it in up there at the top. So that's what those are. Next, we have the hostname. We've already talked about that. This at symbol, you only need the at symbol if you've included a username and password. So it's kind of an optional delimiter there that separates the username and password from the hostname. The port. This is the port that we'll be using to access the information on the server. Now, a port is not necessarily optional in the sense that we're always using a port to make an HTTP request. However, there are defaults. So for the HTTP protocol, 80, just 80, not 8080 like we have here, would be the default. So if you don't specify a port, the request will by default use port 80. And then HTTPS, which is what you use for a secure HTTP call, defaults to port 443. Again, when you're browsing the internet in your browser, you don't normally have the port because pretty much every website online uses the default ports. But if you need to specify a port, this is where you would do it. And it's a prefix there with a colon. Next comes the path. It comes after a slash. Very often in websites, the path is used to kind of, you know, show different web pages. So depending on the path in the URL, you'll navigate to a different page. And then pages can kind of be nested, right? So you can have multiple slashes in there with multiple, you know, sections of the path. In this case, we just have the one section of the path. Next is the search, which I don't love that name here. You'll also hear it called the kind of the query parameters. But basically, it starts with a question mark. And then it's a set of key value pairs. So here we have the key is test search and the value is test value. And then we could have kind of more parameters after the question mark. We'll get to that later in the course. Just know that this section of the URL is called the search or the query parameters. And again, we'll talk about the details later. And then the last section is called the hash or the fragment. And very often in websites, the fragment is used to kind of link to a specific section on a page. But again, it could be used for other things outside of the context of websites. OK, so that was a lot of information. You don't need to memorize that. I just wanted to give you a high level overview. We will be talking about each section kind of in more detail as necessary throughout the rest of the course. So let's get to the assignment. We'll be using the URL API again. So if you remember over here on the MDN docs, we're going to be using this kind of built in function in JavaScript to split apart a URL. And we're tasked with splitting and printing all the different sections. So, again, using this example URL, we actually want to be printing this kind of breakdown. Let's go ahead and start. It looks like the print URL parts function was already partially written for us. A URL string is coming in as a parameter and we're kind of breaking that down into a URL object. Now, our task is to actually print out these messages. So it looks like these are what the properties on the URL objects are. So let's go ahead and print do console dot log URL object dot protocol and see what happens. OK, so that gives us HTTP. I'm guessing dot username will give me username dragon slayer. Yeah, great. So we should be able to use this URL object to print everything we need to. I'm going to go ahead and use some string literals to do this. So on the first line, it looks like we want to be printing protocol and then we want this to be dynamic. So we're going to use a JavaScript template literal to interpolate URL object dot protocol in there. So I'll just run that. And we should get this first line, protocol colon and then the protocol itself, which is HTTP colon. Well, now we can just do this for the rest of the sections. Eight, nine, eight. OK, cool. Username, password, post name, court, path name, search. And run that and see if it looks reasonable. So the URL that in actuality we're splitting up is this one right here. And it's looking pretty good. I think we've got all the sections there. I'm going to go ahead and submit that. Let's go into just a little bit more detail about these eight sections or eight parts of a URL. I'm going to scroll down so you can see this image just a little bit better. Starting with the protocol, the protocol is required. You cannot not have a protocol. The computer needs to know how to take your message and transmit it in code. At the very beginning of this course, we talked about how HTTP is kind of like a language that computers use to speak to one another over a network. And so when we go out to fetch a resource using a URL, we need to know what language, what protocol to use to do that. So not optional. Usenames and passwords are absolutely optional, right? If a resource doesn't need a username and password, it's considered public and doesn't require credentials to get access to. A domain is required, which makes sense, right? There's no way for us to reach out to another server and get information unless we can use that domain name, resolve it into an IP address, and then make a request across a network. So definitely need a domain. The default port for protocol is used if we didn't provide one. So again, HTTP, its default port is 80. HTTPS, its default port is 443. Every protocol has default ports. And so you'll only need to specify a port if for some reason you're deviating from the defaults. This is used a lot in kind of local web development. When you're building websites on your own machine and you want to kind of access the test version of the site in the browser, you might just use different ports so that you can access many different test sites on your own machine, right? Without having to get domain names for every single one of them. So access to the path is optional in that you don't need to put it, but one is always there. The default is just the root kind of slash. A query is absolutely optional and a fragment is completely optional. And then again, I just want to stress you don't need to memorize every section of a URL. This is one of those things where it's useful to be familiar with all of the concepts from a high level and you'll get more familiar with them over time as you have more and more experience as developer. But it's also something that's fairly easily Googleable, right? When you have questions about a specific URL, maybe you're a bit confused. It's something that's fairly easy to look up. So the question is the eight pieces of a URL are always present and that is false. There are some that are optional. The next question is HTTP colon slash slash fantasy quest dot app is a valid URL. That looks valid to me because we have the protocol. We have the domain. Those are the only two sections that are completely required. So I'm going to say true. Next HTTP colon slash slash username colon password is a valid URL. Problem here is that we are missing a domain. All right. We've got a username and password with no domain. So that's invalid. I want to point out just a couple more interesting things about the protocol section of a URL or a URI. So first, as we've already talked about, HTTP is not the only scheme. So HTTP, FTP, mail to HTTPS. These are all valid schemes or protocols that can be used. And not all the schemes are kind of postfixed or end in that slash slash. So HTTP colon slash slash. The slash slash section is there because HTTP as a protocol has an authority component. It has that username colon password section of the URI. Mail to, for example, does not. And so that's why the protocol, which is mail to colon, does not end in a slash slash. With that in mind, let's move on to the assignment. So we're back to working on fantasy quest in the fantasy quest game menu. We show users the email addresses of their friends. We want the email addresses to be clickable hyperlinks. So I don't know if you've ever come into this or come across this online, but you may see email addresses in your browser that you're able to click and it will automatically open your email client. And that's because your computer understands the mail to URI and knows that that means to open an email client with a draft email ready to go, kind of to send to the email address address in that URI. So we've been tasked with completing the get mail to link for email function. It should return a mail to hyperlink for the given email. OK, so this seems like it should be pretty straightforward. We're going to want to return and I'm going to use a JavaScript template literal and do mail to colon. That's the protocol. Right. And then that just is followed by the email address. So mail to colon and the email itself. Go ahead and run that, see what it looks like. The mail to link for Slayer at F quest dot app is mail to colon Slayer at F quest dot app. That looks pretty good. I'm going to go ahead and submit that. Let's talk about ports. So let's say I have my web server over here, my server. And let's say I'm hosting a website. So I've got some piece of software that's hosting my website. Brought here like this. And I've got some customers that want to access my website. They'll be over here and they're sending HTTP requests. To my server so they can get access to my website. And that works just fine. The trouble is, what if I want to host multiple pieces of software on the same physical machine? Right. So let's say, for example, that in addition to my website, I also want to host a database. Maybe for my own purposes. Right. And it's not even public. Maybe it's just something that I use personally. But I want to be able to access it, say, from my home computer. This is me. I want to be able to get access to the database. The trouble is, now, how can my machine, my computer, know when a request comes in to my IP address? Right. To my computer. Whether the incoming request should be handled by my web server or by my database. That's where ports come in. Okay. So ports are virtual little hubs managed by the operating system. That allow us to segment incoming requests and incoming data streams. So now, if I have one host for my database. Sorry, one port for my database. And one port for my web server. And I can direct all my web traffic at that port for my web server. Let's just say that's port 80. And I can access my database on port, say, five, four, two. Those are pretty common ports for those use cases. So with ports, I can now actually run many different instances of different kinds of software. And network with them all at the same time. When I'm developing on my own machine, maybe I'm building a few different websites. I can actually run all of those websites connected to different ports at the same time. You can't bind two pieces of software to the same port. Again, for the reason we talked about at the beginning, right? The operating system wouldn't know which piece of software should handle which incoming network request. But we can run many pieces of software all bound to different ports. And your operating system actually allows for over 65,000 different ports at the same time. So there's not too much of a risk of running out of ports. So there are a ton of different ports that we can use when we are networking. And when we are navigating the web, we can optionally kind of type in the port, the URL, or we cannot. And if we don't, a default port will be used. So the question is, if a port is not present in a URL, the default is used based on the blank. The answers are domain, username, protocol, or domain name service. And the answer here is protocol. The default for the HTTP protocol is port 80, and the default for the HTTPS protocol is 443. The next question is a true-false question. And it says, only ports 80 and 443 can be used for HTTP and HTTPS traffic. That is false, right? Those are the defaults, but other ports could still be used optionally. I installed caddy on Mac by just typing brew install caddy. But if you're on a different operating system, head over to caddyserver.com. Just Google caddy, and I'm sure you'll find it. And you can find instructions for how to install it on your own machine. Once you have it installed, you're going to need to be on your command line in the same directory that has all these files we just created. I'm going to run the test directory here on my command line, and I'll run caddy file-server. Now, by default, the server is going to bind to port 80, which is the default HTTP port. And it's going to use localhost, which is just the domain of my own computer, right? So I'm connecting to my own computer on port 80. So when I go to localhost port 80 in my browser, which again is the default, so I don't really need to type port 80, I should get this rendered view of index.html. So you might be wondering how did caddy know to serve index.html, right? Why didn't it serve name.txt? Well, traditional file servers will typically route the path of a directory, which in this case would just be the root, which is slash, to a file called index.html in that directory, just by convention. For example, if I wanted to get name.txt, if I just wanted to get that text file, I could actually just go directly to name.txt, and I'd get that file. Caddy is taking the path slash name.txt and mapping it to slash name.txt on my file system. Hopefully that makes sense. Similarly, if I wanted to see this blog page, I could go to slash blog. And again, because blog is a directory, caddy is automatically redirecting that request from blog to blog index.html. If I were to type index.html, I'd get the same thing. So at this point, you may be thinking, well, that's great. So a URL path is just the same thing as a file system path once you've hooked up a web server. It's not exactly true. By convention, many static file servers just use the path to the file on your disk as the path in the URL. But it doesn't have to work that way. At the end of the day, the server software, in this case Caddy, can do whatever it wants. We could completely change how we parse the path of a URL and what we want to respond with. Many frameworks, back-end web frameworks, work this way. If you write a server in Go or in Django, there will often be a lot of custom logic that determines how different paths in a URL are handled and what files and data will be responded with when certain paths are accessed. So while it's common for a URL path to map directly to a file on a file system when we're talking about websites, when we're talking about web APIs, that's rarely the case. So in this assignment, what we're doing is getting some data about locations in the FantasyQuest video game. And up until this point, we've really only got, I think, like item data and user data. So now we need to update our path to get locations data instead. Essentially, the API is parsing the path and determining, based on the path, what kind of information we want about the game so that our front-end code can then, you know, maybe render the locations on a map or something like that. Cool. So this code is making a get request, a fetch request, right, to this URL. And the problem is right now we're reaching out to the user's endpoints, the user's path, and we're supposed to use the locations path. So really quickly, let's just take a look at what this code is actually doing. It looks like after making the request response, we're going to log the locations. And the log locations function is really just iterating over those locations and logging out a message where it's expecting a name field and a recommended level. This would be the name of the location and the level your character should probably be if you're going to complete quests in that location, for example. Let's go ahead and run it right now, hitting the user's endpoint and see what happens. It looks like we're getting undefined for location.name and location.recommendedLevel, which makes a lot of sense because we're probably not even getting location data back, right? We're getting user data. So let's go ahead and hit the locations. I believe it was called locations endpoint instead. Awesome. And now we are getting what looks to be correct data. I'm going to go ahead and submit that. Let's talk about query parameters. So while the path of an HTTP request typically changes something big, like the web page that you're looking at or the entire resource that you're requesting from the server, right? User data versus location data. A query parameter typically changes something very small about the request. Maybe some metadata, maybe a filtering option, something like that. At the end of the day, the server can do whatever it wants with the path or the query parameters, but kind of by convention, the query parameters tends to be smaller changes to the request. Maybe something like the options are changing, right? So, for example, we could look at how Google uses query parameters. So if you go to google.com, in fact, let's just go ahead and do that in the browser. Go to google.com, we are on the search page, right? If we search for hello world, we'll get a page with many different search results for the hello world query. And if we look in the URL up here at the top, we can actually see that that first query parameter is q equals hello plus world. So Google is actually taking our search and using it in the URL of this results page as the query parameter, right? So we could actually navigate directly to the hello world query by copying just that section of the URL, right? With the q equals hello plus world, and we'd go directly to those search results. So again, we're kind of on the same page. We're still on the Google home page where we can search for things, but they're using the query parameter to specify what term is being searched for. So let's break down this URL that we were just on, right? HTTPS, it's the protocol, www.google.com, the domain, slash search. It's the path, right? And Google is kind of using slash search as the results page and then using the query parameter, which is this question mark q equals hello plus world to specify which term is being searched for. If you think about it, this makes a lot of sense. There's an infinite number of things we could search for on Google. So using a path might not make a ton of sense, right? We don't really have a separate or new page for every possible search query, right? So they have one page, the search page, and they just change the query parameter based on the term that's being searched for. So this syntax, the question mark, separates the rest of the URL from the query parameters. And then we have key value pairs. So in this case, the key is q, the value is hello world. There could be more key value pairs after the q equals hello world, and they would be separated by ampersands. So an ampersand is the, if you look at your keyboard, it's the shift, shift version of seven, right? Hold shift, press seven, you get an ampersand. So we could do q equals hello plus world ampersand, maybe filter equals seven or something, right? We could just do key value pairs in the rest of the URL. We'll get to that more in detail later. So the question is, query parameters are required in a URL. That is false. Okay, so this next question is in a website, query parameters rarely change blank. The answers are which page is served, marketing and analytics information, or the contents of the page. So we talked about how query parameters typically don't change something big like the resource that's being requested or the web page itself. They typically change something smaller like options or contents, right? So for the example of Google, the page didn't change, we were still on the results page when we changed the query parameter. But the search term that we were looking for results for did change when the query parameter changed. So it's definitely not contents of the page, right? Additionally, marketing and analytics information, I think we can rule that out. Query parameters are very often used to do like affiliate link tracking, commission tracking, that sort of thing. Which page is served though, that rarely changes when the query parameters changes. Usually the page is dependent upon the path component. So I'm going to go with which page is served. Let's talk about synchronous and asynchronous code. To start, let's jump right into synchronous code. So these three console log statements are a great example of synchronous code, right? Synchronous code is simple. It's what we usually work with and we do it for good reason. It's simple and predictable, right? If I were to run this code here, I'd be printing first, I print first. Next, I would log, I print second. And next, I would log, I print third, right? We call this synchronous code because it's just happening in the order that each kind of line of code is called. There's no fuzzy timings or anything happening. It's very predictable, very straightforward. The other kind of processing we could do is asynchronous processing. Asynchronous processing is a little bit more complex, which is why we only use it when we need to. But in a nutshell, asynchronous processing allows us to do multiple things at the same time. So for example, if we need to render something in the browser, but at the same time we want to go fetch some data from a remote server by an HTTP call, we can do that asynchronously so that the browser can do its work while we're also waiting on a server to send us back some information. Let's head over to the JavaScript Playground to see what that looks like. Okay, let's write some simple code. We're going to use the same example from before. In fact, I'm going to go ahead and copy paste the example we had before. So if we run this, we just see I print first, second, and I print third. Again, this is very simple and kind of what we would expect. Now let's do something a little more interesting. We're going to use the setTimeout function. This is a built-in JavaScript. What it takes as the first parameter is a callback, which is just a function. So it takes a function as input. And then the second input is how long it should wait to execute that function. And it takes it in milliseconds. So in fact, let's go ahead and define that. We're going to use our waitTime to wait time milliseconds. Let's just wait 100 milliseconds. So we're going to wait 100 milliseconds to call the callback function. Now we need to define what the callback function actually is. Let's go ahead and do that. We're going to use our const callback, a little anonymous here. Sorry, fat arrow function. And inside that function, we'll just log I print second. So what do we expect to happen here? Well, first, we expect to log I print first. Then we expect to wait 100 milliseconds and log I print second. And then log I print third, right? Something like that. See if that works. In fact, I'm going to move this up. And this is just a function definition. Go ahead and run that. OK, interestingly enough, we didn't even get the I print second message at all. It didn't even print. And the reason for that is this last function call, the console.log statement that prints I print third, when that exits, our whole program exits. So there was no time to wait 100 milliseconds to call I print second. So let's go ahead and wait. Let's make sure our program doesn't exit too quickly. So this function here, I've just named it sleep. And we won't get into the details of this right now. We'll get into promises very shortly. But in a nutshell, this is going to allow me to do this await call here where I can await, sleep, and I can sleep for that amount of time, plus a few extra milliseconds. So now, in theory, this line here will stop our program from exiting for 105 milliseconds. OK, so now we can see what's really happening. OK, cool. So it printed first, third, and then second, right? Again, because this one was waiting. So let's update the logs. Let's just say this one prints second. And this one will say I print after that many wait time milliseconds, milliseconds. Now this should be more accurate. Cool. So we got I print first, I print second, and I print after 100 milliseconds. Again, because we're waiting, right? We're telling the JavaScript runtime, hey, I'm going to do some other stuff for a minute. In 100 milliseconds, can you please execute this other code? So we're doing that asynchronously. We are simultaneously printing, I print second, and telling the JavaScript engine, hey, queue up this other operation that will happen in the future. Let's see what happens if I just increase this to a full second. 1000 milliseconds is a full second. We should be able to see it pop up there, OK? Hopefully that helps you understand just a little bit about how set timeout works, right? And how asynchronous programming works from a high level. So to tie that back into this diagram, the first example that we used, right? The I print first, I print second, I print third. That's an example of synchronous programming. If you look here in the diagram, you'll see that although we have maybe two processes, only one thing is happening at a time, right? Only one console.log statement is being printed at a time. They're being printed in order. It's very simple. Asynchronous programming, on the other hand, is where we can have one process doing some work and the other process is doing work at the same time. So how does this tie into our FantasyQuest server? Well, we have our code running in the browser in JavaScript, right? That's process A. And we have some execution logic, some program running on the server. That's process B. And with asynchronous programming, we can ensure that both processes are running at the same time and we're able to kind of sync them up as we need to. So as far as the assignment goes for this exercise, what we're told is to help us visualize how asynchronous code executes, let's practice with an example from the game. Keep in mind that we probably wouldn't use setTimeout in this way, but I think it's important for us to practice this anyways, even though it's not really a realistic example. We'll get to more realistic examples in a minute, just so that we can really get a feel for how the setTimeout function works. So the assignment says, update the waiting durations so that the text is printed in the following order. So again, if you remember, setTimeout is a function that takes a function as input and waits a certain number of milliseconds before starting that function, right? So setTimeout is kind of called and asynchronously, it will go spawn that function in the future after some number of milliseconds. Okay, cool. So we want to print in the following order, first firing up the forge, then smelting iron bars, then combining materials, then shaping iron, and then iron longsword complete. And it looks like up here we have some waiting durations. We are not supposed to change anything below this line. So all we can do is change these durations. So firing up the forge, that should happen first, right? Because it's the only thing not being kind of deferred for later. So the next thing we want to print is smelting iron bars. So let's go ahead and set that to, well, it looks like down here we are waiting for 2.5 seconds total. So we need to keep this all within two seconds, right? Let's wait 500 milliseconds for the, what is it, smelting iron bars. So smelting iron bars wait, so let's wait 500 milliseconds on that. And then combining materials, so let's wait a full second on this one. Next we've got shaping iron, so we can wait 1.5 seconds, or 1500 milliseconds. And then we've got iron longsword complete. So we can do a full two seconds on that one. Cool, so just by changing the amount of time we're waiting for each log statement, we should get them in the right order. Let's go ahead and run that. I'm going to check to make sure we did it right. That looks good to me. So as we mentioned before, async code, we really should only use it if we need it. But sometimes we really do need it. Essentially every time we make an HTTP request over the network, we need to do that asynchronously. Because if we didn't, the device that we were making the request from would need to just freeze up and wait for that request to come back. So we have to asynchronously wait for the request to come back. I mean, think about it. The server that you're communicating with is probably in another state or another country, right? So it may take, you know, whatever the ping of your internet connection to that data center is, it's going to take that amount of time, which again can be, you know, 200 milliseconds. It could be a full second. We can't really be sure. So we need asynchronous code so that we can keep doing things in the meantime, right? OK, so the question for this quiz is sync is better than async. And that's a tricky one. I'm going to go with false because in some cases we need async. Async is better than sync. I'm just going to say false again, right? It depends on the situation. We prefer synchronous code when possible, but sometimes you just can't get away from it. Another quiz, async code should blank. Answers are always be used, only be used for performance reasons, never be used, only be used on fast computers. Generally speaking, it should only be used for performance reasons, right? If we need to use asynchronous code to speed up a process, that's a great reason to use asynchronous code. Great. Now we can finally talk about promises. So setTimeout is a built in function to JavaScript that does some asynchronous stuff behind the scenes. But promises are kind of the most modern way to write asynchronous code in JavaScript. And the best way to think about a promise in JavaScript is like a promise in the real world. For example, if I were to promise to you that I'm going to explain how promises work, there's only two potential outcomes for that promise. Either I keep my promise and I teach you, or I don't keep my promise. Those are the two options. And just like those two options in the real world, with JavaScript, a promise can either be accepted or rejected. So in JavaScript, there's actually an object called the promise object. I can head over to the docs on MDN. Again, this is built into the JavaScript language. And a promise object basically just has three states, right? We talked about fulfilled, which means the promise was essentially kept. Rejected, meaning the promise was not kept or the operation failed. Or pending, right? We're still waiting on the outcome of the promise or on the operation to finish. So here's the syntax for how we can declare a new promise or create a new promise in JavaScript. Use the new keyword to create a new instance of the promise. Then we use the promise constructor. It takes as input, as its parameter, a function. This function here. The function has its first parameter, a resolve function, and the second parameter, a reject function. And then basically in the body of the promise, we're going to call resolve when we've successfully finished the operation we care about. Or we'll call reject if for some reason something went wrong with the operation. So in this example here, we've got a promise. When it starts, it will wait 1000 milliseconds. And then randomly, it looks like there is a 50% chance here that the promise will resolve with the string resolved or reject with the string rejected. Okay, so really all this promise says is in one second from now, I have a 50% chance of resolving the 50% chance of rejecting all we're really seeing here. Okay, using the promise. So that's how we declared the promise. Now we want to actually use it. Now that we've created a promise, how do we use it? The promise object has two callback functions that you can attach to it. One is the dot then the other is the dot catch. Okay, so dot then the function that is passed into dot then runs after the promise is successfully completed right once the promise is fulfilled. So it would run. If this line executes right if this resolve line execute. The dot catch function will run if the promise rejects. In this case. And then here in this example, we're basically just saying okay so wait for the promise to finish. So again, in the case of our example, wait the 1000 milliseconds, and then if it was successful, print the promise finally blank and it's going to use that string it's going to use that parameter that was passed into the resolve function so it would print the promise finally resolved. Or in the case of something going wrong, it would print the promise finally rejected. Right. Cool. Do that down here. Okay, hopefully that all made sense but if it didn't that's okay this was just an explanation it will all make more sense once we write some code. So, the assignment says, complete the apply damage function so that's this function here that takes two parameters so it takes damage and current HP or current hit points as inputs. And it says it should return a promise after one second, because in fantasy quest dealing damage with an attack takes some time. Okay, if the damage inflicted would reduce the players HP to zero or less the promise will reject with this string. Otherwise, the promise will resolve with this string. Okay, cool. So, we are creating a new promise that part looks good. We've got our resolve and reject callbacks here. We have set timeout called one second so that looks correct right we're going to wait 1000 milliseconds before doing whatever is in the body of this set timeout. So, we need to do some calculations, if the damage inflicted would reduce the players HP to zero or less. So, if damage is greater than grant HP Oh zero or less so greater than or equal to. If that's the case. The promise will reject with the string that okay so that. And then we're going to eat the string. Cool, reject the player suffers. Damage points of damage as. Fallen unconscious okay that looks good to me. Otherwise, we're going to resolve the promise with the string. The player suffers blank points of damage and has some amount of hit point. Okay, so we actually need to calculate new HP, new HP const new HP. Bulls going to be the current HP minus the damage right. In fact, now we can simplify this and we can just say if new HP is less than or equal to zero. And then over here, we can say, new HP that looks good damage points of damage. Okay, cool. Let's go ahead and run that. See what happened so applying 27 damage to a player with 50 HP. Apply damage resolved with the player suffers 27 points damage and has 23 hit points remaining that math looks good to me. And again, you can see how it's it's going over time right it's it's using that full second to apply the damage and again this makes sense in the context of a video game. Right. Sometimes, when a character attacks, there's like a second long animation right maybe a fireball has to fly across the screen or a sword has to actually be swung, there's an element of time to these programs. That's where asynchronous or at least time based programming can be really useful. I'm going to go ahead and submit that. So let's talk a little bit more about why promises are useful in the real world. So promises are the cleanest, but not necessarily the only way to write asynchronous code in JavaScript. In fact, the fetch function so you if you if you remember some of the code snippets that I said we get to later earlier in the course that fetch function actually returns a promise. And that's how we're able to do all that asynchronous kind of HTTP programming that we've been doing earlier and that we'll do more of in a couple chapters from now. So, when do we use promises and when do we use asynchronous code well the most common time especially in front end development is whenever we're doing IO or input output. So that's basically whenever we're making a network request right whenever we want to interact with a remote server, another machine will be doing asynchronous programming. Sometimes I owed isn't even just over a network sometimes we need to go to the hard disk of a computer right to the file system and read a file. Very often, especially in JavaScript that is done asynchronously again so that the rest of the program doesn't have to sit around and wait for your for your program to physically go grab that data from the hard drive of the machine. So, yeah, common examples of IO include HTTP requests reading files from hard drives or interacting with Bluetooth devices. Right. It's almost like it's not a perfect analogy but when you're interacting with like the real physical world and not just within the software of your own program, it's likely you'll need to do something asynchronously to keep everything running smoothly. So the question is, promises are the only way to write async JavaScript, and then the answers are true and false. Promises are not the only way to write async JavaScript, there are ways. Remember promises are just the most popular way, or the cleanest way. I found that it can be really useful as a developer to have kind of a ballpark idea of how long certain kinds of IO operations will take. And as you can imagine that's really useful when you're working with HTTP requests, for example, we'll start with the most basic type of kind of IO operation. In fact, it's so basic that we often don't even think about it as IO, and it's getting data out of RAM. So in your code, whenever you create a variable, there's a good chance that the data inside that variable, right, something as simple as name equals lane, right, the string lane, that string will very likely be stored in RAM in your computer. And whenever you want to access that variable, your computer, your CPU actually has to go fetch that data from your RAM. And that usually happens on the order of nanoseconds. It's very, very fast. In fact, it's practically imperceptible to humans, right, it's so fast. And so for all intents and purposes, we can just do it synchronously, we can synchronously, you know, update variables, create variables, delete variables, all that kind of stuff, without worrying about doing any of these operations asynchronously. Cool. So that's really, really fast. Next, we have disk. So this would be like going to the file system, right, and reading in, say, a text file, maybe doing some analysis on it. For example, programs that have to read in configuration data might have to fetch some data from disk. Fetching data from disk takes on the order of like one millisecond. One thousandth of a second. In fact, it's often much faster than this, especially if you're using in a solid state drive. But it's slow enough that in some cases, it can be problematic to do it synchronously, especially if you're doing a lot of reading from disk or you're reading large amounts of data from disk. It can be useful to do it asynchronously so the rest of your program can continue kind of functioning as normal. For example, imagine you're using VS code, your text editor, and you need to load a very large file from disk. Well, it would really suck if your entire VS code completely froze until the file was loaded. So very often these sorts of things are done asynchronously. So I'm just going to put async or sync. Not great. Cool. And then kind of the last category that I think is important to have top of mind is the network. So this would be things like HTTP requests. And obviously it's going to be much slower if you're making the HTTP request, you know, to a computer in another building, even slower if it's a computer in another state, even slower if it's a computer across the world. So it's going to change a little bit. But generally speaking, if you're working on the internet with a fairly robust backend server that's maybe in a data center somewhere near you, we're talking about something like 100 milliseconds to maybe 2000 milliseconds, which is two seconds. And because this is quite slow, it's definitely noticeable by humans, right? We typically work over the network asynchronously. Because it's really inefficient to have our programs pausing for seconds at a time, just waiting for bits to be sent halfway across the world and to come back. Hopefully this gives you a decent breakdown idea how some of these IO timings will fit into your applications. Next question is, which would be considered IO or input output? JavaScript programming language and HTTP request are really slow algorithm or a function that adds two numbers together. Well, the JavaScript programming language doesn't even make sense. So I'm going to ignore that. A slow algorithm, not necessarily, right? Like most algorithms will just run again in your program, right on your CPU. You don't necessarily need to fetch data in order to make that happen. Same with a function that adds two numbers together. That sounds like a pure function that doesn't need to request any data from the outside world. So it's going to be an HTTP request. So we've already interacted with promises in their most raw form. That is, we've used the dot then and the dot catch methods to provide callbacks that are executed once the promise either resolves or rejects. Well, the await keyword is actually a kind of a newer syntax for interacting with promises that just it looks a little different. And I would argue that in most cases, you should prefer the await keyword over the dot then syntax, just because it's easier to work with and a little easier to read. So let's take a look at the difference. So here we have a promise, right, that, you know, at some point in the future is either going to resolve or reject. With the dot then syntax, we are passing in that callback function, which takes as input the results of the promise. In this case is just a message. With await, instead of a callback function, we just use the await keyword and provided the promise. And then what happens is this expression does not evaluate until the promise resolves. So our code is actually going to block at this point, it's going to sit and wait until the promise resolves, at which point it would, you know, save the value, the return value, right, the resolved value of the promise. Into that message variable, and we could go ahead and log it to the screen. This doesn't mean that the entire JavaScript engine or JavaScript program is going to stop and wait. It just means that this specific part of your code, right, this specific function is going to wait. So this allows us to write asynchronous code so that it looks like it's synchronous, which, again, just makes it so much easier to work with. So let's jump into the assignment. It says similar to before, the apply damage function takes a damage value and the player's current HP as inputs and returns a promise. On line one, call apply damage with inputs, damages 25, current HP is 50, then await the returned promise and save the resolved value and a variable called message so that it can be logged to the console. OK, so that was that was long. Let's let's take a look at the code. So here we have the apply damage function. It's responsible for creating the promise. So this is really very similar to that function we wrote in the last assignment. But now we are going to be interacting with the promise. So what we need to do is call apply damage. We need to provide damage as 25, current HP is 50, right. I'm just getting this from the assignment description over here. Now, because a promise is returned, right, return new promise, we need to await the results. So we need to await that promise. So this will evaluate to a promise, right, just like up here. And then we need to await that promise. And it says the return promise, save the resolved value in a variable called message. OK, so const message. Cool. And then down here, that message should be logged to the console. Let's take a look at that. 50 HP suffers 25 points of damage and has 25 HP remaining. I'm just going to test it really quick. What happens if we bump this up to 70? 70 HP, 25, 45. That looks great to me. So I'm going to change that back to what the assignment asked for and submit it. So if the await keyword replaces the dot then syntax, the async keyword effectively replaces the new promise syntax. It's a way to create a promise, basically. And the way it works is if you specify that a function is asynchronous, it will automatically return a new promise and the resolved value will just be whatever value was returned from the promise. So let's take a look at that in code. So here we have the traditional way of doing it with the new promise syntax. We create a function and then we return a new promise. And that, you know, resolves some value at some point. Well, with async, all we have to do is put that async keyword in front of the function keyword. And now this function get promise for user data will automatically return a new promise. And the value that that promise resolves to is just whatever is returned from the function. Additionally, the await keyword can be used within asynchronous functions. In fact, it can only be used within asynchronous functions. So async and await allow us to do, you know, tons of different asynchronous operations while writing very clean code that frankly looks synchronous or at least allows us to think about the code in a more synchronous way. So let's get down to the assignment. It says, as it turns out, the await keyword can only be used within async functions like we just mentioned. It says, go ahead and try to run the code. You'll get an error. Expand this a little bit so we can see. We'll run the code and we get a syntax error that says await is only valid in async functions and the top level bodies of modules. So if we look at the code itself, it looks like awaits being called here. That should be OK because the top level body of the module. And then await is also called here, which is inside a function that is not asynchronous. And that is the problem. So what's going on? This function is supposed to be fetching some data over the network, right? Making an HTTP call. And then it looks like it's parsing that response data as a JavaScript object. And both of these operations are asynchronous. But the function itself is not. That's why we're getting this error. So let's go ahead and mark that as an async function, which just means it's going to wrap the entire function in a promise. And when the promise resolves, it will resolve with that response data. Cool. So that looks like it worked. We got all the item data coming back. Go ahead and submit that. So we've used this word callback function a few times. And all we mean by that is a function that we hand to another function to be executed at a certain point in time. So it might be, for example, a function that is called when a button is clicked. It could be a callback function. Or if we look here in the example of set timeout, the callback function is the function that we give to set timeout to be executed later. So the promise API in its original form was more callback based, right? We have this dot then syntax or dot catch syntax that took a function as input to be executed once the promise successfully completed. Async and await these newer keywords kind of do away with that callback syntax. And in my opinion and the opinion of many others, it's just much easier to read this way. Let's take a look at this example. So here we have some code using dot then. And here we have some code using await. So here the dot then functions have been chained one after the other. Right. First, we're going to fetch the user. Once that happens, we'll fetch the user's location. So that happens. We'll fetch the server. Right. And then we're able to finally log that data. So three nested dot then callbacks. Right. And then down here, we're using the await keyword. This looks a lot more like kind of traditional synchronous programming that we're used to. Right. First, we'll fetch the user and we'll wait for that operation to complete. Now we have a user object. Then we fetch that user's location. Wait for it. Then fetch the server for the location. Wait for it. Then we're finally able to log result. Again, both of these approaches are valid. And there are certainly cases in which it might make sense to use dot then over await. But generally speaking, I, I prefer to use the async and the await syntax. So the question is blank is generally cleaner syntax async await new promise or it doesn't matter. Use what you feel like. I would say generally speaking, async and await is cleaner. Now we need to talk about error handling in JavaScript. So this is really, really important when it comes to working with HTTP or networks in general. A lot of things can go wrong on networks. For example, if you're running some JavaScript code on your computer and all of a sudden you lose Internet connection, there's not necessarily anything wrong with your code. Right. There's no bug, but there's no way you're going to be able to fetch data from a remote server. You don't have an Internet connection. So error handling allows us to detect that something's wrong and handle it gracefully. So let's take a look at this example. Here we have a car object and we expect a speed property to be defined on that car object. But when we run this code, if the car object has not yet been defined, we'll get this reference error, right? Reference error, car is not defined. Now, for the sake of this example, let's imagine that we don't have control over this car object. Maybe it's coming to us from an external library. Well, what we can do is we can wrap it in a try block. Now, anything within the try block, if it produces an error, we can catch that error and handle it gracefully. In other words, as code within the try block is executing, if at any point an error is thrown, the try block will stop and we'll jump down to that catch block where we can handle the error. So moving down to the assignment so we can see this in action says the code is failing to execute. It is throwing an error that isn't being handled gracefully. As a result, the entire program is crashing. Properly handle the error in the code by wrapping all of the function calls within a try catch block. The catch block should log any errors to the console. Okay, so if I run it in its current state, we're getting an error that says parameter is not a number. So it looks like we have a print character stats function that takes a number as input. And if the number that's passed in or rather I should say if the data passed in is not a number, which is what this function here does, says is not a number, then this error is thrown. Parameter is not a number. Otherwise, we'll log to the console, your character is whatever level. So this call, right, number two, this isn't a number, this is a string. But according to the assignment, we are essentially writing a test suite here. So we want to make sure that our function is properly catching not a number values and throwing an error. So rather than just erroring out, we should be handling it. Go ahead and write a try block, you'll dump all of these in here. Catch block with an error. So this error value, this is going to be whatever was thrown. So in this case, it's possible that a string is thrown. So we're expecting this to be a string. And our assignment just says the catch block should log any errors to the console. So we can just do console.log. Okay. So remember, the try block will just keep executing code, you know, one line after the other, just like normal, until something bad happens. When something bad happens, it will jump right down to the catch block. So I would expect because we were told to wrap all of the function calls within a single try catch block, that we actually don't want this line to execute at all, right, in this test suite. We're making sure that this bad call to print character stats properly, essentially stops execution. Let's go ahead and run that. So your character is level four, and then parameter is not a number. That looks good to me. Let's talk about the difference between bugs, bugs and errors in code. So bugs and errors are not the same. When I first started writing code, I was kind of fuzzy on this whole idea. And to be honest, I think for a long time, I assumed they were just kind of the same thing. In reality, bugs are bad, as you probably know. The interesting thing is that errors are not always bad, not always bad. Okay, as we've defined them, bugs are kind of unexpected things that are happening in our code. So we want our code to do something, right, hopefully something useful. And if it's not doing that thing, then we say we have a bug in our code. So bugs are things that are unexpected. To give an example, let's say that we had a function that we had a function in our code that adds two numbers, something like add, and it takes x and y as inputs, and it produces some result. We'll just call it z. If we called add with two and three, and we got the result of seven, we would say we have a bug, right? We expected the add function when given two and three to produce a five, but we are getting a seven instead. That's a bug. It's something that's unexpected in our code. An error, on the other hand, is very often expected, expected. And our job as programmers is to handle that error gracefully. So the way we approach fixing bugs and errors is quite different, right? When we have a bug, really, the only thing we can do is go fix the code, right? We actually have to go fix our application. Now, to be fair, there are certain circumstances where fixing a bug might be too expensive or maybe the bug is there, but it's actually not affecting many of the users, so we can kind of ignore it for now. That's more of a business and cost decision, right? In a perfect world with unlimited resources, of course, we would always fix all bugs. But the point that I'm trying to get across is when there's a bug, really, all you can do is fix the code. Errors, on the other hand, sometimes are unavoidable. You can't actually fix the code to remove errors. So, for example, an error that you might get in your code is there is no internet, no internet connection. So if you're writing a web app that needs to communicate over the internet to get its job done and the user's computer has been disconnected from the internet, there's really nothing in your code that you can do to fix that, right? You can't magically bring back an internet connection. So all you can really do is handle that error gracefully, right? Maybe by showing a message to the user. Another common error might be something like the remote, the remote server. Server is down. So you're writing some code on one machine that, again, maybe reaches out over a network to a server or a database. And while you have internet connection, maybe that server is offline. So that's another common type of error that crops up that, again, nothing's wrong with your code, but it's something that you need to be able to handle and maybe display to the user, something like that. Another common error would be user input. Bad user input, right? So maybe you're writing an application that requires users to have passwords over 12 characters, right? When a user puts in a 10-character password, we might consider that an error. Again, it doesn't mean there's anything wrong with our code, but we need to handle that error and maybe display some message to the user. One final note on the distinction between bugs and errors is that bugs have to be fixed in code. Errors have to be handled in code, right? So we expect errors. We must handle them. If we don't handle our errors properly, they may result in bugs. So like we just talked about, bugs and errors are not the same, right? Debugging is the process by which we remove bugs from the code, right? And then error handling is how we handle errors in our code. They're two very different and distinct things. Try-catch blocks are not a debugging tool. They're not ways to find bugs in our code, right? They're a way to handle errors at runtime, things like the internet connection being down. The whole takeaway from this is do not try to use try-catch for debugging and do not try to use a debugger to introduce error handling in your application. Understand those are two very different things. Okay, so the question that goes along with this is the presence of which of the following indicates that the developer has something to fix. Bugs, errors. Well, errors are always going to be present. Unless you have the most simple app, right, that doesn't do anything with a network or anything with user input, there will always be some sort of error handling. So I'm going to go with bugs. Next question is you should use error handling techniques to fix bugs. No, you should use a debugger or debugging techniques, right? Log statements, print statements, things like that to find bugs and remove them from your code. Error handling is to handle errors at runtime. So that's going to be false. This next question is a debugger is required to debug a program and the answers are false and true. This is something I want to talk about for just a second. So. Debuggers are optional. Debuggers are tools that we can use to help us debug our code. Some people like them. Some people don't. Some people use them for certain things or not for other things. If you've ever used a debugger, you'll know what I'm talking about. But basically a debugger is something that allows you to kind of step through your code one line at a time. So instead of running your entire program at once, you run it in debug mode, which allows you to kind of step through the program. Just one line of execution at a time. So you can see like what's going wrong. Where a more primitive form of debugging is to just add, you know, console dot log statements or print statements. So you can see what data is present in your application at which points. How you debug is up to you. Some people like debuggers. Some people don't. To be honest, I think it depends a lot on the kind of code you're working on as to whether you should use a debugger or not. But at the end of the day, they are not they're not required. So I'm going to go with false on this one. So now that we've talked a little bit about error handling, we can talk about how async and await that other syntax for working with promises makes error handling a little nicer. So let's take a look at how it works in the non async await world and just kind of the raw promise usage world. The way it works is if everything goes well with a promise, then the dot then handler resolves the promise, right? We get some value, in this case, a user object that we can then, you know, do something with. If something goes wrong in the promise. So, again, let's just assume an HTTP call, something goes wrong, Internet connections down, servers down, then instead of the dot then handler firing the dot catch handler fires instead. OK. And this is the syntax for that. With async await, we can use kind of normal JavaScript error handling. We can use try catch blocks. So one of the great things about using async await is we get kind of this just more traditional JavaScript feel, right? Look and feel or style to our app. So the way this works is when we are awaiting a promise, if that promise rejects, right? In other words, if something goes wrong in that promise, then the error is thrown so we can just catch it in a catch block. OK. So, again, rejecting promises results in essentially thrown errors. And if we throw within an async function, that's essentially the same thing as rejecting the promise. So, again, it kind of just remaps everything into normal JavaScript, which is why I really, really like the syntax. So let's see what that looks like in action. The assignment says we're trying to fetch the worldwide leaderboard from the fantasy quest server. So we're writing the game code, right? We're interacting with the fantasy quest server. Unfortunately, we're getting an error. The good news is there's actually nothing wrong with our code. The reason we're getting an error is because the server is down. So as good software engineers, instead of just allowing the error to kind of bubble up and explode, right, which in essence terminates the program, unhandled JavaScript errors can just kill execution. We need to actually just handle that error and display a message to our users through the console. So our task is to wrap this within a try catch block. So we'll do try. And again, the reason we can do this is because this asynchronous function is awaited. So any errors thrown within can be caught in just a normal try catch block. The assignment says wrap the network call within a try catch block. Within the catch block, log the text. OK, let's do console.log. Our servers are down, but we will be up and running soon. Cool. In this case, we don't even need the specific error. We're just going to kind of give our users a canned response. On that note, this console.log, in case this wasn't clear before, will not happen if something goes wrong here. If something goes wrong on line two, we're going to jump immediately down into the catch block. So we will only log the leaderboard data if the server properly responds with the leaderboard. Go ahead and run that. Great. And this looks good because this is supposedly the URL of a server and it is supposedly down. So everything seems to be working as intended. Submit it. It's time to work with some HTTP headers. So headers allow clients and servers to pass additional information about an HTTP request. So up to this point, all we've really cared about is the actual body of an HTTP request. And the body of the HTTP request is typically used to transport the information itself. I'm sorry, I don't know a better way to say it. We'll get into some examples that will really help. But metadata, which is what headers are typically used for, is like data about the data. So for example, let's say that the body of an HTTP request maybe contains item data in a video game, right? Things like iron sword, bronze sword, wooden arrow, whatever. The headers might contain things about that data. For example, the type of client the person sending the request to the server is on. So for example, maybe Google Chrome, right? Or the operating system or the user's preferred language. This is kind of data about the data, right? It's not the information we really care about, which would be included in the body. It's additional information that the server can use to figure certain things out. Again, things like location are very often transmitted in headers. Maybe you have a website that changes the currency for the pricing page based on the location of the user. Headers would often be used to communicate that location. Let's jump into the assignment to really see kind of what the heck's going on. So the assignment says complete the log content type function. It takes a response object as input and should log out the content type header to the console. Okay, so we're taking a responses input. We're supposed to log something to the console to a response object, right? This is just what comes back from a fetch call. So it is the entire HTTP response object, right? As exposed to us kind of through the fetch API. Cool. So how do we get headers? Well, let's take a look. Here it says we can access the headers object through the request.headers and response.headers property. So in this case, we don't care about the request. That's already sent for us down here. We care about the response. We're logging the content type of the response. So response dot lowercase headers that this object holds all of the headers, right? And then it's telling us we should use this dot get method. So this is a method, again, exposed to us through the headers API, which means it's a built in function, right? To the browser. And this method, method of the headers interface returns a byte string of all the values of a header within a headers object with a given name. The requested header doesn't exist. We'll get null. OK, cool. So let's test that. Let's just do dot get garbage header. OK, so I don't expect that to return anything interesting. Let's just all this header about and let's go ahead and log it out to the console. So in theory, we'll get a null, right? Because I don't expect that this this key in the headers object actually has an associated value. OK, great. So that was null. So if we want to get the content type of the response now, content type. Oh, content type is actually a really common header in HTTP requests. And its whole purpose is to tell kind of the computer on the other end what type of data is being sent in the body. We can send JSON data, right, which is basically a string that looks like a JavaScript object. We can send HTML data, right, to be rendered in a browser. We can send CSS. We could send plain text, all sorts of kinds of data we could send or formats of data we can send. The content type header is designed to kind of communicate that here's some data and here's the type of data it is. So if we log the content type, I would expect probably that this is JSON data. Yeah, so we get the application slash JSON header in very, very common header value when you're just working with data. So I'm going to go ahead and run that. I think we got it right. Let's talk about browser dev tools. So I'm here in Google Chrome, but this will work in pretty much any popular browser. It just might be a little different. If I right click anywhere on the page and click inspect, my dev tools will pop up and I'm going to head over to the network tab. That's what's most interesting to us. The network tab records all network activity that's being done by the browser. Here we don't have any network activity, but if I reload the page, then we should see all these requests fire off. These are all the requests that are required basically to run the boot dev software. So there's tons of stuff in here. We've got JavaScript that we're loading. We've got, let's see, fetch requests. This is actually the JavaScript that's running on the boot dev website in the browser that's making additional HTTP requests from the browser back to kind of the boot dev server. So we got some of those. We've got, let's see, got an HTML document here. And if we actually click on that request, we can see the response headers and sure enough, the content type header is text HTML. So now that we've taken a peek, that's what we were supposed to do. I just kind of skipped reading and showed you. But the question is, you can use the developer tools to view responses to your clients' requests. And that is definitely true. So the next question is about how long do most of the network requests take? Keep in mind, this is the time it takes your browser to reach boot dev servers over the Internet. OK, so we're going to pull up the network tab again. Refresh the page. And if we look at these fetch requests, we can see how long they took in milliseconds right over here on the right. And it looks like most of them are taking, you know, somewhere between 50 and 500 milliseconds. It just depends, right? Some of them are much faster, right? 50 milliseconds here. Some of them are taking a little bit longer. I think we had one that even maybe took a full second. There it is. One whole second there on that one. But I would say kind of on average around 100 milliseconds. So let's see what the answers are. 5,000 milliseconds, that's much longer than they were taking. 100 seconds is way longer. None of them were as fast as two milliseconds. So I'm going to go with about 100 milliseconds. So we've talked about how headers are often metadata or data about the data. Another very common use case for headers is authentication, right? So take the boot dev website as an example on boot dev. Yes, we, you know, we write code, we submit assignments, but we also keep track of our progress, right? So we have achievements. We've got gems that we unlock levels and experience. A lot of that information is private, right? You can't necessarily, as a different boot dev user, see how many gems that I have. That information is kind of private to me. So whenever I make a request to the boot dev server, essentially asking how many gems do I have, I need to be able to prove to the server that I am who I say I am. And that's often done in headers. In fact, on the boot dev website, it is done with headers. Headers are a convenient mechanism for this because we don't need to kind of muddy up the actual request body itself with authentication information, right? The request itself can just be about gems. How many gems do I have? I get kind of a number back, right? All of the authentication information, that additional metadata can exist strictly within the headers. So let's jump into the assignment to kind of see an example of this. So because we don't want any of our users to accidentally overwrite another user's saved data, our back end team has required that we use the X API key header, which means that X API key is the key in the header, right? And then we would set a value for all requests to the fantasy quest servers. By using different values, we are telling the server that we are a different person. OK, so there's some headers being sent to the fantasy quest server. And what we're reading here is basically if we change the value in that X API key header, we're telling the server we are now a different person. If we use the same API key header, then the server is going to treat us as the same person. So some code has been written for us. Let's take a look at what it's doing. So here at the top, it looks like a new API key is being generated. So that's what we're going to include in the header, right, for authentication purposes. It looks like we are getting location data here, which is just a fetch, a fetch call to the server. So that's where we're getting kind of a single locations information. And that matches up with what's being talked about in the assignment. So it says, number one, get a location object from the game server. So that's happening right here, right? And we're using this API key to do it. Number two, log that object to the console. That's happening here just so that we can see the location. Next, updates the object and sends those changes back to the server. So put location. So that's this function here, which is defined down here. So it looks like this, this HTTP method is going to update the location on the server. And then number four, gets the location object again and logs to the console to display the changes. OK, so it looks like we're getting a location, making a change to it, putting it back, and then getting it again, essentially to make sure that it changed. Let's go ahead and run the code and see what we get. So got old location, Heartfin Marsh recommended level one, location updated, got new location Heartfin Marsh recommended level one. OK, yeah, so it's not changing, right? That's the problem. We go ahead and read the assignment. It says run the code in its current state to see the return values. You'll notice that the two objects that are logged to the console are the same. So because the X API key we're using is different, the update is not being applied to the same user account. OK, so we need to figure out why that's happening. So this new location data, it looks like this is what should be being updated on the server. So we should be getting a new name, Bloodstone Swamp, and a new recommended level of 10. So the problem is new generated API key. It looks like we're using the same API key to get the data in the first place and put it back. But we just need to use that same API key to get it the third time. And if we look at why that's important, let's go look at the get location response. We can see here that the API key variable that's being passed in is being added to the headers as a value. OK, cool. So this is now this is looking good. We've we've deleted that new API key and we'll just use that same original API key all the way through. Go ahead and run that. Cool. Now we have the updated one. I'm going to go ahead and submit that. OK, now we'll be practicing with the network tab just a little bit more. And by the way, if you do a lot of work on the Web, I recommend just poking around in the DevTools. Anyways, there's so much useful stuff in here. Again, in this course, we're really just going to be focusing on the network tab, but the DevTools are just they're just fantastic. OK, so I'm going to go ahead and refresh the page. The assignment said to open your DevTools, navigate to the network tab and refresh the page. Poke around through the request that you see. Notice that you can select a request and see its request and response headers. OK, so we're really just supposed to kind of poke around. And the question, so we're really looking to get out of this is question is header values are always numbers. And is that true or false? So let's take a look. Here's a fetch request. Let's take a look at it. Here's some headers, right? These are the headers in the request. These are the headers in the response. So again, that's just important to understand. We can send headers as the clients to the server, but the server can also send headers back in the response. And I mean, if you look at this, right, we've got status code of 200. We've got access control, allow headers, star. So clearly they are not all just numbers. We've we've definitely got strings in here. So I'm going to go with false on that one. All right, we've got another another practice with the DevTools. Wants us to take a look at some headers. I think I remember the answer to this one. So it says the status code header on the request have blank values. Let's just let's just open it up again and take a look. So here's the status codes, right, which are kind of you can think of status codes as a part from headers. But if you open them in the DevTools, you'll see they're also in the headers. So the status code is 200. So it's definitely a numeric value. One interesting thing to note, even though the answer here I believe is is going to be numeric in code when we're working with headers, we typically parse them as strings because that's what they are at the end of the day. That's how they're encoded. That said, you can obviously encode a number as a string. I'm just pointing this out because you may need to cast headers from strings to numbers if you want to deal with them as numbers. So anyhow, you go with numeric on that. OK, again, with the DevTools, this time the question is, what is one of the values you see in the response headers for the content type key? You might have to look at several requests. So we've seen the content type key before. We know that it deals with the kind of data like what kind of data is being sent in a response. So let's pull up a few examples. OK, so here we've got a fetch request. This one's content type is application JSON. Let's look at just a couple more. This is an image and its content type is image JPEG with one of these. OK, perfect. There's application JSON. Get post options, put delete. This wouldn't be a content type, right? Those are HTTP methods, which is something we'll get to soon. 921, that looks like it could almost be a status code, but it's actually too high. Status codes don't go over 599 or so. So application JSON it is. So it's finally time to talk about JSON or JavaScript Object Notation. So JSON is a standard for representing structured data on the web. So in the last assignment, or I should say in the entire last chapter, we were looking at how different network requests send different kinds of information in the request body. For example, sometimes it's image data, right? Just literally an image like as a JPEG or PNG. Sometimes it's an HTML document, right? And sometimes it's kind of more raw data. When we've been working with the boot dev API and we've been getting item data, right? Things like names of weapons, names of locations, their damage. We've been using JSON in order to transmit that data over the network. So let's take a look at some actual JSON data. Here is a JSON object that represents kind of a list of movies. So at the top level, there is a key called movies and its value is an array. And inside that array, we have some objects. So each object, it looks like, is intended to represent one single movie. We've got some information about each movie, right? It's ID, it's genre, it's title, director. So JSON data can really be structured in many different ways, right? We could have another key in here. We have movies in one array and we could have actors in another array under a different key. But the important point is that it's a very flexible way to kind of send structured data as plain text. So as developers, it's really simple to work with. We can take that plain text and parse it into our program and work with that data as just variables, right, within our program. So because JSON is plain text, if we're to bring that network response directly into our code and parse the response body, we would really just be looking at binary data that represents text. That's not super useful to us in our JavaScript code, right? It would be much more useful to have a JavaScript object, right? Because then we can use kind of dot notation and array indexing to actually get into that response and get to the data that we actually care about. Luckily, the fetch API makes it really, really easy to do this. So whenever we make a fetch request, we are getting a promise back, right? So we have to await it. Once that promise is resolved, we get a response object. That response object has a dot JSON method that we can call. That dot JSON method is responsible for parsing kind of the textual JSON into an actual JavaScript object that we can use. And that method is asynchronous, so we also need to await it. So again, what's happening here is we're making a fetch request to a server. We're getting a response. And then we're parsing that response. Well, we're parsing that response's body, right, into an actual JavaScript object. Now that's just a JavaScript object, right? We can treat it like any other variable in code. And that's really the way we're going to want to work with it. OK, let's take a look at the assignment. So it says our getLocations function is almost done. We just need to parse the response data as JSON and return it. So we need to parse that JSON data into a JavaScript object and return the JavaScript object. Let's take a look at this code. So getLocations is clearly just a fetch call. That's the response, right, that we're supposed to parse into JSON and return. I'm curious what it's being used for. So if we scroll down, getLocations returns here. Supposedly, again, this is supposed to be the JavaScript object. We're going to log, got some locations from the server. And then we're going to iterate over the location. So this is interesting. Actually, this response is not an object at the top level. It's an array at the top level. And then we're going to iterate over that array and kind of log each location in turn. Let me show you what I mean by that. So here we have some valid JSON data. Let's head over to JSONLint. This is one of my favorite websites for parsing JSON data. If I validate it, I'm getting some ads. Hold on. If I validate it, valid JSON, right? Again, this is an object at the top level. Here's something that's interesting. Instead of having movies as a key in the object, we could also put this array at the top level. Validate that. It's also valid JSON, OK? So in this case, we have an array at the top level, and then there are objects within. Both are valid. And in this case, it looks like this is the structure that we'll be working with in the assignment. We're just working with an array at the top level, and then within the array, we have location objects. Let me show you that one more time. So here is the JavaScript object coming back. It's actually an array. We iterate over it and log out each location in turn. If I run it now, I'm guessing it just totally breaks. Yeah, locations is not iterable. That's because we never actually returned anything. So let's go ahead and fix the code. So we should be doing something similar to this, right? We want to take this response. All the dot JSON method on it. So that should give us a promise. If we await the promise, we'll get a JavaScript object, and then we just need to return that. Run that. Awesome. That looks good to me. I'm going to go ahead and submit it. So a couple more notes here about JSON. So again, JSON is just a string. So when we parse it into our code, that's the time when we're converting it into kind of an in-memory object. And it's important to note that it's not only used in JavaScript. JSON is so ubiquitous because it's used across programming language, right? Every programming language is comfortable working in JSON. For example, in JavaScript, we parse JSON into either JavaScript objects or arrays. In Python, for example, we parse JSON into either arrays or they're called lists in Python or dictionaries. And in Go, we parse JSON data into slices or maps or structs. It's all kind of the same idea, right? Key value pairs and arrays. Cool. So some common use cases of JSON data are obviously HTTP requests and response bodies. But JSON at the end of the day is just a structured text format. So you'll often also see JSON files, right, that are actually saved on your file system, used as kind of configuration files that can be read from disk and parsed that way. There are also some popular NoSQL databases that kind of use JSON as their API that you interface with, things like MongoDB, Elasticsearch, and Firestore. And then as a final note, I pronounce JSON, right, J-S-O-N, J-S-A-N, Leviosa, but some people do pronounce it Jason, and they're wrong. I'm just kidding. People pronounce it however they want, but I do say JSON. Okay, cool. The question here is, JSON can only be used by JavaScript programs. That is absolutely false. Okay, next question is, in JSON, different fields in an object are separated by a, and we've got period, semicolon, comma, and colon. So here's some, here's some valid JSON, I hope. Valid JSON, yep. You'll see it's commas that separate the fields. So make is a field, year is a field, and we're using commas to separate them. Okay, next one is, JSON is a database programming language, format for communicating and storing data, or plugin for JavaScript. All of these are false except for the format, format for communicating and storing data. So in the last coding assignment, we parsed JSON data, right, using that dot JSON method. But JSON is also very useful for sending data. So we can send data to a server using JSON. So there's two kind of methods that are really important to understand. There's JSON dot parse and JSON dot stringify. Both built in to kind of the browser API or, you know, pretty much any JavaScript runtime. And JSON dot stringify is the one that's particularly useful for sending data, right? Because we can build a JavaScript object in our code, and then we can call JSON dot stringify to get the string form of the JSON data, which is what we want to actually send in an HTTP request. So let's jump down into the assignment. Assignment says we need to keep track of when players discover new locations. However, there's a bug in the update location by ID function. Yes, that's this function here. It looks like the location's discovered property is not getting saved properly by the server. Okay, so location object as a discovered property. Let's expand this and take a look at the code. You find, hmm, let's run the code actually. Okay, cool. So the bandit camp location, which was fetched by the server, looks like this. Got discovered, false, ID, some big long ID string, a name, and recommended level. Okay, so it's saying the discovered property isn't getting saved properly. So after discovering the bandit camp, we fetched again from the server and it's still false. Okay, I'm understanding what's wrong now. So if we look at the code, we're getting a location, we're updating its discovered property, we're updating the location by sending a fetch request, and then we're getting the location again. And the problem is that this should now be true once we've run that update location by ID function. Okay, so location is just a JavaScript object with a discovered property that is a Boolean, and we're setting it to true. Problem is up here, that location object is somehow not making it to the server properly. So we need to stringify it, right? Because the body section of the fetch call takes a string, not a JavaScript object. And the reason for that is the body of an HTTP request does not always need to be a JavaScript object. It doesn't need to be a JSON object, right? It could be something else. So we can't rely on the fetch API to stringify the object for us, right? Because we could be sending HTML here, we could be sending plain text. So we need to be responsible for stringifying it. Let's see if that works. Discovered true. Right there. Looks good. I'm going to go ahead and submit that. So if JSON dot stringify takes a JavaScript object and converts it into the stringify JSON form, JSON dot parse does the opposite. It takes as input a string that is meant to be structured as a JSON object and returns the associated JavaScript object so we can use it in our code. So let's take a look at this code example here. We've got this JSON string. Again, this is just a string that looks like a JavaScript object. The JSON string. And we call JSON dot parse on it and we get back the object and now we can use the dot operator to access fields of that object. So let's jump down to the assignment. It says it's common for developers to write local tests using mock or fake data that looks like real data. So you're working with some server and you expect it to give you some data response, maybe some JavaScript object. And rather than testing with the server directly, you can kind of just build the string representation of what you expect to get back from the server and test with that. And then once everything's working, you can kind of plug in the server to the rest of your code. But sometimes mocking out the data makes testing a little bit easier, especially if there's certain conditions under which the server would return a certain response. And so rather than meet those conditions while you're testing, you can kind of just mock that data response and make sure that your code handles that data in the way that you expect. So moving on with the assignment, it says let's ensure that the JSON format that the back end fantasy quest developers provided to us is valid JSON. It would be a shame to write a bunch of code just to find out that the back end has given us the wrong format. Right. Really suck to just do all the work and then just have to go back to the back end developers and tell them they have a bug and then have to wait on them again. So the assignment says complete the parse location function, use a try catch block to safely call JSON dot parse on the location string provided. Keep in mind that JSON dot parse throws an error if it is given a string that isn't a valid JSON. If we give it garbage, it's going to throw an error and we need to capture that. If you could parse the string successfully, use the print location object function to print the parsed object. If an error was thrown log invalid JSON strings. OK, cool. So let's first let's just get that error so that we know what's going on. So let's do JSON dot parse and let's just pass in some garbage garbage data. Just a just a one word string is not valid JSON. So let's go ahead and on that syntax error unexpected token D data is not valid JSON. OK, cool. That's doing what we'd expect. So now we throw this into a try catch block. This in the try section because it's the dangerous code. And then we're supposed to log invalid JSON string if there's a problem. So let's not log invalid JSON strings. Now I'm going to run that, see what happens. Invalid JSON string, invalid JSON string. OK, so we're always getting invalid JSON string for these two calls to parse location. And that's OK, because we hard coded this. OK, so now we need to pass in the actual location data. At this location information, which is stringified. So something like this or something like this. And let's save that as JavaScript object. So this would be location object. And then we're supposed to, if you can parse this string successfully, use the print location object function. Print location object. Pass the location. Cool. So if this parse goes well, we'll print out the location. Otherwise, we'll log invalid JSON string. I'm going to go ahead and run that. So it looks like the first call to parse location was invalid JSON. The second one parsed properly. If we go down and look. Yeah, this one's invalid because it's missing. It's missing a closing bracket there. That makes sense. I'm going to go ahead and submit that. It would be a shame to go into so much detail about JSON without bringing up any alternative formats. So XML is essentially a JSON alternative. And you know, my perception of it is that XML seems to be used a lot more kind of before JSON took over. I would argue JSON is used in more modern applications. You might see more XML used, at least for kind of HTTP requests in more legacy code. That said, there's nothing wrong with XML. You can still use it. But at the end of the day, XML and JSON, when it comes to sending data in HTTP requests, serve a very similar purpose. They allow us to structure data in a textual or kind of text format. So let's take a look at what the syntax looks like, because at the end of the day, it's really just a difference in syntax. So this data in XML would look like this in JSON. So instead of these kind of root tags, we just have brackets in JSON. And then instead of opening and closing tags for each field, we have a single key that corresponds to a value. So at the end of the day, I would argue JSON is more popular these days. And that's why we're going to spend most of this course working with JSON. But it is important to understand that XML is out there and that you may encounter it. One of the reasons I think that so many developers prefer JSON for, again, networking or HTTP work is it's a little easier to read and write and work with. For example, writing this field, all I have to do is type genre colon action. I don't have to do open tag genre, you know, then write action and then close tag genre. I'm not writing genre twice, I guess is my point. Cool. So now that we know a little bit about XML, the question is XML and blank serve similar purposes. The answer is going to be based on the next question is XML is blank verbose than JSON. So more verbose or less verbose verbose just means like lengthy. Like how much do you have to type and read in order to get the same message across? And I mean, if we just look here, I would argue that XML is more verbose. Right. Like we brought up before, we have to type each the name of each field twice. So I'm going to go with XML is more verbose. So JSON and XML both accomplish very similar things. The question is, which should you use? Right. Both JSON and XML are are used to send network requests, right? Like HTTP requests transfer data from one system to another. They're also used as configuration files. Maybe you need to configure some settings in a project repository you're working on or a dependency management system that you're using or just a piece of software that upon startup kind of reads in some configuration data. Very often, JSON and XML are used for those use cases. Sometimes you're going to be forced to use one or the other. Right. If you're working with a server that expects JSON, congratulations, you're using JSON. If you're working with a server that expects XML, you're using XML. Right. That said, if you have the choice, maybe you're helping design an API before it's built. Maybe you're the back end developer or maybe you're configuring a project and you just have optionality for how to structure your configuration files. I would argue that you should generally prefer JSON. It's lighter weight, right, because it's less verbose. So it actually lacks less kind of bytes of data in the file itself. But there's also this kind of readability component to it. So I would argue JSON is just a little easier to read and write. So the question is, which should you prefer to use? And I would say you should prefer JSON. The next question is, which can be used to send data from the browser to a web server? JSON, XML, both or neither? The answer is definitely both. Every time we send an HTTP request, we have to specify an HTTP method. It's not optional. And up until this point in the course, we've been using a few different HTTP methods. In fact, you've probably noticed them and maybe even thought like, what the heck is this thing? I think we've used get requests, maybe even post and put requests. But we haven't explained what they do yet. And in this chapter, we're really going to dive into HTTP methods. So let's take a look at some of the most common methods. Just really quick. This is on the MDN documentation. We've got get, add, post, put, delete. Those five are the most common. I would argue you could kind of exclude head from that category. Actually, maybe get, post, put and delete are the most common. And then there's some other less common ones like connect options, trace and patch. Cool. In this exercise, we're going to be diving into the get method. So the get method does kind of what it sounds like it does. It gets stuff. Specifically, it gets representations or copies of stuff from a server. So for example, if you want to get some information about a user, say a user is logged into your website and they want to see their profile photo, right? They might do a get request to the server to get a copy of their profile photo so they can look at it. Because get is all about getting information, getting copies of information so that it can be displayed visually in a browser or in a mobile app or whatever. It's considered safe because it doesn't actually change any of the information on the server, right? Updating something or creating something or deleting something, right? Like, I don't know, changing a profile photo, swapping out a password, that kind of stuff. Those are considered a little more dangerous because data is changing. Get methods can be called multiple times and nothing changes. So they're very safe to use. Let's talk about how to make get requests using the fetch API. So we've already used the fetch API pretty extensively. Here's the syntax down here. We have the fetch function that again is built into the browser. It returns a promise. So we're going to have to await our fetch calls. The first parameter is a URL. And the second parameter is an options object. So the URL is just the URL we are making our request to. The options object, however, has a few other interesting things. So the first is the method, right? We're making a get request. So we have to specify the get string in the method. The mode, we're not going to go into a ton of detail about this. It is browser specific. Essentially, cores is a mode that we need to use in the browser so that the browser doesn't kill our request for security purposes. That's for cross origin resource sharing. And essentially by setting this mode, we're going to allow the browser to make our HTTP request. Then we have this headers object, which is just these kind of string to string key value pairs, right? In this case, we are setting this header that specifies that we are on a Mac OS client. But again, these headers could be any key value pairs. And then what we are not including, what we are not including in this fetch call is the body. We have done that in the past. When we are sending get requests, we typically do not include a body because we are saying we want a resource, right? So when we want to update something or we want to create something, we're usually sending data. So we need to include a body. But when we were just asking for data, we often don't need to include a body. So that's why we don't have it here. So on to the assignment. We need to write a reusable function that retrieves all of the fantasy quest users from the fantasy quest server. So we'll be completing the get users function. It should number one, call fetch using the URL parameter. All right, so let's do we'll need to await a fetch call. All right. Which first parameter is the URL we want to make the request to. Next, we'll need an options object. So use the get methods method. Yet. We'll need to use the cores mode. We need a header. In this case, it's the X API key header. This is the this is the header that the server expects to authenticate us so we can prove who we are. You can think of an API key is kind of like a special password. And so you could imagine, I mean, we're getting all the users on the server. This is something that would probably be behind some form of authentication on a production server. Typically, only admins would be allowed to get all of the user data for an entire for an entire server. Cool. And then we can just use the API key provided to function. And then return the resulting JSON data from the response. OK, so this returns a response object. Then we'll need to await. Based on method, we're basically saying we expect this response. You have JSON data in the body, and we're going to parse that into a JavaScript object and return the JavaScript object. OK, let's go ahead and run this and see what we get. Character name, class warrior, level 14 username. OK, that all looks pretty good. Let's take a look at the code just to see what the heck's going on over here. So first, a new API key was generated. We are hitting this user's endpoints on the boot dev server. You're getting users and then we're logging the users. And that log user function is just going to iterate over all the users that we get and kind of log out some information. So this this is all looking good to me. I'm going to go ahead and submit that code. Let's talk about CRUD. So CRUD is an acronym. Stands for create, read, update and delete. Create. I'll do these. Let me do these in different colors. Create. Read. Update. So when we're dealing with a web server, pretty much every action we take, nearly every action we take when interacting with a server is one of these actions. We either want to create some new resource on the server, right? Create a new user account, create a new weapon in a video game, right? Create a new tweet. We want to read something, right? Read a tweet, read a list of tweets, see which of our friends are online. We want to update something, right? Maybe update a password or we want to delete something, delete an account, et cetera. Like I said, almost all actions fall into one of these four buckets, right? Create, read, update, delete. As it happens, these actions map very simply to the HTTP methods. So create is a post HTTP method. Read, this is the one we've already covered, is a get. Update is going to be a put. And I'll just like special mention, sometimes it's a patch, but I would underscore it's usually a put. And then delete is the easiest one because it's delete HTTP method. Okay, so when people talk about CRUD servers, they're talking about a web server that allows the users of the API to create, read, update, and delete resources. And then typically, they will use these HTTP methods to do it. Now, I want to point out, this is a convention. This is a convention, this mapping, right? Technically, there's no reason a server can't use get requests to create resources. There's nothing stopping the server from being designed that way. However, the convention in HTTP, specifically kind of RESTful servers, which is something we'll get into later, is to map kind of the create, read, update, and delete to these specific methods. And it's just so that it makes it easier on us, right? We get used to doing things this way. And then when we're working with APIs, which every API is different and complicated, because we're using something familiar, it makes it easier to interact with. So again, this is a mapping that you don't necessarily need to memorize it now, but it's something that you definitely will get really familiar with over time as you work with more and more servers, and it will become second nature. Okay, so now we've got a question that goes along with all of that, create, read, update, delete, kind of mapping to HTTP method stuff. So the question is, HTTP methods map to CRUD actions by convention, automatically, or always? The answer is by convention. Next question is, to update a user account, I would probably use a blank request. The answer is going to be put. So an HTTP post request sends data to a server typically to create some new resource on the server. And the way that differs from a get request we can see down here. Well, there's actually a couple things. So first of all, the method is post, one should be fairly obvious. But additionally, we need to send a content type header. Not every server is going to require it. But it's definitely best practice to always inform the entity on the other side of a fetch or an HTTP request, the kind of data that we're sending in the body, right, so they don't try to parse it the wrong way. So in this case, we're going to be sending JSON data. So we should specify in the headers that we are sending JSON data. And then finally is the inclusion of a body. So when we were getting data from a server, we didn't need to send any data in the body. We could leave it empty. Right. But now that we are actually sending some information to the server so that that information can be saved on the server, we need to specify that in the body. It's also important to point out that those requests are not safe in the same way that get requests are. Get requests are very safe inherently, right. Nothing's changing on the server. Typically, we're just asking for kind of copies of information that's stored there. Post requests are altering the state of the server in some form or another. There are strategies that servers can use to kind of deduplicate. Say you were to accidentally send the same post request twice to create a new user. Smarter servers can can kind of deduplicate that and make sure only one entity is created. But that's not always the case. So it's just important to keep in mind that because post requests are kind of mutating data on the server, they're changing something that's stored there. You do need to just be a little more careful with them. So let's jump down to the assignment where it says we need to save newly created characters on the fantasy quest server. So we'll be using the get users function that we already wrote. And we'll be using that function to verify that our create user function is working. So if we look at the test suite, it looks like we'll be getting all of the users, creating a new user, and then getting all of the users again. So we can see that the one that we created exists after we create it. Make sure it works. OK, cool. So take URL and data as parameters. We got those there. We also have an API key. Calling fetch. So let's save the results of fetch in a response. And then we need to await the fetch call. We'll be making the fetch request to the URL that we're given. And then we need to specify the options. So the first option is going to be the method, which in this case is a post request because we're creating a user. Next, we need to set that cores mode, just like we did last time. And then for headers, we're actually going to have two. The first will be this content type header. Server knows we're sending JSON data. And then the second will be that authentication header, the API key header. This will be the API key that we're given. Finally, we need to actually send the data itself. Now, here we have this parameter called data. Let's go take a look. We scroll down to where create user is being called. The user to create is what's passed in there. And user to create is just this JavaScript. OK, cool. So we just need to stringify that. Body. JSON.string. Stringify data. OK, let's run that. Get. Cool. So retrieving user data, we got all of these characters or characters. And then we created creation response body undefined. Probably not good. What did we screw up? Response. Oh, let's finish reading the instructions maybe. OK, include the body field, pass the data, return the responses JSON body. OK, so we need to return response.JSON. And don't forget, we need to await this because the.JSON returns a promise. Await that. All right, let's run it again. OK, so there's our users creating new character. Here's the response body. OK, so this is the new character that we created. Name is Grendel. We scroll down. OK, cool. Grendel is in this list, but he's not in the first list. That makes sense, right? We got all the users. He wasn't there. We created Grendel and we got all the users again. He was there. So this is looking like it works to me. I'm going to go ahead and submit that. Let's talk about status codes. HTTP status codes. So status codes are basically one additional thing that we need to check to ensure that an HTTP request went the way that we hope it did, right? That everything went well. The first thing that we typically check, right, is if an error was produced. And we've already talked about errors. Status codes are kind of the second level, the second layer that we need to pass in order to make sure that, again, the request kind of went off without a hitch. Let's diagram the lifecycle of a fetch request so that we can see kind of how status codes and errors differ and when you might get one versus the other. So over here, I'm going to draw the client. And on this side, I'll have the server. You're probably super familiar with this diagram I'm going to draw by now. This will be the HTTP request going out. And this will be the HTTP response back. Okay. If something goes wrong in this area, it will produce, most likely produce an error. So for example, if we don't give a valid domain name to the fetch API, we might get an error. Or if the internet connection is down, so we're not able to complete the request, then we would probably get an error from the fetch API. Okay. If something goes wrong on the server. So this means our request successfully made it to the server. And now the server is processing the request. If something goes wrong during the process of the request, then that results in a status code. And we will still get a valid HTTP response back. The status code is contained in in the response. Okay. So if something goes wrong on the server and the server detects it, it will let us know in the HTTP response via a status code. So this might be something like, hey, you don't have permission to access that resource. That you're requesting, right? Or one of our backend services is down. We're not able to process that request. Those are the kinds of things that you'll see in status codes. So another one of the big differences between status codes and errors are that errors only occur if something goes wrong. Whereas every HTTP response has a status code. We just need to check what the status code is to see if something actually went wrong. So the codes are broken up into kind of different error levels by by number. So status codes are between 100 and 599. And basically 100 level codes are informational responses. Frankly, they're very rare. You won't see these very often. 200 level codes are hopefully what you see the most often. And they're success codes. So 200 is the famous. Everything's OK. You'll see 200 quite often. 300 are redirection messages. So they're basically telling the client, hey, the server moved. You need to redirect your quest to somewhere else. If you've ever been to a link, like if you ever visited a link that doesn't actually end up taking you to the URL in your browser that you visually saw, you likely were redirected through a 300 level status code. And then 400 are client errors. So it's basically the server telling the client, hey, you're not authenticated. You don't have permissions. You formatted something improperly. And then 500 level errors are server errors. So it's actually the server saying, hey, something messed up on our end. Maybe our database is down or something like that. So let's take a look here at some of the more common status codes that you'll see. So 200 is just OK. Right. This is by far the most common code and basically just means your request worked. Right. You can move on. 201 is just a little bit more specific. So it's saying everything went well, but specifically you created something. Right. So 201 is often the response to a post request when a resource is created on the server. 301 moved permanently. So this means, hey, the server that handled this type of request used to live here. Now it lives somewhere else. You should go there to deal with it. So, for example, browser redirects use 301s all the time. If a web page moves to a new domain, typically 301 redirects are set up so that all the new traffic can move to that. All the traffic can be redirected to that new location. 400 is just kind of a general bad request. Again, it's like a catch all. It's just like you did something wrong in your request. You know, figure it out. And a lot of times when kind of error status codes like 400 and 500 level status codes are sent back in an HTTP response, it's very often that the body of that response will contain more information about what went wrong. So we would typically check the status code, see that it's a four or five hundred, and then maybe read out the HTTP response body to see what specifically happened. Maybe log that to the console or to some logging system for us to check on later. 403 is one that comes up quite a lot. It's just unauthorized. Maybe you forgot to include an API key. Maybe using the wrong API key. Maybe you don't have a session token, something like that. 404, you've probably encountered this on the web. It just means not found. Maybe you tried to navigate to a web page on a domain that doesn't exist. You could get a not found error. And then 500 is by far the most common 500 level code. It just means internal server error. Very often this is like a bug on the server or maybe the database is down or something like that. I don't think you need to memorize all the status codes. In fact, there's a link here to all the status codes, and there's there's quite a lot of them. You don't need to memorize them. What I would recommend is just knowing the basics, which are basically 200 level is good. 400 is client error. 500 is server error. And that's going to be kind of really all you need to know to get going as a web developer. So the question for this exercise is a successful response returns a blank status code. And that is going to be a 200 level status code. Okay, same topic, different question. The purpose of status codes is to blank to alter the method of the request to inform the client about whether or not the request was successful. Do not need to send a response body or to speed up the request. It's going to be number two to inform the clients about whether or not the request was. Okay, let's take a look at what checking a status code actually looks like in code. So the dot status property on a response object gives us the code. Okay, so we've got our response object from the fetch call. This is the property that has the status code update to get user code function to return the status code of the response. Okay, so we're making a call to that get user endpoint, and we want this to return a status code. So if I just run this without making any changes, I'm getting ID invalid, status code undefined, ID, some ID, and then status code undefined. If we instead return on status, run it again, invalid ID, status code 404, good ID, status code 200. Okay, cool. So when we pass in the test suite and invalid ID, we're getting back a 404. That makes a lot of sense. We're not able to find the user with the invalid ID. And then on the other hand, when we pass a valid ID, we get we get back 200. Let me go ahead and submit that. Let's talk about HTTP put, right? This is the method that is most commonly associated with updating a resource. It is worth mentioning, however, though, that it can sometimes be used to create a new resource that does happen. I would argue it's it's a little less common. Sometimes you'll see an endpoint where you are able to give it data via a put request. And the action on the server is to create it if it doesn't exist or to update it if it does exist. That would typically be done with a put request. So the main difference between post and put is that post is almost always just creating. Whereas put is like update or create or just update. Another big difference is that put is meant to be idempotent or idempotent. I don't know how to pronounce that. But basically what it means is you should be able to send a put request multiple times without creating multiple resources. Effectively, the you send the put request the first time and it makes an update. If you send the identical put request, nothing happens. It's a no op or a no operation. So put requests should be safer, generally speaking, than post requests. And the reason they're able to be idempotent, again, generally speaking, is because you'll often be including the ID of the resource that you're updating. Whereas with a post request, you'll often be kind of relying on the server to generate new IDs. So that makes sense. If you make multiple post requests, you're generating new IDs for new resources. Whereas if you're making put requests, you're always using that same ID. So you just be updating fields you already updated. Let's jump down to the assignment to see what this actually looks like in code. OK, it says you will now use the knowledge you have gained thus far to update a fantasy quest character using a put request and retrieve that character using a get request. OK, so we're going to update the user and then get the user by its ID. So if you remember kind of before we were doing a get users request where we were kind of getting all the users, now we're going to be getting a specific user by its ID and updating it by its ID as well. OK, complete the update user and get user by ID functions. They should update and retrieve individual user resources respectively. They should also return a promise that resolves to the JSON response body of their respective fetch requests. OK, so we're returning JavaScript objects that we're able to parse from some JSON. We've included the full URL creation logic for you in both functions. We'll be talking more about URL building in the next chapter. OK, cool. So these are the URLs we'll be making the requests to. We need to just make the requests. So go ahead and do that. Const response equals await fetch, right? We've done this a couple times now. We're going to use that full URL. And this time our method and the options object will be put. We still need mode scores, headers. In this case, we will be sending a request body. So we do need to specify that content, content type header, the application slash JSON. And then we need a body. And that's going to be JSON dot stringify data. OK, so let's take a look at what all this means so far. So we're building a URL based on kind of the base URL and the ID of the resource we want to update. So hopefully that makes sense. We have the URL path and essentially the last section on the path is going to be the ID of the resource we want to update. And if you look down here in the get by ID, it's the same thing. We're essentially just adding that last section, which is the ID of the resource. And that's how the server knows which user we want to update. And then in the body of the request, we're just sending the new fields. So let's go look and expand this, let's go look at the code that actually uses this. So here's our base URL, right? It's kind of the server URL. And we're hitting that user's endpoint. So again, here it's going to be users and then kind of slash the ID again. OK, so down here, base URL user ID generated key. That's the get. Where is the update? Update has this user data in here. OK, cool. So what's happening is first we get the user and we log it out. Next, we're going to change some of the user data. So we're changing the character name. We're changing the level, the class, whether PVP was enabled, et cetera. And then we're calling the update user function and passing in that new data that's now been modified. And then we'll get the user again just to make sure that it worked. OK, so all that makes sense. What was I supposed to return again? Return a promise that resolves to the center spot. OK, so return resp. All right, cool. To get request, I'm going to do a sin here and copy. Look very similar. Let's swap out the things that are specific to the get request. So first of all, there is no body in a get request, right? We don't need to inform the server of anything. You swap that out to a get. And we also don't need these headers. Oh, we forgot our API key. That reminds me. Almost forgot the X API X dash API dash key. Close. And here we do not need a content type because we are not sending a body of data. Cool. Let's go ahead and run that code. So user UID character name. Third. Updating user with ID. F8 too, so that would be the Lithria. And then we got the user and now its name is Delby are warrior level seven. OK, cool. I want to pause here and just make sure you understand what's happening. So we we got the user with this ID. And we logged their kind of their metadata, right? Name, class, level, all that stuff. Then we updated the user. Again, that's happening in code. Right here. And then after the update, we get that same user, right? Notice the same ID. But all the fields have changed. Well, so that seems to have worked. I'm going to go ahead and submit it. So we've offhandedly mentioned the patch method a couple of times. And I want to just kind of clear up any confusion. So HTTP put is by far the most common HTTP method for updating resources. However, patch is another one that is sometimes used. And here's the difference. The intended difference for these two HTTP methods was that put would swap out entire resources or update entire resources. Whereas patch would just update partial sections of a resource. So here's an example with the users that we just updated. We essentially put an ID in the URL path to indicate which user we wanted to update. And then we sent an entire user object, right? And we said this is the new user object with all the properties update the whole thing. Right. So that's kind of the intended use case of a put a patch would have looked something like. Well, we still include that ID in the URL. But now instead of an entire user object, maybe we just have a name field. And we say we just want to swap out the name or we just want to update the name. Right. So that would be what patch was intended for. Now, in practice, you won't see patch used very often. Typically, servers just use put for all sorts of updates, whether it's partial or full. Whether or not that's a good idea. I don't know. I don't really have a strong opinion on it. I'm OK with, you know, back end developers doing it however they want. Like with everything, when you're making HTTP requests to a server, you really just at the end of the day need to read the documentation to figure out whether you should be sending a put or a patch. Right. By convention, it's probably going to be one of those. And more likely than not, it will be a put. So the question is blank is more popular than blank. And the answer is put is more popular than patch. OK. Talking about patch still. The question is, put typically updates blank while patch updates blank. And the answers are, you know, the swapped forms of partial resource and entire resource. The answer is that put typically updates entire resources while patch is intended for partial. Let's talk about HTTP delete. It is the most obvious of the HTTP methods. Right. It does what you think. It deletes resources. Let me expand this so we can see what the heck is going on here. So here's an example of an HTTP delete using the Fetch API. Basically, in this case, we have the URL of the API, right, which technically for us spans all the way out to here. This is the API that's specific to the Learn HTTP course. And the first section of the path is telling us what resource we are working with. So locations resources. And then this next section is what's the ID of the resource that we're working with. So it's going to be this big long ID here. And then when we make a fetch request or an HTTP request to that URL and use the delete method, we're telling the server, hey, I want to delete that exact resource. So jumping down to the assignment, it says users need to be able to delete their player accounts. Complete the delete user function. Send a delete request to the given full URL. You'll need to set the method, mode, and headers like before, set the X API key. Cool. This all makes sense. All of it. So const resp equals await fetch URL. So again, the ID of the, in this case, user that we're deleting is going into the path. Options object will be using the delete method. Specifying cores always because we're working within a browser. And then we're going to need some headers because we need to pass that API key, right? We need permission to delete thing. Cool. And that should be all we need, right? No body on most delete requests. The thing we're deleting is in the path. So the server already knows what we are deleting. OK, so let's look down at the test suite now and see what is expected to happen. So it looks like first all of the users will be retrieved and logged out to the console. And then we'll delete one user and then we'll get them all again. Cool. That makes sense. OK, logging user records. So we got a whole ton of user records and then we deleted the one with ID 0194. Where is that? 0194. OK, great. Convenient. That's the first one. Looks like it's my user record. OK, cool. So after it's deleted, it should no longer be here. And indeed, it looks like it is no longer there. And now the first one is the Allen record. Awesome. I'm going to go ahead and submit that. So we briefly talked about URL paths in the URIs chapter quite a few chapters ago. And now we're going to dive quite a bit deeper into URL paths and how they relate to kind of modern web servers. So if you remember, we talked about how traditional file servers will very often map the file path on the server to the URL path that's used to make kind of HTTP requests into that server. So, for example, if this were the URL test domain dot com slash root slash next on a traditional file server, then the path to that next file would be slash root slash next from wherever the file server is running. When we're working with web applications as opposed to just websites that might use traditional file servers, paths in our HTTP requests are often treated very differently or they're used for very different things. In fact, let's break that down now. So I'm going to grab this example endpoint. From the fantasy quest server that we've been interacting with so far. And let's just look at this big we have this big long path here and I just want to kind of explain how I've designed the boot dev API. So you can understand that again, it's not a traditional file server. So it's got a little bit more. It's just got a little bit more to it. So the first section of the URL obviously is just the server. It's our domain API dot boot dot dev. The first section of the path is the version of the API that's being interacted with. So if I ever made big breaking changes to this API, I could just increment this from v1 to v2. Right. And then everyone that's using the v1 endpoints would still be able to use them under the v1 path. And anyone who wants to upgrade to the new version to API could start using the v2 path. That's a pretty common convention when designing APIs. This next section is just essentially to separate kind of the APIs for my students to work with from the APIs that kind of power the website. So these this courses rest API, this is the rest APIs for my courses that students interact with directly. And it kind of separates the rest of the API that I'm using internally to power the website. The next section is, you know, which course are we interacting with? And obviously we're in the Learn HTTP course. So that's what that section means. And then the final section is just the resource we're working with. So here we're working with locations, but this might also be users or I don't know, weapons or something like that. So let's jump down to the assignment. Our goal is to fix the get resources function. It's supposed to take as input a path. Let me expand this so we can actually see. See the whole thing. It takes as input a path and makes a get request to kind of the base URL plus that path and then returns the kind of the JavaScript object that was retrieved from that path. So if we look here, basically we're making a get request to the locations endpoint, and then we'll be logging out all those locations. Then we'll do the same for the items and users endpoints. Again, the interesting thing about this function as to ones that we've built in the past is that it's a little more flexible. The user of the function can kind of pass in a dynamic path. We'll be making making that request. So it looks like all we need to do really is append the path to the URL because this variable full URL is currently what's being used in the fetch request. So if I just ran it in its current state, yeah, we're getting an error because we're just making a request to the root domain and that that part of the server is not designed to return any sort of JSON response. So let's make an update here and let's append the path and the path to the base URL. And that should work. That should be all we need to do. Take a look at the response data. Okay, cool. So now we get all of this location data, right? Heartfin Marsh, Iron Deep, Lakewood. And next, we get all this item data, healing potion, light leather, padded leather. And then finally, it looks like we're getting all the user data. Rundorf, Varus. Cool. Looks like it's all working. I'm going to go ahead and submit that code. So you've heard me toss around the term RESTful so far in this course, probably. But I haven't really explained what it is yet. REST is an acronym, stands for Representational State Transfer. But all it really is is a set of standards and conventions for building better APIs. At least that's the promise, right? So again, the whole goal of a RESTful API is to adhere to certain rules so that a client, right, us in this case, that's interacting with the API, kind of knows what to expect and has a really good developer experience working with that API. So what are the rules and conventions that RESTful servers adhere to that kind of separate them from other kinds of servers, other kinds of APIs? Well, the first is that one of the kind of core tenets of a RESTful API is that the two pieces, the client and the server, are kind of separate and agnostic, right? So the client can be written in any programming language. The server can be written in a different programming, right? There's kind of a minimalist set of things that the two need to agree on, things like, you know, what names are we going to use for resources? Are we going to use JSON or the encoding scheme? But everything else can be completely separate and distinct. Another key tenet of RESTful APIs is statelessness. So the server will have its state and the client will have its state. But in a good RESTful server, you won't have kind of persistent state that deals with the connection between the two. Let me give an example. So on the server, let's just say it's the FantasyQuest server and we have 100 registered players in the game. Well, then one player deletes their account. Now there's 99. So the server is inherently stateful, right? It has state, it's keeping track of state, and that's totally okay. There's really no way to get around that. What's interesting about REST is that the client and the server shouldn't be keeping track of, like, past interactions that they've had. So, for example, if the client says, give me the first 10 users, and it gets 10 user records, and then it makes the same requests, the same request, excuse me, the server shouldn't remember that it already gave that client 10 users and give them the next 10 users. It shouldn't do that. What should have to happen is the client specifically asks for the next 10 users. So now it's a separate request. And basically what that means is the server doesn't have to keep track of any state. It doesn't have to remember, hey, this client already asked me for some stuff. I'm going to change the way I respond. It's just a very clean way to write an API that kind of puts some responsibility on the client to ask for things in, like, new and different ways when it needs new and different information. Another key component of RESTful servers that really sets them apart from other types of servers is that the path is used to designate the resource that's being interacted with. So whenever you're working with a RESTful server, you should be able to hit, say, the slash locations endpoint, if there is a locations resource available on the server, and be able to create, read, update and delete that resource in a predictable way, right, just by by making requests to that path with the appropriate HTTP method. So the question is a true false statement. And it says RESTful servers communicate the state of resources. They don't expose access to arbitrary server commands. So this is true, right? REST is all about state transfer, just transferring the state from the client to the server, from the server to the client, right? Both are staying in sync with each other by making requests back and forth. Arbitrary commands are not allowed. Next question, does a backend server need to adhere to RESTful standards? The answers are no, but it's a common convention designed to create better API's. Yes, otherwise the requests will fail. Yes, if it doesn't, it's an insecure server. And no, it's generally a bad idea. Well, the answer is definitely one of the no answers, right? REST is just an option. A server or an API doesn't need to use it. So that narrows it down to it's generally a bad idea. And it's a common convention designed to create better API's. Well, it's not generally a bad idea. REST is very popular and it's popular for a reason. I wouldn't go so far as to say REST is better or REST should always be used. There are other popular conventions, things like GraphQL, for example, if you've ever heard of that, completely ignore all of the RESTful conventions and do things a completely different way. But I will say it's this top one. It's a common convention and it is designed to help create better API's. Next question, the blank part of a URL path typically denotes the resource. Scroll down to paths section. The answers are first, last or middle. And the answer is going to be last, right? Last section of the path typically denotes the name of the resource. So we already talked about query parameters a little bit in the chapter on URLs, right? You'll recognize this example where Q equals boot.dev is a query parameter, right? The query parameters in the URL come after the question mark and their key value pairs. So here the key is Q and the value is boot.dev. In RESTful servers, query parameters are very often used in get requests to kind of change the way that data is sent back to us from the server. Essentially, we're asking for the data in a slightly different format. So let's jump into the assignment to kind of see what that looks like. It says the backend team building FantasyQuest server has added support for query parameters. The server offers a sort parameter that can be optionally added by appending this sort equals field name to the end of the URL, where field name is the property of the resource we want to sort the kind of response records by. Our goal is to create a leaderboard and we want to sort user records by their level. So we want everyone on the leaderboard to be sorted by their level, highest level at the top. Update the get users functions full URL with a query parameter that will sort the user records by level. OK, so we've got this get users function and now we need to add some sorting. Let's go ahead and run the code in its current state just to see what we get. OK, so we got all these users and it looks like they're kind of out of order when you're looking at their level, right? We got 14 and one and six and eight. And yeah, if we look down here, it looks like get users is just being called and then we're iterating over all of the users. So we want this user's object to be sorted by level. So if this is the full URL, in fact, I want to just I just want to log it out really quick to see what it looks like. This is the full URL here. We just need to add sort the field name. The field name we're interested in is level. I'll remove my little debugging statement there. Let's run that again. Cool. Now everything looks like this. It's sorted. And this is this is important that the server is doing the sorting for us. We're just telling the server with this query parameter, hey, could you sort those before you give them back to me? Which is really important because sometimes we can sort on the client, but sometimes we can't write. Right. Imagine in the case of Google, right, when we ask for a response to a query and you know how it sometimes says three million responses, we'd never be able to sort that many responses on our machine. So we rely on the server to do it for us. Anyhow, this is looking good to me. So I'm going to go ahead and submit it. So you may be wondering, well, how am I supposed to know what resources are supported, what query parameters are supported, what HTTP methods are supported by a given server that I'm working with? Well, at the end of the day, there's there is no way to know unless you have some documentation. So when you go out to work with a specific API, right, let's say you're integrating with the Stripe API because you need to do some payments on a website. Right. Well, there's really no way to know what the Stripe API allows for unless you get access to the documentation that says, here are the URL paths with the associated resources. Here's the HTTP methods you're allowed to use. Here's the request and response bodies that you can expect. Right. All of that will exist within the documentation. If a server doesn't have support or a feature that you need, right, say the sorting feature that we just interacted with on the fantasy quest server, the only option is to either contact the maintainers of that server side code, right, the back end developers and ask them to implement it. Or if you are a back end or full stack developer yourself, obviously you can go change the code and update the documentation on your own. But it's important to understand that as a client, you don't get to make the rules. Right. At the end of the day, the server supports the functionality that it supports. And you're kind of stuck with that, again, unless you're able to lobby or make changes yourself. So the question is, when working with a back end server, you will need to access blank. The server code, the server's API documentation, the server's private keys. So this is assuming when we say working with a back end server, we are writing client side code that works with a back end server. We don't need the code. Right. We're just interacting with the back end through an API. We don't need the server's private keys, but we do need the server's API documentation if we're going to be able to interact with that API. The next question is, what should you do if the documentation isn't clear about how to use the API? Number one, contact the back end developers and ask for better docs or clarification. Number two, keep trying random requests until you find something that works. Or three, try to hack your way into the server. Well, the answer on this one is definitely the first. Right. Just contact the back end developers and ask for clarification. As I've mentioned earlier, it's possible to have multiple query parameters. So a couple of exercises ago, we sorted users by their level with one query parameter. Right. But we could also add another key value pair that does something entirely different. And the syntax for that is the question mark separates the query parameters, right, plural, from the rest of the URL. But ampersands separate individual key value pairs from one another. So to practice with multiple query parameters, let's jump down into the assignment. It says our game designer wants us to build a treasure chest. We don't want to award players with too many items, though. So we'll be adding a limit equals X query parameter where X is the number of records we want to limit to. And then we'll also sort the items using a sort query parameter. So the assignment is to complete the loot treasure function. It should add two query parameters to the URL passed to get items. OK, so this is the base URL. And we're using the get items function. We just need to alter this URL to add those query parameters. So sort and limit will be sorting by the quality property and set the limit based on the rarity passed into loot treasure. Oh, interesting. OK. So rarity is a string, common, rare or legendary. And then sort is just based on the property. OK. So let's jump into this. It should be something like this. Let limit. Let's default to one. No, let's default to zero. We default to default to null. I guess that makes sense. And we'll do if rarity equals common. Getting my parentheses. Then limit will be one. Else. Rarity is rare. Then the limit will be three. If the rarity is legendary. Then we'll set the limit equal to five. OK. And just to go above and beyond here, what I'm going to say is. If. Limit. Null. I'm going to throw a little air. Bad rarity provided. I'll just prevent kind of weirdness from happening. We're expecting the rarity to be one of these three. OK, so now we've got our limit. It's a number. And the sort is always going to be based on quality. OK, cool. So we should be able to just add these now. So we'll start our query parameters with quality. Equals. Oh, excuse me. Sort equals quality. And then another query parameter. Will be limits. Equals. Interpolate that limit parameter in. We run that code. So looting common treasure chest acquired a light leather. The name of the item with quality score one being a rare treasure chest. Three items here, they're sorted, you'll notice. Legendary treasure chest, five items. So this looks right to me, right? We're getting the right number and they are sorted. So I'm going to go ahead and submit that code. So you may have been wondering, isn't this an HTTP course? Why is every request in the course kind of prefixed with HTTPS instead of HTTP? Well, the reason is HTTPS is essentially the exact same thing as HTTP with one added component, and that is security. The most important security feature of HTTPS is encryption. It keeps the information that we are sending in our request and the information coming back in the response secure and encrypted so that only us as the sender and the server as the receiver are able to decrypt and process that information. When we send HTTP requests over the internet, they don't go directly from our computer to the server that we are communicating with. There are intermediaries, for example, your internet service provider, right? Or if you're working on public Wi-Fi, like within a coffee shop, then whoever owns that Wi-Fi or that router would be able to intercept your network traffic and process it. So if you're using plain HTTP, the data that you're transmitting is never safe in the process of being transmitted. Anyone in the middle can look at everything you're sending. Take the example of an e-commerce website. If you're entering in a credit card in your browser and sending it off to some server through a normal HTTP request, every intermediary now has access to your credit card information. That's why good e-commerce sites use HTTPS and should never use anything less secure. So let's hop down to the assignment. The assignment says the fantasy quest server only allows encrypted requests. This is very common. The entire boot dev API is all encrypted with HTTPS. We don't allow anything over HTTP. The assignment says there's a bug in the code that's causing the request to be blocked. Fix the bug. So if we send the request as it is, we get type error failed to fetch. And that's just because the server's rejecting this plain HTTP connection. So we have to send our request over HTTPS. Awesome. I'm going to go ahead and submit that. Let's talk about HTTPS and specifically its encryption. So how it keeps our data safe on the Internet. Start with a little diagram, one that you're probably very familiar with. So over here, we'll draw the client. And over here, the server. The problem that HTTPS solves is when we send an HTTP request across the Internet, there are potentially intermediaries here in the middle. Draw them in. For example, maybe this is the public Wi-Fi that you're using. It's the controller of that router and see your Internet traffic. Another one might be your ISP. Another one might be the cloud provider of the server. There's potentially many different entities that your Internet traffic is passing through on the way from your computer to the server. So in order to stop these different entities from being able to see all of the information that we are communicating privately with the server, we want to encrypt it. What that means is we need to encrypt. We need to encrypt over here on the client before sending the data. And then we want the server to be able to decrypt the data once it receives that encrypted packet of information. So how does that work? Well, we're not going to go into the math behind the cryptography. If you're curious about that, you can check out my Learn Cryptography course on boot dev. But from a very high level, there are essentially two keys. Two keys. Actually, I'm going to do them in different colors. This blue one is the public key. Public. And we'll do the private key. Red. Public and private. Now, the public key is used for encrypting. It's used for encrypting information. The private key is used for decrypting. It's important to understand that the public key can encrypt information, but it cannot decrypt it. That's solely the responsibility of the private key. The private key stays on the server, while the public key can actually be given to the client. Now, this would be good enough, right? Using the public key, the client can encrypt some information and send it to the server. Unfortunately, there's another direction that we forgot about. We also need to be able to send private responses from the server back to the client. HTTP is a request response flow. So actually, instead of using this public key to encrypt and send the request, this asymmetric cryptography, which is what public-private key cryptography is called, is actually used to negotiate a third key. So we actually generate this third key. This is a symmetric encryption key. What this key does is it allows both parties using this single key to encrypt and decrypt messages. So once both the client and the server have this key, they can share messages back and forth and encrypt and decrypt both of them. This public-private key section is only used so that the client can communicate to the server privately and figure out what this key is going to be. This key is effectively generated on a per-request basis. Let's review the order in which all of this happens just to make sure it's crystal clear. The very first thing that happens is that the server generates public and private key. This only needs to happen once for the server. It's done when the server is very first configured for HTTPS. If the server doesn't have a public-private key pair, it's not going to be HTTPS enabled. It's not ready for HTTPS traffic. So this only needs to happen once per server. Then when an individual request is going to start, the client contacts the server and says, hey, I want to send you a secure request. At that point, the server sends the public key to the client. So this blue public key is getting sent over to the client. Next, the client encrypts... Well, actually, let's write it a slightly different way. Let's say the client and the server negotiate a symmetric key. So from a very high level, the way that this works is the client has the public key. So it's able to send encrypted messages to the server. But remember, the server cannot yet send encrypted messages back to the client. So the client generates a secret token, sends it to the server, and then both parties using that secret token are able to generate the same symmetric key. So again, public key comes from server to client. Client generates a secret token. The client sends that secret token encrypted with the public key to the server. The server decrypts it using the private key. And then both parties are able to generate the same symmetric key so that they can both use that key for encryption and decryption. So once that symmetric key has been negotiated, now the client sends an encrypted HTTP request. Then the server is able to decrypt the request. And the server sends an encrypted response. And finally, client is able to decrypt the response. And that's the whole lifecycle. So this exercise talks about all of that HTTPS stuff that we just went over on the blackboard. I will point out really quick before we answer this question that SSL and TLS, they're the kind of the security pieces of HTTPS. So you could think of HTTPS as SSL or TLS tacked on to HTTP in order to add security features. So the question is, HTTPS is essentially the same as HTTP with some additional security. And the answers are true or false. This one's going to be true. Basically the same as HTTP, we've just added some security. I want to cover two more key points when it comes to HTTPS. So we've talked about how HTTPS encrypts what it is that we are saying when we communicate online. So that people snooping in on our internet traffic can't read the content of our messages. And it does that very well. One thing that it does not do is protect your identity or the identity of who you are communicating with. So if someone is snooping in on your network traffic, even though you're using HTTPS, they will be able to see the website that you're browsing on. Right. So if you're on boot dev, for example, and you're using public Wi-Fi, whoever controls that public Wi-Fi will still be able to see that you're browsing boot dev. That being said, HTTPS does ensure that you're talking to who you think you're talking to. So if you're connected to boot dev over an HTTPS connection, then you can be sure that you're actually talking to boot dev and not some malicious hacker that's just pretending to be boot dev. That's enabled through digital signature technology, which is, again, something you can look up. It's kind of out of the scope of the course. It's just important to know that HTTPS does give you that assurity if you're using it. So the question is, HTTPS ensures that you know who you are communicating with. And as we just discussed, the answer is true. Next question is also a true false statement. It says HTTPS protects your identity online. And as we just talked about, it does encrypt our messages, but it doesn't really protect who we're communicating with or who we are. So the answer is going to be false on this one. Congratulations, we finished all of the coding exercises and challenges in the course. Now we're going to be building a project from scratch. This project is going to be a command line application that runs on our local machines using Node.js. The purpose of the application is to crawl a website and produce a report of the internal linking structure of the website. Basically, which pages link to which other pages on the site. This is a tool that would traditionally be used by kind of web marketers or SEO search engine optimization experts. So I'm here on my personal GitHub account. And I usually start by just creating a new repository for whatever project I'm starting. So here I'm going to go ahead and call it web crawler web crawler HTTP. Project for the learn HTTP course. I'm going to go ahead and make it private. You can keep it public if you want, whatever you want. I'm going to automatically initialize it with a read me dot MD file. And that should be all I need to go ahead and create that repo. I am working on Mac OS, but everything we do should work just fine on Linux and Windows. You may just need to install things slightly differently. I'm also working in VS code, my editor of choice. But again, whatever editor you prefer will be totally fine. I'm going to go ahead and clone down that repo that we just created on GitHub. Now that we've cloned the project down locally, I'm going to go ahead and install NVM or the node version. If you just Google NVM, you should be able to find the GitHub page where you can download the or where you can find the installation script that I'm about to use. But I'll also include a link below in the description GitHub page. So I'm going to run that install scripts. And I already have it installed, so it doesn't even do anything. But just so you know, that's what you would run to install NVM. Once it's installed, we'll add a dot NVM RC file to the root of our repository. And inside, we're going to add the version of node that we want to be using, which in this case is going to be 18.7.0. You can use a newer node version than this, probably, but I wouldn't I wouldn't recommend going much older than 18. Once we have that NVM RC file created, we can do a NVM install. And that's going to download node version 18.7.0 if we don't yet have it. And then once we have it installed, we can do NVM use, and that will switch the current version of node that we're using in our shell. And then just to check, we can do node dash dash version to make sure that we're on the right version of node. Next, let's run a little program just to make sure everything's working properly. I'm going to create a main.js file, and we'll just have it log Hello World to the console. And then from the command line, we should be able to just run node main.js. So this starts the node interpreter and passes in main.js as the entry point. And it logs Hello World. So we're good to go. So running a node kind of manually from the command line, like node and then the name of a file, it works just fine. But because we're going to be installing a test suite and packages, we're actually going to use NPM to manage all of that kind of environment and ecosystem. So in order to get that going, we just run NPM in it. And it's going to take us through kind of a little interactive setup. And I'm just going to press enter a bunch of times to use all the defaults. And it will create a little package JSON file. And with a bunch of kind of default values, the only thing we need to change at this point is we need to add a new script. And I'll just call it start that runs node main.js for us. So now instead of manually typing node main.js into the command line, we will type NPM start. And that's just going to call this little script here. Again, the purpose of this is now that we're using a package JSON file, when we pull in dependencies and run our code through NPM, the dependencies will be available to our code. Similarly, we'll be using NPM to run our tests. So it's just nice and convenient to have everything in the dark command. Cool. And it worked, right? It logs out current version of node and then runs the program. Next, let's go ahead and install Jest. So we're going to be doing some test driven development as we build out this project. And in order to do so we need a test suite. So we're going to install Jest a fairly popular testing runtime. And we can do that by typing NPM, install dash dash save dash dev Jest. And that should update our package dot JSON file with a new dev dependency, as well as install the code in the node modules folder and create a package lock dot JSON file, which just keeps track of all the dependencies in our project. So there it is. That looks good. Like I said, the package lock dot JSON is an auto generated thing. You can just leave it same with node modules. But we do need to ignore. So let me create a new file called git dot git ignore. We do need to ignore this node module folders or this node modules folder. And the reason for that is the node modules folder contains all of the dependencies for our project. And it's actually quite a lot. Jest has a lot of dependencies. And so we don't want to commit that code to our repository. We just want the code that's kind of unique to our application. So we'll keep track of our dependencies in the package dot JSON, which is committed to our source control. But we are not going to commit the node modules any time we pull down this repository fresh. It won't have the node modules. So a developer will just need to type npm install in order to install all the dependencies listed here in the package dot JSON. And then they'll get their own copy of that node modules folder. So now that we have just added as a dependency, let's add a script that actually runs it. If we were to run npm, let me clear this out. If we were to run npm test right now, all we'd get is this error no test specified. That's because here in the test script, all we're doing is echoing error no test specified. So let's swap this out for Jest. And now if we run npm test, we get something a little different. It says no tests found exiting with code one. OK, so we're properly calling Jest now. We just need to add some tests. Before we get to adding some tests, let's go ahead and commit what we have to Git so we don't lose it. So I'm going to do a Git add dot. This adds everything in the working directory to kind of the Git staging area, ignoring everything that's ignored. So you can see node modules is in gray. That's because we added it to the Git ignore there. Then we'll do a commit where we say added Jest and then see and created a main.js file. And then we can push that up to Git. The first function that we write tests for is going to be called normalize URLs. And I'm going to create a file called crawl.js to write that function in. So usually when you're doing test driven development, there's kind of three steps. The first step is to stub out the function that you want to test. The second step is to write the tests for the function. And then the third step is to go back and actually implement kind of the meat of the function. So first, let's stub out the functions. It'll be function normalize URL. And as input, it's going to take a URL string, so a string representing a URL, and it's going to return a string. For now, I'll just return a URL string. Again, this is kind of a stub function. It doesn't really do anything interesting yet, but it allows us to see, you know, what's going in and what's coming out. Next, let's go write the test file. So it's going to be crawl dot test dot JS. And the way Jest works is that it just automatically looks for files in your package or in your folder project directory that end in dot test dot JS. So it is important that we name it with that suffix. Let's hop back over to the crawl dot JS file, and we'll run where we'll add a module dot exports. And this will make the normalize URL function available to other JavaScript files that want to import it. So now we can hop back over to crawl dot test dot JS, and we can import that function and be const normalize URL equals require and then the path to that file, which is crawl dot JS. That imports that. And then we also need to import two more functions, this time from Jest. They'll be the test and the expect functions. And so we can require. Jest. I think it's at Jest. Cool. Let's write our first test. So the way testing works with Jest is there's kind of this top level test function and it takes as its first input the name of the test. So I'm going to call it for now, just normalize URL. That is the name of our function, right? I'm not crazy. Okay. Normalize URL. The second input is a function. Let's go ahead and write an empty function. Within the function, we can use this expect function to do a test. Let's start by specifying what's the input to the normalize normalize URL function. For now, I'm going to just call it input and we'll just use an empty string. Next, I'm going to run the function and get some output. So const output equals. Normalize URL and give it the input. And actually, I'm going to change the name of output to expected. Just to be a little bit more clear. Sorry, not expected. I'm crazy to actual. This is the actual output of our normalize URL function. Next, I'm going to specify the expected output. So expected. What do I expect to come out of the function? And again, for now, I'm just going to use an empty string. Finally, we can use the expect function. So expect. Give it the actual the actual results of the normalize URL function. And then use the dot to equal method and give it the expected. So this line basically says, I'm expecting the actual output of normalize URL to equal the expected output that I've specified. If they do equal each other, just will log it as a past test, right? Test is good. If they don't equal each other, then just is going to say the test failed. Let's go ahead and run that and see what that looks like. So again, because we have specified here as a script, I can just run NPM test. And the test passed, right, which actually makes sense because we specify the input as an empty string and the expected as an empty string. And right now all our function is doing is returning the string that was given. So let's see if we can get it to break really quickly. Something else. I'm changing the expected value and running NPM test again. And now we get a failed test, which makes sense, right? We expected the string something else and we received empty string. So now let's talk about what the normalize URL function is actually supposed to accomplish. Sometimes on the Internet, there are different URLs that all points to effectively the same page. Let me give you an example. So something like HTTPS colon slash slash boot dot dev. This this string. It points to the boot to have home page. Effectively, this URL also points to the boot to have home page. They're really the same page, right? One has HTTPS specified, one has HTTP, but it's really the same web page, right? Similarly, if we capitalize the B again, it's a different string, same page. So what we are trying to do with the normalize URL function is find. Well, it's to take a URL as input and normalize it so that the output will be the same or all other inputs that are the same page. Right. So basically, if all these strings are put into the normalize URL function, we want the same thing to come out the other end. So, for example, this might normalize to boot dot dev. This also normalizes to dev and this normalizes to boot dot dev. Right. We want to normalize different strings to the same strings if they represent the same web page. So let's erase all of that. And let's build an actual test suite. So given the input, let's say HTTPS colon slash slash blog dot boot dot dev. I want to do a couple of things. The first thing I want to do is strip the protocol. OK, I don't care about the protocol. In fact, let me add a path here. Let's say slash. Let's just call it path. This part of the URL, the domain and the path effectively make up the page. Right. Query parameters, protocols, all that other crap. We don't really care about when it comes to what page does this URL represent. So given this input, I expect the output to just be blog dot boot dot dev slash path. Protocol is gone. I'm going to call this test normalize URL strip protocol. OK. And now if I were to run NPM test, I fail. Right. Because I have not gone and fixed my code up to actually do that. So let's go ahead and do that now. Top over to crawl dot JS. The way I'm going to implement this function is to use the built in URL constructor. We actually used it in the course. So I'll do const URL object equals new URL and pass it the URL string. So this will give me a URL object that has a few properties on it that are interesting. The first is the host name. So let's do return a new string template, template literal. And the first part of the string will be URL object dot host name. And the second part will be URL object path name. So we are parsing the URL string into a URL object and then saying, all I care about is the host name and the path name. And I'm just going to kind of shove them together and return all that as a string. Let's see if that works. We're good to go. The next thing I want to make sure that the normalize URL function handles is trailing slashes. So let me add a new test suite here for a new test to my test suite. And what I'm going to do is add a trailing slash on the URL slash right there. It's pretty common on the Internet to have URLs that end in slashes as well as URLs that don't. But generally speaking, we expect a page with or without the slash to both kind of go to the same place, right? Go to the same Web page. So what we're going to do is just trim trailing slashes in the normalize URL function. So if we're given a trailing slash, we'll just take it off. That way, again, everything comes out to kind of one string that identifies a given Web page. OK, so if we just run our test now with this new test, we're failing, right? Because we don't have any logic that strips that last slash off. So let's go ahead and add that. The first thing we can do is write a little test. We can say if URL object. Oh, excuse me, URL object. So let's grab the. We'll call this the post path. And we can say if post path dot length is greater than zero. So if there's at least something there and. Post path dot slice. Oh, negative one. So this this dot slice method on a JavaScript string, this will return just the last character of of the string. So given that there's at least one character, we're just going to go grab the last character and check. Does it equal a slash? If the string has a slash, then we want to return everything up to the slash. So we'll do return. Post path dot slice. And this returns a subset of the host path string from zero to the last index. So this is everything except the last character. Right. Slice at negative one is just the last character slice from zero to negative one is everything except the last character. Otherwise, if it doesn't end in a slash, we can just return this path. Cool. That seems right. Let's let's give that a run. Cool. We got it. So now that we're stripping trailing slashes and we're stripping the protocol, let's add just a couple more tests. Make sure that we are doing a few more things that are important in terms of normalizing these URLs. The first is capitals. Capitals. Oh, also, this is this is bad. We should rename this test. So this first test, sorry, strip protocol is good on this first one, because that's what we're doing. This one, we should say strip trailing slash. And this one, let's do capitals. So blog.boot.dev slash path. Remove that trailing slash. This this should not remain capitalized. Capitals in a URL don't matter. They should be case insensitive. So we expect to normalize it down to lowercase. Go ahead and run that and make sure it works. It does already work, which is awesome. I mean, you'll notice that we don't really have any logic that deals with lower casing a string. But the URL constructor is actually doing it for us because it knows that URLs are case insensitive. That's great. We made sure that that works. Let's add one more. Let's just make sure that another another protocol is also stripped properly. We'll just do the HTTP protocol and omit the S. Make sure that that works as well. Again, I'd expect that same output. I'm going to remove those capitals, though, just so we don't get confused what this test is about and change this name to be a strip HTTP. Cool. Yep, still good to go. Now that we've got that function tested and working, I'm going to go ahead and commit that source control as well. Well, normalize URLs, normalize URL. Sorry, it's singular. And push that up. Next, we're going to be adding another function called get URLs from HTML. And in order to write that function, we're going to need a new dependency. That dependency is JS DOM. So the interesting thing about this dependency, NPM install JS DOM, is that it's not a dev dependencies. You notice we're not doing the dash dash save dev when we install it. Maybe go ahead and run that. And it should add it to our package JSON. This time, again, not as a dev dependency, but as a normal dependency. And the differentiation here is that dev dependencies are just intended to be used by the developer. Whereas dependencies are intended to be used by the application when it's running in production. So we're actually going to be using JS DOM in our production code, crawl.js, rather than just our test suite. Which is where Jest is used. So let's stub out the get URLs from HTML function. We'll just add it right here above. Above normalize URL to get URLs from ML. I'm going to take two inputs. The first will be an HTML body. Just a string representing some HTML. And the second input will be the base URL. Again, another string. The base URL represents the URL of the website that we are crawling. So HTML body, HTML of the page, base URL, the URL of the website that we are in the process of crawling. And then it just returns a string. Excuse me, it doesn't return a string. It returns an array of strings representing URLs. So we'll do URLs equals empty array. And then for now, we'll just return that empty array. Again, we're just stubbing out the function so that we can go write some tests for it. Let's not forget to export the get URLs from HTML function down in the module that exports. And then over here in crawl.test.js, we can import it. Cool. And now let's add some more tests. The first test for this function, let's name it properly. Call it get URLs from HTML. Let's just stub everything out. So the first input, because there's actually two inputs to this function, right, is some HTML. So let's write some very simple HTML. Have an opening HTML tag. Do an open body tag. Add a link. We'll have the link just go to the blog. The anchor text of the link, we can just do boot.blog. Okay. Little snippet of HTML. Again, this is just a string. In JavaScript, when you use backticks, it allows you to do kind of a multi-line string. So this will be the first input. We'll just call it input HTML body. Then we need one more input, right, because our function takes two arguments. So we'll do const input base URL. And in this case, the base URL will just be that blog.boot. Cool. Now we need to call the function that we're trying to test, which is get URLs from HTML, and pass it in its inputs. So input HTML body and input base URL. Now we need to specify our expected value. So the purpose of the get URLs from HTML function is to grab all of the URLs or links embedded within an HTML page. So when we visit a web page, and there's tons of clickable links, we want to grab all of those clickable links out of the document and kind of return them in an array of strings. So taking a look at this HTML document, there's one link in here, right, and it's this link to blog.boot.dev. So we want this link to come back here in the in the return array. Okay. Now if you're not familiar with HTML, that's okay. It really isn't too complex. Let me explain what this is. So HTML is just a markup language that the browser uses to structure a web page. And this top level HTML tag is just saying, hey, this is some HTML. That's not too interesting. The body tag essentially says, hey, this is what you're going to see on the screen. And then a tags are links, where anything within the href property, right, in this case, log.boot.dev is is what we're going to link to. So when you click the link, that's where you're going to go. And then the text within the a tag is the kind of word you see that's clickable. So this would render to boot.dev blog as you know, that underlined clickable text. And if you click it, it would take you to the boot dev blog. So now that our test is written, let's go ahead and run the test. And I would expect it to fail, right? Yep, it's failing, because we're just getting back that empty array. And we are saying we expect an array with this with this value inside it. Cool. Let's go actually write the code so that we can pass the test. We'll be using the JS DOM package, right, the one that we already installed. And so I'll import it over here, JS DOM. The way I know how to structure the syntax is because I went and read the JS DOM docs, which again, I'll link to in the description below. But that's what I'll be consulting on the side over here. The JS DOM equals and then we'll just require. Why are the JS DOM package? So the way JS DOM works is it basically gives us a way in node to access DOM APIs or document object model APIs. So what that looks like is we create this little DOM object by calling new JS DOM and passing it some HTML. Give it this HTML body. So it's taking that HTML as a string and it's creating a document object model, basically an in-memory object that represents that HTML tree structure. So now we can just use DOM.window.document.get. No, no, query selector all. That's the one we want. And we pass in the name of the tag that we are interested in. So this will return some link elements, elements. OK, so this this syntax window document query selector all. This is the kind of syntax you would use in the browser to work with the document object model or the DOM. Again, this package JS DOM is allowing us to do that in node so we can write a command line script that interacts with with the DOM and crawls web pages. The reason we're passing in a here is because what we want is a list of all the a tags in the HTML. So this gives us a nice little array. So now let's loop over loop over that array so we can do const link element of link elements. And for now, let's just log these out to see what they look like. So a link element is an object that has a property and the property we're interested in is the href property. And that corresponds to the link text. So href, this should just be a string. Let's just log those out and see if we're on the right track by rerunning our tests. They failed again, which we expected. That's fine. Up here at the top, it does any console logs from the code itself. And yeah, it looks like it's logging out that that link text is exactly what we want. Now that we know that's working, let's just go ahead and append that to URLs. So URLs dot push that link text. Cool. So we should be iterating through all of the links in the HTML and appending them to that array and returning them. Let's see if that fixed our tests. Awesome. We're all green, all passing. Let's add another test case. As it turns out, we don't just want to pull out absolute URLs, but we also want to pull out relative URLs. So let's rename this because in this test, we are testing absolute URLs. And let's make a new test. Hard to work with VS code so zoomed in. And we'll make a new test where we test relative URLs. OK, so a relative URL is a URL that does not include the protocol and the domain. It just includes a path. So in this case, let's just call it path. And just to show you what I mean, I'll update this test to contain a path. It'll look like this. Now, down here, this relative URL is the same as this absolute URL up here, because the way the browser works is it looks at what domain you are on when you are viewing a page and all of the links in the page. If they are relative, which means they start with a slash, it will assume that they are kind of attached to the same domain. So if this is a relative URL slash path, and this is the base URL, then the actual URL when you click the link would be the same. It would be blog.boot.dev slash path. We don't want to bring this out. We don't want to extract it from the get URLs from HTML function as a relative URL. Instead, we want to kind of combine it into a full absolute URL so we can actually make requests to it later. So hopefully that makes some amount of sense. Basically, at the end of the day, what it means is if this relative URL is in the HTML, and this is the base URL, we want to pull it out as a full as a full URL looks like this. Okay, go ahead and run that test. And expect it to fail. Because, yep, we're pulling it out as relative URL and we want it as the full absolute. Let's go ahead and write some logic that satisfies that test. So the first thing I think we should do is kind of break down the logic tree based on whether we're working with absolute or relative URLs. So we can write a little test by saying if link element dot href dot slice from zero to one slash. So if the first character in the string is a slash, then this is a relative URL. Otherwise, we're working with an absolute. Now we know our logic works for absolute URLs. We can just throw that in there. For relative URLs, we need to do something just a little bit different. And we're going to want to push URLs dot push. The base URL appended to the relative URL. So we get an absolute URL. So we could do something like this. Base URL. Evaluate that properly. And link element dot href. So now we're concatenating them together. What happens when we run that? We passed. Next, let's add a test that pulls out multiple, multiple links. So I'm just going to copy this. In fact, let's have it do both a relative and an absolute URL. So we'll grab that full absolute URL here and we'll do, let's just call them path one and path two. And then we would expect to get back path one and path two. Take a look at that. Make sure I'm just going to say the blog path one and boot path two. Take a look at that. Yeah, that looks right. And I don't think we'll need to change our code to pass this test. Hopefully this just works. Yep, we're good to go. One last test for this function is that I want to make sure that if we're given a bad URL, that we don't include it in the extracted URLs, right? That come back from the get URLs from HTML function. So let's write an invalid URL here. Let's just say invalid, right? Invalid URL. Now, this is invalid because this isn't a relative URL. It doesn't start with a slash and it doesn't start with a protocol either. So it's not an absolute URL. So if this were rendered in a browser, this would just be a broken link and invalid. So what do we expect to happen? Well, given these inputs, I expect our function to not extract the not extract any any URLs. Let's go ahead and run the test and see if it works in its current state. Oh, we are failing, failing that test. Looks like we expected an empty array and we did get back that invalid string in the array. Let's go fix our code. One way we can test for URL validity is just to use that same URL constructor that we used in normalize URL. Go ahead and do that. So a const URL string. This same thing. Actually, let's do URL object. Let's just parse it directly into URL object. Now, if the URL constructor is given an invalid URL, so if this string is not a valid URL, this will throw an error. Okay, so it doesn't throw an error, we should be safe to push the URL objects href href property, which is just the stringified link. Right. However, if it does throw, throw an error, we need to be able to catch that. So let's do try. Throw all this in the try block and then catch error. And if we get an error, we'll just log it out. So we'll do console dot log. Error with relative URL. It will be error dot message. And the reason there's a dot message property here is because this is an error object rather than just a string. So that's how we access the error message. Cool. And then we're going to do a similar thing for the absolute URL. That of course, we don't need to parse it this way. Cool. This should work, right? If the URL constructor throws an error, we'll catch it. I should probably fix that copy pasta absolute URL. And let's go ahead and rerun the tests. Awesome. Looks like we pass now. And if I scroll up, yep, there's a console log there saying error with absolute URL, invalid URL. Now that we've got that all working and tested, let's go ahead and commit it. It was the get URLs from HTML function and tests. And I'll push that up. Let's get back to the actual application. So, so far, we've been writing these helper functions in the crawl.js file. And we've written some tests to make sure they're working. But our actual program still just prints hello world. So let's update that a bit. For now, what I think we should do is write a main function. So this will be the entry point to the application. And all it should do at this point is process command line arguments. So at the end of the day, our script is going to take as input a website and crawl that website. So we need to be able to take that input from the command line. And in Node, this is pretty easy. There's an object at the global level called process. And argv is a property that we can use to grab the command line arguments. So within the main function, let's go ahead and check to make sure that the command line arguments have been passed in properly. So if process, oops, process.argv.length is less than three. And we'll say no website provided. And if everything goes well, we'll just print console.log, starting, crawl. All right, let's test that really quick just to make sure it's doing what we think it is. So when I run npm start, what I'm expecting to happen, by the way, I should probably also like exit here or something. OK, cool. That same process object has a process.exit. And then we can pass it an error code. Yep. OK, so one is a pretty standard error code when things go wrong. So if we have not been given enough command line input, we'll log this and error. So let's just run that. See, yep, no website provided and it error. And then let's see what happens if I do provide. Let's just give it the name of my personal blog. Starting crawl. OK, perfect. So you might be wondering, why are you checking for length three? Wouldn't it be like one or something? Well, let's print out each entry so that you can see what's actually in that argv. So let's say arg of process.argv, console.log. OK, so what are the arguments? Interestingly enough, when I give it one command line argument, the length of this array process.argv is actually three. The first argument is the interpreter. It's kind of strange, right? It makes sense if you've done some more, I don't know, low level programming. But basically the first argument to any program is always the name of the program. That's just kind of how these operating systems work. So node is technically the name of our program because node is what's running and interpreting our code because we're using an interpreted programming language. The second argument to our program is the name of our code or the entry point file. In our case, main.js. So this is just that full path to main.js. And then the third argument finally is the one that we are actually passing in to our program. Let's fix up this typo, provided. And let's make our checking just a little bit more robust. So if it's less than three, we're going to say no website provided. But I also don't want people thinking that they can pass in, you know, a second website, right? Boot.dev or something like that. That's that's not going to do anything different. So we should probably add another check and say if it's greater than three or more than the standard amounts of arguments are provided, we'll say. Too many command line hargs and we'll exit. So by now, we should know that we have exactly three command line arguments. So the base URL that we'll be crawling is going to be process.argv at index two. Right. And now let's update this log statement to kind of echo that back. Say starting crawl of base URL. Cool. Make sure that works. Oh, yeah, I should probably provide a website. Starting crawl of Wagslane.dev. Perfect. Let's start actually crawling some web pages. So now we're going to get to the HTTP part of this of this project. Let's hop over to the crawl.js file, and we're going to create a new function called crawl page. And as input, it'll take a current URL. And to start, let's just log that we're crawling the page. So console.log actively crawling. Current URL. Cool. Next, let's make a fetch request to that URL. This is going to be a get request because we're hoping to get back some HTML. So const resp equals wait. We should probably await the promise, but we're using the fetch API. And then we don't even need to specify options kind of by default. Fetch does get requests. So we can just fetch that current URL. You'll see this red squiggly here. That's because this actually needs to be an async function. So it returns a promise. Oops. Async function. What do we do with the resp object? Well, for now, let's To start, let's just Let's just log out the response body. So this might be new to you. You'll remember that when we were working with the JavaScript, the JSON APIs, right in the course, we were often using dot JSON. And that's because we expected the response body to be formatted as JSON. Well, now we're expecting the response body to be formatted as HTML. So we're going to just parse it as text again, because we already wrote get URLs from HTML, which takes HTML as a string, as text, as input. So we can just kind of grab it as text. Cool. Let's run this. Actually, we're going to need to hook it up, right? We need to hook it up to main. So let's import Const crawl page equals require. And it's going to be Crawl.js. Here we can just crawl page and give it that base URL. OK, so let's rerun our application and see if we make that get request properly. And again, this is just my blog. It's a little website with about 20 pages on it. It's not huge. You can use my blog to test. I don't care. Or you can use a different website. If you are going to use my site, please don't DDoS it. I really appreciate that. But anyways, let's get going. Crawl page is not a function. I'm a big idiot. I didn't export this. Try again. Let's see. Starting crawl, actively crawling. OK. Is that what we expect? Did we log anything? I saw log res.text. Ah, yes, probably await that. This dot text method returns a promise. So let's await the promise and log it out. Awesome. OK, now we're getting some HTML. So this is all the HTML on the home page of my blog. Just a big fat dump of HTML data. So we have a problem. If a user types in bad input. So let's do npm start and then HTTP garbage URL. Just an invalid URL. At the moment, our code just kind of panics and dumps out the stack trace. We don't want that for a couple of reasons. The first is this isn't a very good user experience. We probably should give just a prettier error message and exit cleanly. When there's bad user input, but also as we crawl pages, because we're going to be crawling multiple pages with our website or with our with our tool, we don't want to encounter one bad link and have the whole process just kind of vomit and exit. We want to be able to handle that and keep moving and keep crawling more pages, even if there is an invalid link somewhere on a website. So let's add some try catch handling here. That will be step number one. This fetch call can error. So let's throw it within a try block and then we'll catch the error and log it out. Console dot log. Error in fetch. Page. All right, you are. Cool. And let's try running that. Well, much better. We don't get a huge stack trace. We just we just log out. I got a little message here. So you may have noticed this little warning here. That's because fetch historically was only available in the browser environment. And we're running JavaScript's now on the command line using node. Recently, node did add support for the fetch API for this fetch call as a built in. But it's still considered experimental. They could rip it back out. I think that's unlikely. I think it's likely here to stay if for whatever reason they did rip it out. We could always install node fetch, which is a package that does essentially the same thing. But in order to ignore this warning, because I'm aware of the fact that it's experimental and I don't want it telling me that every time I run my program, I'm going to come in here and pass the no warnings flag into the node interpreter. And so now if we run it. We don't get that warning. So we've checked for an error in the fetch call. But now I want to check for some more things because there are some other things that could go wrong. The first thing is the status code, right? We need to check and ensure that the status code of this fetch request is a 200 level code. If it's anything over, well, I guess 300 would be OK as well. We're OK with a redirect. But we shouldn't be in the 400s or above, right? The 400 level, 500 level error codes. So I'm going to write a little if statement here that says if resp.status is greater than 399. That would mean, again, 400 client error, 500 server error. Then let's log an error and say. Error in fetch with status code. Status code on page. That current URL. And if that if that happens, then we'll just return. We'll just stop crawling this page, right? If this page is broken, we're just done. We can just return from this function. Well, let's go ahead and make sure that is working. So if I if I crawl a valid URL like Waxlane.dev, we get the HTML. That's that's all still good. Now let's try a URL that doesn't exist. So I would expect this to result in a 404. So I'm just going to type in like a garbage path. Cool. Error in fetch with status code 404 on page garbage path. That seems to be working. One last thing I want to check just to be really sure that everything in this function is going off without a hitch is that we are actually getting HTML back. So when we parse the response body, right, does it actually contain text that represents valid HTML? And the way I'm going to do that is just by parsing the headers. So we can do this const content content type equals rest dot headers dot yet. And we'll grab that content. And then we could say if content type got my parentheses, content type does not equal text HTML, then we'll want to throw a similar log and say. On HTML fonts. Do not need to know response content type. Content type there. Content type on page. OK, cool. How do we test this one? So we want a valid URL that doesn't have an HTML response. My site map. Let's use my site map. So my site map is not HTML. It is XML. So if I do site map dot XML here, we should trigger this. Awesome. Non HTML response content type application XML on page. OK, and then let's just make sure that we didn't break another normal flow of things. Text HTML char set, you see a fate. OK, so what we should probably do then is maybe do an includes here. So rather than saying the content type is text HTML, because that header can have additional information like the the character set of the response. So it is valid HTML, but it's also giving us kind of an encoding type. So I'm going to change this to dot includes HTML. Now, as long as that string is in the header will be good to go. Run that. What did I do wrong now? Content type dot includes text HTML. Oh, it does not include text HTML, making all sorts of embarrassing mistakes. Cool. There we go. And just to be super sure, let's make sure that I didn't break that. OK, perfect. Now that we're grabbing HTML with some basic kind of error handling, let's go ahead and commit this. All page. Let's just say crawl single page with error. And we can push that up. Now it's time to update the crawl page function so that it can crawl an entire website rather than rather than just a single page. So instead of just accepting a current URL, it's going to have three arguments. There's going to be a base, oops, a base URL, a current URL, and a pages object. So the base URL is going to be essentially the starting point, right? The home page of the website. Typically, the current URL will be the page that we are actively crawling. And then the pages object is going to be an object that keeps track of all the pages we've crawled so far. First, we need to make sure that the current URL is actually on the same domain as the base URL. We don't want to crawl the entire Internet here. Whenever we encounter a link that links out to an external site, we basically just want to ignore it. So we can do that by using the new URL constructor, right? So we could say base URL object. Make that a const equals new URL, base URL. Do that with the current URL as well. And say if base URL, base URL object dot hostname. It's not equal. Current URL object dot hostname. Then we can just return, turn early. And actually, what this function returns isn't nothing anymore. It returns the pages object. That's going to be how we keep track as we call crawl page multiple times. It's going to be how we keep track of all the pages that we've already crawled. So we can return early and just return that same pages object that we received as input. Because, again, if these hostnames are different, we're going to skip this page. Next, we need to see if we've already crawled this page. So let's grab a normalized version of the current URL. So const normalized. Current URL. And we can just use that normalized URL function. Let me expand this so we can see more code. So we just call normalize URL and give it the current URL. So the way we can check if we've already crawled this page is if the normalized current URL exists within the pages object. The pages object is essentially just a map of URLs, normalized URLs, right? So we don't get duplicates to the number of times that we've seen that URL or that we've crawled that URL. So what we can say is if pages at normalized URL greater than zero, then we can just increment pages at normalized current URL plus plus and return pages. So we're basically just saying if I've already seen this page, I'm going to increment the count, right? The number of times I've seen this page. And this is so that when we actually go generating a report, we can tell the user how many times a certain page is linked to on the site. So if we already have an entry, we increment it and return. Assuming we've made it this far, we do not yet have an entry for the normalized current URL. So let's create it and we'll initialize the count to one, right? So this is the first time we've seen this URL or that we're crawling this URL. Next, I'm going to move this log statement down where the crawling actually happens. If any of this happens, right? If we're seeing an external link or if we're processing a link that we've already processed, I don't want to be logging this crawling message over and over and over again. I only want to log it when I'm crawling a new page. So after logging which page we are crawling, we'll go ahead and fetch the page just like we were before. But instead of logging out the HTML, that's something we definitely don't want to do, let's save it in a variable. response.html body response.text. And the next step will be to extract all of the links from the HTML so that we have more pages to go crawl. Luckily, we already wrote and tested that function. So we can do next URLs equals get URLs from HTML and we'll pass in the HTML body and the base URL. And let's say that's a const. Cool, now we can iterate over the next URL. So const for const next URL of next URLs. Now we're going to recursively crawl these pages. So if you're not familiar with recursive functions, our recursive function is just a function that calls itself. So we crawl the root page and then we find all the links in that page and then we crawl those pages and those may have links and we'll continue to crawl all of the pages on the website until basically these base cases are completely satisfied. Either we're just hitting links that are external to the site so we're not going to crawl those or we're just hitting links that we've already crawled. So we should just exhaustively crawl the entire website. So we're getting a new pages object because remember now this function returns a pages object. So pages equals await crawl page and we need to pass in the same base URL. But now rather than passing in the current URL, we'll pass in the next URL as the next current URL and then the current value of the pages object. So we're getting back a new updated pages object. We're giving it the old one. And then when we're done with that, so once we've crawled every page, then we can just at the end return the pages object, right? That makes sense to me. Let's go update main.js so that it actually works the way we'd expect. So now the crawl page function takes three arguments. To start, it will take the base URL twice, right? We'll pass in base URL not only as the base URL but also as the current URL because it's our starting point. And we'll start with an empty object for the pages object because we haven't crawled anything yet. That object will get filled in as we recursively crawl more and more pages. And then let's save the results in a pages object. And for now, let's just log those out. So for const page of pages, we'll just log out the entire page object. Cool. Let's go ahead and run this and see where it gets us. So I'm going to, again, just crawl my blog. See what happens. And we already screwed up. What do we got? For const page of pages. Pages is not iterable. That makes sense. This was, for some reason in my head, I thought this was an array. It is an object. So we need to do object.entries. That will allow us to iterate over the entries in the object. Okay. Let's go ahead and run that code again. Cool. Another issue here. See, you cannot read properties of undefined after hitting this XML page. Oh, you know what? We've got a return in here. Remember that we always want to return the pages object. So even when there's an issue, we still want to return the current pages object. We're relying on that. Okay. Let's fix that and try again. Cool. So it looks like it's crawling. That's good. We crawled a bunch of pages. We didn't see anything on the other side here. So when we iterated over this at the end, oh, you know what? RollPage returns a promise. Let's not forget to await that. And the main function also needs to be asynchronous. Okay. Let's try that. There we go. Now we're seeing some results. Okay. So we crawled through all the pages on the site. That's great. We even handled the site map. And this is what we got out the other side. So we have this giant list of all of the pages and how many times they are linked to. That's great. This is the data that we need now to do the last step, which is building a report of all this information. And before I forget, let's go ahead and commit all of this to Git. Don't lose our work. So crawl, page, recursive. Let's convert this kind of nasty just dump of JavaScript objects and arrays into a nice report. To do that, create two new files, report.js and report.test.js. That'll be one function we write that I think is worth writing some unit tests for. So the function that we'll be writing a unit test for is called sortPages. So function sortPages. As input, it takes that pages object and it's going to return an array. So for now, I'm just going to stub this out, have it return an empty array. Let's hop over to the test and let's grab a test. Give us somewhere to start. Instead of testing stuff in crawl.js, we'll be testing within report.js. And the function we care to test is called sortPages. So sortPages. All right. The input is a pages object. So let's build kind of a manual pages object. So it could be something like https://wagslane.dev. And remember, each URL maps to how many times that URL shows up as a link on the website. So let's just do three. So it's a very simple pages object. And then we'll call sortPages. And the expected in this case be an array. And it will be an array of, I believe we want it to be an array of essentially tuples, which are basically just an array of arrays where the first index in the array is the URL and the second index in the array is the number. So in this case, the expected would look something like this. And then the whole point of sortPages is it actually sorts the output array, putting the kind of highest count URLs at the top, lowest count at the bottom or first and then last. So let's add another entry here. Make this one a smaller count. So that should show up next in the array, right? We want highest to lowest. Should come after. OK, so hopefully that makes sense. Taking this object as input, we're converting it to an array and we're sorting the array. First index is the URL. Second index is the count. OK, that looks pretty good. Now, we haven't actually fixed up our function yet. So I'm expecting this test to fail. NPM run test. Cool. One test is failing, right? This is what we expected and we got. Should have got an empty array. Am I reading this wrong? Actual sortPages input. SortPages is not a function. OK, so I forgot to export, of course. Module.exports sortPages. Now we can actually import it and use it to make sure that works. OK, now we're failing for the right reason. We have unexpected. OK, cool. Now let's go ahead and write the let's write the sortPages function. So the first step is just to convert this into an array. So pages array equals object dot entries ages. Now, it's important to understand the way this function works is it will take the pages object and convert it into that structure that we're expecting. Right. And array of arrays where the first index is the key in the object and the second index is the value. So this gives us the structure we want, but now we need to sort it. So we can do pages array dot sort. JavaScript has a built in sorting function, but we need to tell it how we want to sort how we want it to sort all these entries. So A and B represent two individual entries in the pages array. Right. So two individual arrays and the value that we want to sort on. So A hits is A at one and B hits equals B at one. Remember, because index one has the count that we want to sort by. Right. Index zero has the URL. OK. And then what this compare function returns is a number. And because we want to sort from greatest to largest, we'll want to return B at one minus A at one. Essentially, the way the sort function works is whether it's given a kind of positive or a negative return value is how it sorts all of the individual of the individual entries in the array. I'd encourage you to go take a look at the documentation on the sort function if you're a little fuzzy on how that's working. But if we wanted to sort the other way, we could swap this. Right. B minus B. But because we want to go greatest to least, we'll do B minus A. Cool. And then we just return the pages array. Let's see where that gets us. Awesome. We passed that test. Let's go ahead and write another test just to be sure that the sort is working. Right. So we're only really sorting two pages here, two sort pages, two pages. Let's do something a little bigger. Let's do like five pages. And I want them all out of order in the input object. One, two, three, four. And let's just go way out of order here. All right. One, three, two, two. That's super out of order. Right. And what do we expect from this? Well, we expect nine to be at the top. Copy pastas of these. So path four. Nine hits at the top. Next, we'd have path two. Five hits. Next, we would have three hits and then path three. Two hits. And finally. OK. Let's make sure that test passes as well. Cool. We're good to go. So now that we have sort pages working, let's write one more function that will format the entire pages object as kind of that beautiful report in the console. So this one we're just going to call print report. Takes pages as input. And the first thing we're going to do is kind of separate. If you remember, our program, as it crawls, kind of writes some stuff to the console. So I think we really want to separate all of that text from the report itself. So I'm going to go ahead and log a bunch of equal signs to the console. And then something like. Report. Maybe another one of these. OK, then I'm going to get the sorted pages of cons sorted ages. Ages packed in the pages object. Cool. Then we could iterate over all of the sorted pages. So for honest. Or the page of assorted ages. And remember, each sorted page is an array so we can get the URL is sorted page at zero. Right. And honest. It's is sorted page at one. So the page URL and how many times it appeared in the site. How many times it was linked to. Right. Then we can log an entry and say something like. How should we reformat this? Found links to page URL. That. And then let's end the report. Similar kind of output. Cool. I think that'll look good. Now we can hop back over. Well, let's not forget to export this. Now we can hop back over to main.js. Let's import. Print report. And now, instead of this, this nonsense, we can just call report pages. Cool. After all that, let's go ahead and run it again. So NPM run start. Start up from the root of my blog. Right. Crawling. Awesome. Okay. So actively crawling all the different pages. That's great. Report starts. Sixty one links to the home page. Sixty links to tag. So it makes sense that these have lots of links because they're kind of in the menu of the site. So essentially every page links to those pages. And then these are links that are actually found kind of within the body text of the individual posts. If you've gotten this far, great. You've got a working web crawler. I'd encourage you to update and enhance your web crawler. Test it on different sites. See if there's any edge cases in the code that maybe we're not handling. And this will make a great kind of base project for you to build on top of and add to your portfolio. Of course, don't forget to commit your code. Print. Oops. Print report. Finished. Push it up to either GitHub or GitLab or wherever you store your code. Again, I'd encourage you to extend this project if you want to make it part of your portfolio that you're showing off to employers and that sort of thing. There's plenty of things that could be improved here, right? Maybe instead of logging the report to the console, maybe you create an Excel spreadsheet CSV format. But at the end of the day, the more personal you can make it and the more functionality you add to it, the more impressive it will look on your portfolio. We've written a lot of code. Good job making it through the entire course and the entire project. This course is actually just one small part of a larger back end development career path. If you are curious about building back end applications in Python, JavaScript and Go, then definitely go check out the rest of my courses over on boot.dev. If you want to stay up to date with all of the content I'm publishing, then there are three places I'd recommend you go check out. The first is follow me on Twitter at Wags Lane. The second is go subscribe to my YouTube channel, which is boot.dev. And then finally, I have a personal blog over on wagslane.dev and I'll include links to all of those down in the description below.

Channel: freeCodeCamp.org

Views: 790,773

Keywords:

Id: 2JYT5f2isg4

Length: 308min 47sec (18527 seconds)

Published: Mon Jan 30 2023