Turn an API into a Startup?! Build & Sell an API with JavaScript

hello everyone and welcome to another javascript mastery video this one is quite special i'm going to teach you how to build and deploy your own fully custom javascript api from scratch more importantly in this one video you'll learn how to start selling it and turn it into a profitable business we'll go over the thought process of coming up with an api idea and then we're going to build it in this video you'll learn how to build a custom amazon scraper api once we build and deploy the api i'll teach you how to put it on a platform called rapid api set the pricing features start selling it and finally build a business around it all in one video this is a project for developers of all skill levels there are no prerequisites if you're a beginner you learn what apis are and how to create a backend javascript application using node and express and much more and if you're already an established developer you'll more appreciate the deployment and business aspects of this video with that said let me show you what will we build in this video i'll showcase the entire process from the idea to deployment on a real example we'll build an amazon data scraper api this api will parse all the valuable product data in the html response and return it in a json format of course this was just my idea i want to encourage you to think of something you've been wanting to create for quite some time now and do it by following my example that way you'll have something unique that you've created and you'll be able to put it on rapid api by following the same steps as you already know in this video i'm going to teach you how you can build your own api but let me know if you'd like to see it the other way around in one of the next videos i can go on the rapid apis marketplace find one of the apis somebody else posted and then build an entire application around it definitely let me know in the comments i don't want to take a lot of your time before we dive into the code so let me just quickly say that around 70 percent of you watching these videos are still not subscribed so if you like the videos definitely make sure to hit that subscribe button and if you like it like and comment for the youtube algorithm now let's dive right in before we dive right into the code let's quickly explain what an api is api stands for application programming interface and it allows two applications to talk to each other each time you use an app like facebook send an instant message or check the weather on your phone you're using an api so what exactly is an api when you use an app on your phone the application connects to the internet and sends the data to a server the server then retrieves that data performs the necessary actions and sends it back to your phone the application then gets that data and presents you with the information you wanted in a readable way and that's what an api is to explain this better let's take a familiar example imagine you're sitting at a table in a restaurant with a menu of choices to order from the kitchen is the part of the system that will prepare your order what is missing is the critical link to communicate your order to the kitchen and deliver your food back to your table that's where the waiter or api comes in the waiter is the messenger or api that takes your request or order and tells the kitchen the system what to do then the waiter delivers the response back to you in this case it's the food hopefully this makes some sense but here in javascript mastery we strongly believe that you learn best while doing so now allow me to show you what you will build today using node.js the most popular javascript runtime and express which is a framework for node which is going to allow us to create a server we are going to create a backend application or an api then we are going to deploy and host that api and put it on rapid api's marketplace where hundreds of thousands of people will be able to use it in this case i've built the amazon data scraper api let me show you what this api does amazon data scraper is the easiest way to get access to product price sales rank and reviews data from amazon in json format we have four different endpoints here one for the search results offers reviews and product details if you scroll right there you can see everything that this specific request can do for example we can pass some information and the product id based on that we're going to get all the information about that specific amazon product in this case it's going to be the apple macbook air then you can also see the schema of what you're going gonna get for each specific different api request that you want to make but we talked for quite some time now let's actually download node.js and build it out of course if you already have node.js installed you can skip this step but if you don't definitely make sure to download and install it once you install it you can head to your desktop and there you can create a new folder in this case let's call it something like amazon scraper of course you can call it something else or you can create a different project entirely once you've created it you can simply drag and drop it into your code editor of choice in this case we'll be using visual studio code once you're in there we'll be able to create our package.json which is a starting point of every application to create our package.json you can go to view and then finally terminal this is going to open up a visual studio code's integrated terminal we can run the clear command and then in here you can say npm init and then dash y this is going to generate the package json the package.json is looking quite empty right now it just has the name the version and the main file index.js which we're going to create in just a moment and then we have some scripts in here we can start off by removing the test script and we can create a start script we're going to use this for starting our application that is going to be equal to node index.js so we want to run the index.js file with our node command make sure to wrap both start and node index.js in double quoted strings let's install all the necessary dependencies by running npm install the first one is going to be express a back-end framework for nodejs then we can install request dash promise we're gonna need that for making api requests and finally we can install nodemon nodemon is going to reload our terminal every time that we make changes in our code let's press enter and wait for this to be installed once our packages are installed we can add that nodemon command right here we can say something like dev and then under that we can specify a new command which is going to be nodemon index.js we can add a comma at the end now we can use the dev command to run our application while we're developing it now that we are done with the setup we can right click right there and create a new index.js file this is going to be our api instead of here we can say cons express is equal to require express as we've mentioned a few times before we're going to use express to create a backend application then we can also require const request is equal to require and that is going to be request dash promise we're going to use this to make api requests now that we have express we can initialize our application by saying const app is equal to express and then we call it as a function just below that we can also create our port we can say const port all uppercase because it is a constant is equal to and that's going to be process dot env dot port all uppercase like this later on we're going to get a dynamic port from where we deploy our application but for now we can say or or or it's just one ore and that's going to be port 5000 finally below that we can say app.use and then in there we can say express dot json and call it that's going to allow our application to parse json input which is exactly what we want then every express application needs at least one route so let's create it app.get the first one is the initial route route and that's going to be just forward slash then we're going to have a callback function and then inside of that callback function we have our request and the response then we can return something we're going to have a function block right inside of here so what do we want to send back once somebody visits this specific url on our api well we can say res.send and in this case right here we can simply say welcome to amazon scraper api of course you can put anything you'd like here it's just a message to let us know that our server is running the last thing that we have to do to actually start this server is say app.listen we need to make our server listen on a specific port and would you look at that we already have a port we want to listen on so we can put that port there and express also allows us to create a callback function that's going to execute once we run the application so we can say console.log and then in there we can simply say something like server running on port and then we can use template strings to specify on which port are we currently running our application great this is the bare bones express application now you can run it by running npm run dev this is going to run nodemon and run our application and it looks like instead of running the app we broke it from what i've noticed the request promise library requires one more dependency and that is just the request dependency so let's install it you can stop your terminal from running by pressing ctrl c and then y and then instead of here you can say npm install and then specify request this took just a second to install and now we can run our app by running npm run dev as you can see our server running on port 5000 in the next part of the video you'll see that i have just a request right there but that's a mistake and it's going to be fixed later so just keep the request promised here at all times great now let's check it out in the browser to see it in action just open up your browser and go to localhost 5000 and just to see it a lot better i zoomed this page all the way to 500 there we go welcome to the amazon scraper api we've successfully created a node.js express backend javascript server and now we're going to add the routes that are going to give it additional logic let's do that right away to scrape useful data from the amazon's website we're going to use a service called scraper api this tool allows you to scrape html off of any site and then turn it into useful json information you can start with scraper api completely for free you get 5 000 free requests which would be more than enough for a demo application if your own api also uses web scraping and you need just a few extra requests definitely make sure to use the link in the description there's a discount code with that said let's sign up and create an account you can sign up using google or github and once you're in there you're gonna get to a page that looks like this the most important part in this page is the api key you can copy it and then we can go back to the code for now we can put it right here by saying const api key is equal to a string off and then we paste that api key we just copied now that we have our api key we can close this terminal and let's create our base url we can say something like const base url let's make that a template string and let's start with http colon forward slash forward slash and that's going to be api dot scraper then we can add a question mark and say api underscore key is going to be equal to and then you can use the dollar sign and curly braces to provide a real api key in there finally we're going to add one more query parameter by saying and and then we're going to say auto parse is equal to true great now we have our base url and we can make our own routes from our api to fetch specific products so the first route is going to be for fetching product details so let's add a comment get product details that is again going to be an app.get and the url is going to be forward slash products forward slash colon product id this column here means that the product id is going to be dynamic finally we can add a callback function this time it is going to be async and it's also going to have a request and a response we can expand that function and we can first get the product id from parameters whenever you have a colon something you can say const specify the name of that something which is in this case product id and that is going to be populated inside of request dot parameters now that we have that we can open a try and catch block inside of that block we want to get a response from scrapper api const response is going to be equal to a weight and then we use that request that we imported at the top inside of the request we can provide our first url that is going to be equal to a template string in there we can specify our base url that we specified at the top right there and then we can add some extra parameters we can say end url is equal to and then we can say https colon forward slash forward slash and this time it is going to be www.amazon.com and then forward slash dp dp is maybe product details and then forward slash and then in here we have to provide that product id this is going to give us information for a specific product finally we just want to send that response back from our server so we can say res.json and then inside of there we can provide that response also for the error we can do res.json and then in there we can provide the error our first route should be done let's save it and take a look in the browser we are back on localhost 5000 but now we have to go to forward slash products and then we have to go to forward slash and then a specific product id so what exactly is this product id and where can we find it i've just opened amazon.com and i'm gonna search for something like a macbook air and then let's open this 2020 apple macbook air there just in the url you should be able to see vp and then we have our product id you can find that id for every single product now that we have it let's go to our localhost 5000 and simply paste it right there and would you look at that our data is right here but right now it's quite unreadable so let's go back to the code and fix it our data is currently in a text format so we can run json.parse and then parse our response just before we send it back let's save it and then check it one more time and this already looks so much better for you it might still look like this if it does make sure to install json formatter then you can parse it and then in the browser it's immediately going to look like this extremely readable so let's see what do we have here we have the product name product information the weight product dimensions language brand basically everything pricing even images you can expand them and then you can click on a specific image and get all of the images from the amazon listing and the beautiful thing about this is that other developers can use this data to create a website that's going to showcase these products they can create basically an amazon clone or something entirely different but they can use the data and that's where apis come into play and i just want to emphasize something one more time this is just my little api that uses web scraping you can create an even better one and it doesn't have to be a scraping api you can do something completely different you can use your own database and then serve the responses from there or for example take a look at the top 50 apis api for flight data for weather information football finance love calculator hearthstone currency exchange news booking email validation you can create your own algorithms that are gonna serve as apis movie databases jokes anything you can think of you can create an api and then upload it to the rapid api marketplace with that said our first endpoint is done and what would an api be with just one endpoint we need to have more current one is giving us product info now let's create two more endpoints one of which is going to give us more offers about a specific product and the other one is going to give us detailed reviews let's do that right away to create the second route we can simply copy the first one right there and this one is going to be get product reviews we need to change the url we can still leave it as product and then product id but we can add another forward slash and simply say reviews this is going to remain the same we still need a product id and the response is going to change just a bit let's move to the right this here is going to be amazon.com forward slash and then instead of db it's going to be product dash reviews and that was easier than we anticipated let's check it out to go to this route we can simply add forward slash reviews onto our url bar and let's wait a few seconds and check it out it took just a bit because there are a lot of reviews to parse but in here we have detailed information about the reviews how many five star ratings how many four star ratings and so on and we can also see the top positive reviews how many stars the date of the review and finally the entire title and the review data we can even sort it by top positive top critical and we also have access to all of the other reviews beautiful now on to the third one and that is product offers we can again copy this one and then change the comment to say get product and this is going to be offers we can still do the same thing just change reviews to forward slash offers this will no longer be product reviews we can add gp and then forward slash offer dash listing let's save it and check it out in the browser to check it out we just have to change this from reviews to offers and here's our data unfortunately our macbook air doesn't have any current offers and that's why this is empty but that's fully okay some products will have that and some will not now let's go on to the most important thing and that is implementing the search query so if you go back you'll notice right there we are searching for macbook air and there are many things that we got back maybe we want to display that at the top you can see search question mark k is equal to and then we have our term let's implement that to implement that request we can again copy the previous one this time we can say something like get search and that is going to be get search results this route is going to be just a bit different we no longer have products we're going to have something like for search and then forward slash collin search query we need to know what we're searching for then again opposed to getting the product id we're going to get the search query from reg.params finally we have to modify our request just a bit so right here instead of offer listing gp we can simply say s question mark k is equal to and then in here we can provide a search query so one more time s for search question mark k is equal to and then the search query to be completely honest i'm not sure what k stands for maybe you can let me know in the comments great but this is how the request should look like we can save it and take a look we can enter the url for our newly created endpoint and that is going to be search forward slash and let's search for something like macbook air there we go our data is back we even got the array of ads so if you want to hide them well now it's so easy to do that you can use this api on your own website and then just display the results great so as you can see we have many different results here first this 2020 apple macbook then we have the 13-inch version and so on but generally we got back all of the results that have something to do with our search query great all of our four endpoints are created and with that our api is almost done before we go ahead and host this application on heroku there's one more thing we have to make or rather there's one more question we have to ask ourselves do we want to share our own scraper api key to everybody using our own api key so do we want to share it or do we want everybody to also need their own scraper api key in this case i'm going to comment it out because i don't want it to be publicly visible and we're going to require everybody that wants to use our api to also get this api key now that this is commented out we need to change this url because it's no longer going to be static so we can do something like const and say generate scraper url something like that and that's going to be a function that's going to accept an api key and based on that it's going to return a string similar to the one we've had so far we can basically copy our base url and delete it the actual url is going to be the same but notice that now api key is not our api key but rather we're passing it inside when we call the function so the question is where do we call this function and the answer is with every single one of these responses instead of the base url what we want to do is we want to call the generate scraper url that's going to be a function and into that function we need to pass our api key but now you might be wondering where is this api key coming from we're going to add query parameters to every single one of our requests so later on in rapid api marketplace users will be able to say question mark and then specify their api key like this and then that api key will be populated inside of our request.query so we can get that right inside of here const api underscore and key and we're getting that from rec.query great now you can see we're getting the api key from here and we are simply putting it right at the start of our url that way everybody will be able to enter their own api key now we have to copy that for three requests below so i'm going to copy it paste it right here right here and right here and we also need to change our base url to call generate scraper url we can do that by selecting base url doing ctrl f pressing this little arrow right there and then we can copy this specific function call put it under replace and then simply click replace all that's going to replace our base url with the generated scraper url with the dynamic api key if that control f didn't work for you you can simply paste the scraper url instead of every single base url great with that our application is ready to be deployed to heroku last thing that you can do is remove this api key from here and store it somewhere else our app is ready to be hosted on heroku let's do that right now to host your application you can of course use anything you want like hostinger aws or heroku from my experience it's extremely easy to quickly get apps up and running on heroku so that's what we'll be using today to start you can sign up or log in at the top once you do that you'll be on your dashboard as you can see i have quite a few and we're going to open a new one so let's click create new app and let's enter a unique name jsm amazon scraper great you can choose your region and click create app if you scroll a bit down you'll be greeted with all the instructions on how you can deploy your api first you have to download and install the heroku cli after you click the link you'll be created with all the download links right inside of here i already have it downloaded so i'll see you on the next step after that you can see the into your project i think we're already there and then you can run git init i opened our app side by side so it's easier to see and i'm going to stop our server from running by pressing ctrl c and then y you can clear the terminal as well now let's run git init once you're in there copy the next command heroku git remote there we go it's going to create it before you run git add definitely make sure to open your file explorer and add a file called dot git ignore in there we can add our node underscore modules to the ignore list that way they're not going to be pushed to heroku which is exactly what we want when that is done you can run git add and finally git commit and at the end get push heroku master that is going to build out and deploy our server to heroku once that is done you can expand your browser go to overview and then in here you should see build succeeded finally you can click open app and as you can see on our own custom domain jsm amazon scraper it's going to be different for you you'll see that our api is live which means that it's not only on our localhost now it's accessible from the web and here we are to get to rapid apis marketplace click the link in the description it's going to be somewhere around the top so i've promised you something in the beginning and now i'm going to deliver it you you've already built the api you've already hosted it and now is the time to deploy it on the marketplace not only am i going to show you how you can put it on the marketplace i'm also going to show you how to set the pricing fears and finally start selling it and again you don't need to deploy the amazon scraper thing build something your own now you know the entire process i'm going to show you the final things and then after this video you can start creating your own api and then put it right here as well so let me show you how to do it you can visit the top right corner and click my apis that opened up the dashboard for me and as you can see i already have the amazon data scraper but in this case i'm going to create one more just so i can show you the entire process so i'm gonna go to the top left add new api inside of here you can enter your api's name i'm gonna do javascript mastery amazon data scraper and you can enter a short description in this case i'm just gonna copy the sentence i've had before you can also select the category in this case this would be ecommerce and finally now we come to a really important part and that is how do we want to specify our api we can use rapid apis ui open api standard postman collection a graphql schema or kafka in this case we're just gonna use built in ui so let's click add api in here you can enter a long description if you want to and you can upload the image i've just uploaded the amazon logo in here you can enter the website url in our case that can be our api so you can just copy it and paste it right here finally if you have some terms of use you can enter them right here in my case i'm gonna simply click save great we're done with the first part now we have to add a base url we can click configure right here and paste it right inside of here our base url is the same one we've entered to our website and that is our api that is the website that we're hosting on heroku we can add that as a base url and click save of course we also have to specify four of our endpoints so let's create them one by one let's create a rest endpoint and we can start with our get amazon product details i would definitely advise you to enter a lengthier description in here in my case i'm going to simply copy what i had in the name if you have some external documentation links you can paste them right there and then finally you can specify the endpoint in our case that's going to be forward slash products and then finally product id in this case you don't have to put a column right there it says use curly braces to indicate path parameters great so this is our get for the product details a great thing about rapid api is that the api you wrote will be accessible through most of the most popular programming languages that's crazy as you can see they recognize we have one path parameter and we can add the example value and in our case that was the id of the macbook air you can put any random amazon product id finally we can click save to test endpoint and test endpoint they're going to give us an example response and right now it's going to be 401 unauthorized because we haven't yet specified the key but remember i've told you there's going to be an easy way to specify the api key and that is through the query inside of here we can specify api underscore key that is going to be a string and then in here you can choose whether you want to show your key as the example value i definitely wouldn't recommend to do that because then everybody will be able to see it but since this is the demo example i'm going to leave it here and i'm going to make it required because people need to have their api key to be able to use this api and there we go product data is here you can click right there to copy it and then go to example responses select the response code which is going to be 200 and add response code finally we can add our example response so that people can know what they're gonna get even before they start using the api not only that we can also generate a schema so that way people can assume what each amazon product is going to have even before they make a request great now this is done and let's fill in some details in here we can say apple macbook air product details and also for the example we can say macbook air product details just so that people know what the example request is about finally let's click save and there we go one endpoint is successfully added i'm not going to waste your time and create endpoints for all the requests that we have i know you're eager to start creating your own api just after this video so i'm just going to add one more and that is going to be our get amazon search results great so let's scroll down we can specify our url here that's going to be forward search and then inside of curly braces we can say search query for our search query we can also enter a default value let's go with macbook air again and then finally in the query as you know it we can specify our api key and again you can choose to put it here or not finally if we scroll down we can add some example requests so let's go here to example responses choose the status code of 200 which means that everything is okay and i'm going to add my api key just so i can test this endpoint okay i've clicked test endpoint and at the bottom we have our results i'm going to copy them one more time go to example responses and then simply paste them right here and also click generate schema that way people will be able to know what they can search for every single amazon term now let's save it and with that said we should have two endpoints already done before we go ahead and make our api public let's go ahead and check some of the other tabs right there security has a lot of options right here you also have global settings docs announcements and finally plans and pricing that's one that's quite important for now we'll try to keep it simple and price our api by the number of requests that people make in our case a call to any endpoint is considered a one request so you can scroll down and then in here you can add your plans for example for our basic plan we can click edit and say that people can make monthly around 8 000 requests and we're going to set a hard limit that means that if they go over it's simply going to be blocked great we can click save and with that we have a free basic plan now let's add a pro plan it's going to be the same thing we're going to have requests monthly and in this case let's do something like 10 000 and then we can even allow them to go over that and specify that for example each extra request is 0.01 dollar so 1 cent great we can save that and let's add just one more plan in this case this is going to be for huge users we can add requests and then monthly they can make 50 000 requests and we're also going to charge them one cent for extra requests above 50 000. great let's save it once you've added all of your plans you can click preview and there we go this is how the pricing plan is going to look like people will be able to come here subscribe to your api and rapid api will handle the rest with that said we are ready to make our api public let's click right there and switch this from private to public and this is a demo api i'm making but if you're making your own make sure that your api is fully documented working and it's going to become accessible to hundreds of thousands of developer on the marketplace let's make it public and finally on the top right corner you can click view in marketplace there we go our api was hosted in a matter of seconds it's right there accessible to people you can visit it by url and people can actually test it before they start using it as you can see here are two endpoints we've added product details they can immediately see what do they have to enter the api key and the product id and they can test that endpoint we just got the results but we can also see the example responses the entire body and the schema of a response that's great and finally let's try one more with search as you can see people can test the endpoint with any search they want so let's search for something like a keyboard there we go test endpoint and just by entering a search query we are immediately greeted with json data on all of the newest keyboards great in this video you've learned a bit about what apis are you've built an amazon data scraper api hosted it and deployed it to rapid apis marketplace now as i've already told you i strongly believe that you learn best while doing something yourself so after watching this video definitely make sure to think of an idea and create something yourself and then try to replicate the process from creating a backend server hosting it on heroku and also putting it on rapid api if you like this video definitely make sure to like and leave a comment it means so much and also it helps with the youtube algorithm also let me know if you'd like me to create a video using apis but this time the other way around i would go here find an interesting api somebody else has built and then build an entire application around that specific api feel free to browse a bit and let me know which one would you prefer that was it for this video thank you so much for watching subscribe so i can see you in the next one and have a wonderful day [Music] you

Channel: JavaScript Mastery

Views: 121,959

Rating: 4.9695215 out of 5

Keywords: javascript, javascript mastery, js mastery, master javascript, javascript tutorial, javascript tutorial for beginners, javascript tutorials, build an api with javascript, node js api tutorial, node js api tutorial for beginners, node js api development for beginners, node js api development, api development in node js, node js rest api development, rest api node js express, node js rest api tutorial, build an api, build an api with node.js, rest api explained, rest api node js

Id: be9sHQ7xqo0

Length: 40min 15sec (2415 seconds)

Published: Fri Jul 09 2021