Linux Explained | Curl Command Explained

what's up guys it's josh from keep it techy and today i want to do a quick video showing you guys the curl command now as i stated in the intro the video i want to talk to you guys about the curl command it is a basic linux utility that's included in most linux distributions and is used to download and upload data from the web using various network protocols and the curl command is pretty basic to use but it does have a lot of options that can make it very complicated very fast so i want to go through and show you guys the basic understanding of how to actually use the curl command and how it can actually help you okay cool so i'm in my virtual machine and today i'll be using parrot os which is a linux distribution that's similar to kali linux it has all the cyber security tools that you can think of on it pre-installed and i just wanted to use this distribution just to show you guys a different distribution from my current desktop but we'll walk through the curl command and the first thing i like to do is show you guys the man page for curl now i know that curl is pre-installed it's pre-installed on most linux distributions but if you need to install it let's say you're on a debian system then you just use the app command to install it you can basically type sudo apps install and then the package name is curl and then that'll install curl and i'll run it right fast just to show you guys that it's already on the system and as you can see it's on the system so we don't have to install it and it's typically on most linux distributions at least what the most major linux distribution the curl command is already included as a default utility so the first thing i like to do is go to the man page for the curl command or whatever command i'm showing you guys so let's type man and then curl and this will give you basically all the details of the actual command as you can see the name is curl it says transfer a url uh this is the synopsis this is basically how you type it out so curl then you have your options and then you have your url and i want to show you guys the all these supported protocols that you can use with this actual tool as you can see it says curl is a tool to transfer data from or to a server using one of these supported protocols so they got dic t file ftp ftps gopher http https uh imap imaps uh ldap ldaps uh pop3 and a slew of other ones like smb smtp telnet smtp so you can basically download from a lot of different protocols and i just kind of want to show you guys at least a list so you guys can see all of the protocols that it actually supports it says the command is designed to work without user interaction so it's it also says curl offers a busload of useful tricks like proxy support user authentication ftp upload http post uh sql i mean ssl connections cookies file transfer resume uh metalink and more as you will see below the number of features will make your head spin so yeah like i said the curl command is very powerful with all the options that you can think of you know included with it that'll show you how to actually use it now uh just look through this it'll it'll go through everything all the different protocols it'll go through all the options down here at the bottom just to show you guys how to actually use some of it but i'll give you some base examples just to get you started with the curl command that way you can move forward and start using it now let's go down and press q for quits and that'll quit it out but one other thing i want to show you guys if you ever you know just want to see some information about the options and all that stuff that you can use with the curl command you can always type curl dash dash help and this will print out basically the usage and the options you know in a short list but that man command is where you want to go to look at the full details of the actual application now let's go down to clear right fast and show you guys the first way i want to show you guys how to use the curl command and that's just simply pulling down a website so i want to uh just type curl and then let's type in my website which is http uh colon forward slash forward slash and one one thing about the curl command you have to specify the protocol uh so you can't just type in like keep it techy.com or something like that you have to let it know the protocol to actually use to connect to this domain name so in my case i don't have a assert installed on my server which i need to go down and install a search so my stuff is just you know open but uh it's http and then keep it techy dot com and press enter and that'll actually pull down basically everything from the main page of my website and if you scroll through it you'll see is you know it's pretty much in html format this is basically the website and you can download this you know and actually write it to a file which is the next option i want to or the first option i wanted to actually show you guys you can actually save all this information to a file and i'll show you guys that next but let me go down go back down to the bottom but it pulls in basically everything about the website or the main page of the website and the way you actually save this information you can actually save it in html format or html file and i'll show you guys that now we can just type curl and then the option to actually write something to a file is dash oh and then we can specify that file name let's say we want to store it on let's say our home directory so home josh and actually we don't have to type anything we can just type out the file name and it'll store it in our home directory but i just wanted to show you guys at least the absolute path uh first but i want to store it in my home directory that way it's just stored there uh very simple to actually get to but home josh and then let's name the actual file so i'm gonna just name it k i t dot html and then let's go down and type in our website again so http colon forward slash forward slash keep it techy.com and press enter and that'll actually store everything in that file uh and just to show you guys right fast let's go into the file explorer let's go to places and actually we could just hit home the link on the desktop but there you go and we can actually open this within the browser uh and that's not the browser that's the text editor but we can open wix firefox and you'll see that it actually brings up my website so this is pretty much my website right here and let me close that right fast that's one of the plugins privacy badger but as you can see that's just my website right there and if we go to keep it techy dot com boom go to the website right fast it's the exact same you know what i'm saying brings in the images all it or points to the images the links and all that stuff uh so this is basically a clone of the website so this is a cool feature to actually get you know just get a website that you're trying to use the design for let's say you're a developer of websites or you're trying to design a website you can copy down or clone down the actual website and just have a local copy of it so you can look at it to reference whatever you're trying to do or reference the code as well uh it that's if you're you know uh need to look at the actual html of the site you know i'm saying you can download it and actually use it locally versus going to the web let's say you you know don't have internet connection where you're going this is a good way to actually have a copy of the website or at least the code for the page and you can use that to design other websites you you know trying to design but anyway moving on so the next thing i i want to show you guys is actually how to download files because that's the major way i actually use uh the curl command so let's go ahead and close these this right here as well and let's clear the terminal but one thing i like to do is actually download files using curl command and what i want to do is let's go to parrot os or parrotsec.org let's go there right fast this is the main page for parrot os and let's say we want to download the actual iso for this website or for this distribution well you can download it directly from here so i'm gonna go to security uh actually let's go back to the regular one get the home edition because it might be a little smaller i'm not 100 sure uh and then let's uh select the server and let's say we want to download parrot os and the closest mirror is right here in the us so all you have to do is copy that link and once you got the link you know i'm saying you can just use the curl command at this point so curl and then there's another option is basically dash capital o and then space that out and then drop in the link to the iso and then we press enter and i'll actually start downloading the actual page and as you can see it gives you a lot of feedback you know it tells you the total the percentage uh the total size so it's about two gigs in size this iso and this shows you the percentage and how much is received so about 11 megabytes so four average download speed 750 some k or whatever uh time total so forth uh time spent and then time left and then current speed so that gives you that information right there so uh let me go down and cancel this because i don't want to download it i already have it but that's how you actually download files uh and then also you can rename those files as you download them so let's say you want to uh rename that iso instead of it using the full name of parrot dash home that's 4.11.2 underscore amd64.iso you can actually specify the name of it that you want the iso to be once you download it or while it's downloaded so let's go uh dash o and then all you have to do is specify the name of the actual file or the name you want to name it and i won't put the absolute path we already know is going straight to our home directory so i'm gonna just type a name in there and i'm gonna put parrot os dot iso and you wanna make sure you specify the extension correctly uh otherwise it won't it won't look right once you download it or whatever it won't show as the iso you can easily go in and actually change it to iso if you mess that up but it's just just be wary and name it the right file extension that way you don't have to worry about it but anyway just put that uh and actually i gotta back that off and put something weird in there uh let's go down and copy this link address which is that same one from the site and let's go back i'm not sure why it's not pulling in the right one uh let's right click copy link and then let's paste that in there and press enter and that'll download that file and then i'm gonna cancel it now but i'm gonna show you these files and where they're located i locate i put them in my home directory like we stated earlier so all those both of those files should be in my home directory uh it's not the full file obviously uh because i didn't download the full thing it's only this thing is only 75 megabytes but this is the first one that we downloaded and it has the full name and then this is the second one we downloaded and i stopped it right away you know what i'm saying so those files are irrelevant we can't really use them anyway but i just want to show you guys that it goes through that same process you know what i'm saying and download the file it's just that dash capital o will actually allow you to rename the file to whatever you want it want to now let's go down in uh i'm gonna just delete both of those files we don't need them anymore go down close this and that's how you actually download files so pretty simple now we can get a little bit complex you know what i'm saying but this is the basics and this will get you started as far as using the curl command okay cool so it's another option i want to show you guys and that is how to actually get the response header information from the actual remote server that you're actually accessing and basically the headers just simply give you information you know about the requested page as well as the server you know and just some general information about this the server that you're actually connecting to so let me go down and uh run that now but it actually uses the i option and i'll show you guys this right fast uh let's just type curl and then i'm gonna use the dash lowercase i option and then i'm gonna go to let's say keepatech.com again so let's see http press enter and this will get the header information israeli as well as the body information or the page information now in order to exclude the page information which we you know as you can see that's a lot of information got to kind of scroll all the way up to the top um i just wanted to show you guys that one that it pulls everything now let's say you want to just look at the header information let's just type uh the same command but let's put a capital i up there at the front as the option press enter and this will pull just the header information that you want to see now as you can see it pulls in you know http 1.1 uh it's a 200 uh response uh it even shows you the server information so this server is actually running apache on ubuntu so super cool you can get that information you know i'm saying let's say you're a ethical hacker and you're trying to get some information about a website or a server uh then you know this is a good option to actually use and now this rolls right into the other option i want to show you guys is redirects and this is dash l capital l and so i want to use a different website for this because i can't really show it using uh keep it techy.com because i don't have redirect set up on my server so it's not a good example to actually show you guys so i'm gonna use ubuntu.com and i'll actually run that same command but i'm gonna run it against um ubuntu.com so let's type that in right fast and press enter as you can see it'll pull the header information as you can see it's kind of like a redirect it redirects to [Music] https colon four slash four slash ubuntu.com so this site basically has a 300 response which is a redirect 301 response that's a redirect so it's trying to redirect you to http as the secure website so one thing about curl is it will only do what you asked it to do so it won't just automatically go to it you have to use specify using the option of capital l so let's go down and run that same command and drop in the capital l and press enter and you'll see it'll pull in the first one it'll pull in the first one this is the first location the actual site that we were trying to get to the http version of the website which redirects to https and then once it pulls that information it also it redirects or it follows that link to the main page and then it pulls that header information for the main page so that's super cool that's super beneficial in case you know you don't know uh you can actually get it to follow those links or redirect to the correct page now let's say you want to get some connection details like for instance the tls handshake for whatever reason uh then i can show you another option right fast is the dash v option so i'll show you guys that on ubuntu's website as well i'm gonna go down in uh back use that same command just back off those two options and put v in there and this will bring in the connection information and this is kind of long you know well not really that long it only pulls in you know a little bit of information oh and that's right i'm still looking at the http uh version so let's go to the s version and you'll see that it's a whole bunch of information it'll pull in the tls information up in here but let's say you don't want to look through that i'm gonna just go through and uh and run a grep with it as well so i can kind of show you the tls information that it pulls so let's uh do a pipe and then grep and then let's look for tls and press enter and that'll bring in all the tls information as you can see it's kind of a short list uh but that'll bring everything in as far as the connection uh but only only those lines that have tls in it uh so as you can see this kind of pulls in the tls handshakes it gives you basically some information um this is the cert information you know what i'm saying and it has more details in there i'm just doing that grep so it's only pulling lines with tls in it but uh that gives you the connection information of the website so this is beneficial for people that are doing cyber security or ethical ethical hacking this is a way to get some information about a website that you're actually using and so the curl command is a very powerful command uh so i think you guys should check it out this is basically all i want to show you guys a few of the options but like i said this thing is very detailed uh and there are a lot of options and options in there to gather a lot of information about different websites as well as downloading you know files and all that good stuff but i hope you guys enjoyed the video please like share and subscribe to the channel and if you have any questions leave comments down in the comment boxes below and of course keep it techy [Music] you

Channel: KeepItTechie

Views: 430

Keywords: KeepItTechie, learn linux, linux, linux commands, linux command line, linux for beginners, linux tutorial, linux file system, linux terminal, linux tutorial for beginners, command line, curl command tutorial, curl command mac, curl command in postman, curl command to upload file, curl command line

Id: LP_qfztGhrU

Length: 20min 40sec (1240 seconds)

Published: Fri Jun 18 2021