Linux Journey - Text-Fu (Basic text manipulation and navigation)

hey what's up everybody so today i'm going to be going on a linux journey so this is a quick rundown it's just a website called linuxjourney.com and they just start you off with linux like they pretty much tell you everything just to get you started and they go from grasshopper to journeyman to networking nomad and just some recommended books down here and so i'm going to be going through this to help myself go through linux and potentially help me learn faster because i've been reading but sometimes i kind of just don't really want to read like just a plain boring technical book so i'm going to check this out and see if i like it so i already looked through this a little bit i didn't do anything but i wanted to see where i was at at least in terms of the channel so i'm in the text foo area and so just to do a quick rundown of where over everything that i did so basically gives you a history of linux um and then it says choose a distribution and i'm going to be using zorin os which is an ubuntu based distribution also has some debian because ubuntu is based off debian and then you pick from one of these and you can see it has a quiz on the other side so this would be debian and you could check your answer then hit continue so yeah and then the command line i've pretty much shown all this i did print work in directory change directory ls touch file cat i pretty much did all of this um i didn't do alias but basically this is where you can make different shortcuts i guess for commands and exit but yeah so pretty much i'm at text foo so i'm gonna start this and let's see if i like it all right so we're here in text foo and the first lesson we're going to learn is uh std out or standard help so i'm just going to like select the section and then i'm going to close it so we can have more space to read so let's begin okay so this uses the text um file creation and stuff like that with echo okay so we're going to do echo hello hello world and then the greater than sign peanuts.txt okay so let's hit enter all right so it says to check the directory and we are in just the slash home slash green hat directory so let's do an ls and we can see the peanuts.txt and it says to look inside that file and we should see hello world so let's do a cat peanuts.txt and we have hello world inside the file cool all right so it says we're going to break it down we did the echo hello world so just a basic echo hello would just print the hello in the terminal okay so it says the default one we printed to the screen as like right here all right so this redirection right here allows us to change this default behavior by giving us greater file flexibility so the arrow or the greater than symbol is a redirection operator change where the standard output goes when we created the peanuts.txt since it didn't exist it will create it for us which it did and if the file does exist it'll overwrite it okay so it says well let's say i didn't want to overwrite my peanuts.txt luckily there is a redirection operator for that as well okay so it's just a double arrow so go hello world peanuts.txt now we hit enter and it says this will append hello world to the end of the peanuts.txt file so this should show up at like the bottom of hello world so let's do a cat peanuts.txt and we see that it's at the bottom of the first hello world okay what redirect did you use to append to an output file output to a file all right okay so now we're doing stdn or standard in okay so we're going to use the file peanuts.txt that we created before and it had the hello world inside of it so let's do a cat peanuts dot text and we're going to do the standard out into banana.text okay so it says in the cat command you send a file to it and the file becomes the stdn in this case we directed pns.txt to be our stdn then the output of cat peanuts.txt which would be hello world gets redirected to another file called banana.txt so if we do cat banana.txt we can see that it has the same contents as peanuts.text okay so i missed the exercises in the last one but i'm gonna do them here so it says echo let's go copy this [Music] echo standard in peanuts.txt standard out banana.text okay so let's cat that and we have nothing okay so now let's do the ls and so this one alright so this one just took the ls and printed the result inside banana.text and this one took the printing working directory and printed the output inside of banana.text all right so standard error or stderr all right so it says we're going to list the contents of a directory that doesn't exist on your system and redirect the output to penis.txt so we're going to do ls as fake fake directory and then we are going to put that into or what's it called standard out that into the peanuts.txt okay so it says it cannot access the fake directory it's cat peanuts okay so nothing is inside of peanuts.txt and that's because it doesn't exist okay so this message didn't show up inside of peanut.txt when we did that because there's another io stream and it's called standard error by default standard error sends its output to the screen as well it's a completely different stream than standard out so you'll need to redirect its output in a different way okay so we're going to need to use file descriptors okay so has to be a non-negative number and standard in respectively so standard n is zero standard out is one and standard error is 2. okay so in the old command we're now adding a 2 all right so in the old command we're adding a 2 before the standard out to peanut text all right so now we do a cat peanuts and what's inside the file is the error message that we got before okay so now it says what if i wanted to see both standard error and standard out in the peanuts.txt file is it possible to do this with file descriptors as well okay so it's telling us to do the original command and then we're going to do two for the standard error standard out [Music] and the standard in so it's saying it's sends the result of the fake directory the listing of the fake directory to peanuts.txt and when we did that before we got something empty because it didn't exist and then it redirects the standard error to the standard out via two okay it says the order of the operations matter okay so this operation sends the standard error to whatever the standard out is pointing to in this case the standard out is pointing to a file okay so the operation also sends standard error to a file so if you open up pnts.txt you should see both standard error and standard out in our case the above command only output standard error okay so and now it's saying there's a shorter way to redirect both standard out and standard error to a file so what it's saying is let's just put that hand symbol before the standard out symbol okay i'm gonna cut that now what if i don't want any of the craft and want to get rid of standard error messages completely well you can also redirect output to a special file called dev null and it will discard any input okay so what is the following command doing okay so directory and then that was the appending for the standard out and then that operation was putting in the standard error and the standard output okay the redirect of our standard error um i believe that's it yeah okay all right so pipe and t all right so we're gonna do some terminal plumbing all right so it's telling us to list the long list and all the hidden files inside of slash etsy and we're going to get a very long list alright so we're going to be using the less command so let me just skip this so i don't have to see it on my screen and then i'm going to type it into less and so we should be able to scroll up and down with our arrow keys our page up and down yeah so it doesn't start all the way at the bottom like it normally would it starts from the top and we can scroll down with the arrow keys and then press q to quit out of that all right so the pipe operator allows us to get the standard output of a command and make that the standard input to another process so we took the listing of the directory at c and then piped it into the less command and this command is extremely useful it is i use it right so it says well what if i want to write my output of a of my command to two different streams that's possible with the t command i'm going to do what is t read from standard input and write to all right so let's do ls pipe t peanuts.txt and you should see the output of ls on your screen and if you open up the peanuts.txt okay so it takes the ls and then puts it inside of peanuts.txt okay so we're going to do the exercise so this is ls and then we're going to pipe with the t command peanuts.txt and banan.text okay so we get the ls again on the output we're gonna do ls and now we have the banan so let's do a cat banan dot text okay so it just created this but with a second file so what key represents the pipe operator [Music] okay all right so we're doing the environment command so run the following echo home and so echo home displays our home directory echo user displays our user and the command env runs a program in a modified environment let's do an env and believe this is okay so these are a bunch of environment variables these variable variables contain useful information that the shell and other processes can use okay so it's telling us that an important variable here is the path variable which is right here and if we do echo path [Music] we can see it's the same as over here returns a list of paths separated by a colon that your system searches when it runs a command okay so it's saying when you put a package inside a different directory that it usually doesn't go in you're going to put that command in and it's not going to be found even though you know it's there and so it's saying that the path variable doesn't check that directory where you put that command so it's throwing an error okay so it says let's say you had tons of binaries you wanted to run out of that directory and you just modified the path variable to include that directory in your path environment variable okay so when let's say you move a command to a different directory you can just add it to your path okay what does the following output it outputs our um home directory of the user how do you see your environment variables env alright so now we're using the cut command remove sections from each line of files we're going to learn a couple screws process text before we get started let's create a file that will be working with copy and paste the following command add a tab in between lazy and dog hold down control v plus tab okay copy and paste once you do that add a tab in between lazy and dog so between lazy and dog hold down control v and tab all right i guess we hit enter so let's ls all right so now we have sample.txt first commandment learning is to cut command so it it extracts portions of text from a file to extract contents by a list of characters so cut okay okay so cut tax c characters select only these characters okay so cut select only these characters 5 sample dot text this outputs the fifth character in each line of the file in this case it's the queue note that the space also counts as a character okay to extract the contents by a field we'll need to do a little modification okay so cut field to sample.txt the f or field flag cuts text based off fields by default this uses tabs as delimiters so everything separated by a tab is considered a field you should see dog as your output you can combine the field flag with the delimiter flag to extract the contents by custom delimiter okay so field and then the delimiter semicolon and then sample.txt this will change the tab delimiter to the semicolon since we're cutting the first field the result should be the quick round okay so the field is this first part so that was the first field when we put the two oh wait hold on okay so the first field is all of this and then the second field is dog so on this command took the first field and then we shrunk it down to before this so the first field before the semicolon okay all right so what do the followings command do so this one is going to print so cut c5 printed out q so quick all right so this would probably this will probably print out the word quick you know okay and then cut c5 um i think this one is going to print everything from it's just going to print all of this okay yeah because there is nothing here so so there's nothing before so i think it's just going to do the oh yeah that cue yeah what command will you use to get the first character of every line in a file okay so we do cut c and then the first character yeah okay so the next one is paste the paste command is similar to the cat command it merges lines together in a file merge lines of files all right so we are going to create a new file [Music] so let's combine all these lines into one line so merge lines of files so that's paste let's do a help okay so we're doing tag s which is paste one file at a time instead of in parallel okay so paste s sample to that the default delimiter for paste is tab so now there will only be so now there is only one line with tabs separating each word let's change the delimiter type d to something a little more readable so we're going to do paste the delimiter flag and so what this is doing is adding a space in between instead of the tab and then s sample two okay so now instead of tabs we have spaces in between now everything should be one line delimited by spaces try to paste multiple files together what happens i'm not sure what it means by that but i'm going to do paste s sample 2 sample dot text banan.txt okay yeah so that looks ugly what flag do you use to paste do you use with paste to make everything go on one line is that s yeah all right so now we're doing the head command okay so now it says we have a long file or many long files to choose from go ahead and cat var you should see pages upon pages of text what if i just wanted to see the first couple of lines in this text file we'll see what we can do with the head command by default the head command will show you the first 10 lines of a file okay so this is the regular cat command so if we scroll up we have a bunch of lines so now let's do the head command and now we only see the top 10 lines okay so now we do head type n i guess for selecting the number of lines and we're gonna do let's do three yeah okay so now we get the three lines instead what does the following command do in y add tax c 15 well let's see when we do the help command it says c bytes print the first number of bytes okay so that might be just like characters i guess so it's saying 15 yeah okay what flag would you use to change the number of lines you want to view the head command the end flag all right so now it's tail um i guess this is kind of self-explanatory so the head is the top 10 lines and tail is going to be the last yeah so it's the last 10 lines 10 lines by default you can change the number of lines you want to see so let's do the last three all right so now prints the last three lines instead all right so now it says the f flag or follow all right so tail f your system log will be continuously changing while you interact with your system and using tail you can see everything that is being added to that file maybe just open up another terminal alright so what flag what is the flag used to follow a file that's the f flag expand and unexpand in our lesson cut command we had our sample text file contained a tab normally tabs would usually show a noticeable difference but some text files don't show that well enough having a tab may not be desirable to change your tabs to spaces do a cat sample text and we're going to use the expand sample.txt command above will print out each output with each tab convert it into a group of spaces to save this output in a file use the output redirection like below so typically expand sample that text and we're sending it to an output and we're sending its output into a different file called results.txt so let's do a cat results opposite to expand we can convert back each group of spaces to a tab with the an expand so let's do an expand help and we're going to do all okay converts all blank spaces instead of just initial blanks and expand all and then result.text what happens if you just type expand with no file input i don't know nothing i guess i mean it let us type things what command is used to convert tabs to spaces tabs spaces is just expand join and split the join command allows you to join multiple files together by a common field let's say i have two files that i wanted to join together okay so nano file one dot text right so now that we have ls we have the two new files now it wants us to join file so now it wants us to join the two files okay so it says they are joined together by the first field by default and the fields have to be identical if they are not you can sort them so in this case the fields are joined field one two three how would we join the following files so let's take nano cloud file3.txt all right and file2 is the same to join this file you need to specify which fields you are joining in this case we want field two on text file one okay and field one on text two so the command will look like join so it's saying join option file one file two join okay so on file one we want field two and then on file two we want field one and then three dot text and file2.txt okay so in the first file it was this field which is the second and then for the second file we wanted this field and then that's how we got this output okay so yeah the attack number refers to the files we want okay so now we can do split some file this will split into different files by default it will split them once they reach the 1000 line limit okay so let's do split 501 okay and so it said once they reach 1000 line limit file names are x and then some two random characters after by default so we have xaa so we split file one let's do a file one let's do a cat xaa okay split a file into two pieces join two files with different number of lines in each file what happens okay so let's do a nano file eight dot text okay so we're gonna have three lines and then for file nine i'm going to have four lines okay so now it says join two files with different numbers of lines in each file so to join we did join file eight file nine okay so it took the first took the second then third and then just disregarded the fourth line from file nine okay so it just ignores and doesn't print it out what command would you use to join files named cat dog cow um join cat cow alright sort okay so we sort lines all right so let's do let's type l um let's do n [Music] f all right so i deleted all the files okay so nano file one dot text sort file1.txt actually let me file1.txt and sort i don't know why there's a big space there but okay so it sorts it in alphabetical order okay so now i can do a reverse sort i don't know what that this big gap is um you can also sort be a numerical value well i didn't get that but bird cat cow elephant dog whatever the real power of sort comes with its ability to be combined with other commands so let's try the following command see what happens okay so let's etsy see we have all these things here so it's saying we're going to type this into the sort and we're going to do a reverse with numerical value okay so it goes from basically z to a okay flat guy used to reverse okay tr translate translate or delete characters allows you to translate a set of characters into another set of characters translate a to z okay so you do translate the lower case to uppercase and then you type in the lowercase and then you hit enter and it does the uppercase let's do okay so what this does it takes lowercase and makes it a type of case that's it as you can see we made ranges of a c into capital a z so try the following command what happens translate let's do translate oh okay so it says delete delete characters and set one do not translate so we're going to delete characters in here now we hit enter okay so it's just going to delete the character's hello from hello and all our left is with is h well commander used to translate characters tr the unique command there's a command useful tool for parsing text let's say you had a file with lots of duplicate okay so we're going to nano routing dot text and you wanted to remove the duplicates you can use the unique command okay um there's still duplicates here i wanted to remove the tools am i missing something um let me try without the spaces okay so there weren't supposed to be any uh indentations or so when you're doing this just make sure there's no um gaps in between the lines let's count how many of how many occurrences of a line so unique count reading that text okay so book shows up twice paper shows up twice article shows up twice and magazine shows up once and the random little face at the end shows up once okay so let's just get the unique value so unique um i guess that's the thing for getting a unique value reading.txt and it's just magazine and that little space at the bottom so hold on let me yeah okay now let's get duplicate values so unique duplicates reading.text okay so these are words with duplicates unique does not detect duplicate lines unless they are adjacent for example let's say you had a file with duplicates which are not adjacent okay so okay so it says duplicates which are not adjacent so book paper book paper article magazine and article [Music] and we do the unique reading and it just prints out the same thing okay so to overcome this limitation unique we can use the sort in combination with unique so we are going to sort greeting.text and pipe it into unique okay so we're sorting this into alphabetical order and then we are piping it into the unit to get the unique values what result would you get if you tried unique deck uc i don't know man let's let's find out qc okay so i get unique and since they're adjacent they don't count so everything is unique and then you get the what command would you use to remove duplicates in a file okay so we remove duplicates from the file with unique continue okay so wc and nl so word count and number lines do word count and it displays the number of lines number of words and number of bytes now we do word count tag l for just number of lines let's see another command to check the count of lines on a file is the nl so so we can do number of lines file one and we can see that has five lines how would you get the total count of lines by using the nl file without searching through the entire output use some other commands you learned in this course oh so i would do number of lines io1.txt and python into tail actually let's do yeah i mean that's what i would do right what was the tail command though n yeah okay so and two yeah okay so would be two what command would you use to get the total number of words in a file just to words word count type w all right and the last one is grep so grep is a very useful command so basically when you use grub it basically is like a search function so in this case we're going to grab the sample dot text the quick brown fox and so grab fox where's going to search for file existence or director you want to see a string and then a file you can certainly do it with turning one and dig through every line okay so yeah it is searching for strings and so we're gonna do sample.txt and if it's short enough it'll just highlight where it is so you can grip patterns that are case insensitive with the i flag some pattern to get even more flexible with corrupt you can combine it with other commands yeah so this is what i use it for corrupt insensitive user okay so the i just makes it so you can type anything and it'll be either uppercase a lowercase or both whatever so what it does is just prints out the line where the string is in so let's do ls let's and then we're going to grab and it says to search for some text file you know let's do ls in the current directory and we're going to prep the text so now it just grabs all the text files not sure why that dollar sign is there uh whatever you may have heard of egrep or f grep those are depreciated grep calls that have been just replaced with grub e and grab f okay so grab e and grab f are they in here yeah so grab e is extended grab pattern is extended regular expression f is fixed strings there's a set of newline separated strings what command will you use to find a certain pattern and that would be grub all right so that was text foo and i definitely learned quite a bit i knew some of the stuff but it was nice to learn new things and i'll catch you next time

Channel: greenhath4ck3r

Views: 321

Rating: 5 out of 5

Keywords: infosec, information security, cybersec, cyber security, cybersecurity, linux, unix, tech, technology, how to hack, virtual box, vmware, ubuntu, kali, windows, macos, greenhat, green hat, h4ck3r, parrot, red team, blue team, white hat, wireless hacking, learning linux, learning, education, journey, text-fu

Id: XQ-HdHtnGfY

Length: 47min 27sec (2847 seconds)

Published: Tue Mar 02 2021