Linux Command Line - Pipes and Redirection

with with Linux most of the commands we run sometimes they take an input like an option or a file and produce some output let's look at the concepts of the input and output of programs and I've got a presentation it's on the it's in the IT s33 - directory on your computer it's called what is it called Linux standard interaction standard input output and error these are concepts which become quite important when we want to build complex commands in on the terminal we think we run a command the input to that command is referred to as standard input stdin is the short name the standard input so often we run a command and it takes some input we haven't seen many today but we can use an input to determine what that command will do the output is what's printed on the screen you run the command of LS and it shows a set of lines on the screen that's the output that's referred to as standard output if you run a command and there are errors then that's distinguished from the normal output and it's called standard error I will show you an example LS the standard output is this light these lines of text has sustained that output a BCF I tried to run a command that didn't exist my terminal shows me a VCF command not found this is an error message and it's although it's shown on the screen it's referred to as standard error so when you run commands the normal output is standard output error messages are distinguished and sent to what's called standard error so we have the input to commands output normal output and error output simple and the input is typed on the terminal and the standard output and by default standard error are displayed in the terminal normally when we type command the input we type on the terminal it's what we type in the response to the command and the outputs displayed on the terminal now we can change that we can redirect the output to a file so normally the output the standard output displayed on the terminal what we can do is send the output to a file and save it in a file this is called redirection and to redirect to a file we use the greater than sign let's try it LS normally the output of LS is shown on the screen redirect using the greater than sign and then a filename you can choose whatever file name you like press ENTER and see what's shown on the screen nothing so the the output of the command is no longer shown on the terminal where is it it's in the file out dot txt have a look so the file now contains what normally would have been printed on the screen so we redirect the output of the command from the terminal instead from the terminal we send it to a file very useful if you you want to look at that output later so that's redirecting the standard output to a file use the greater than sign you can redirect both the output and error messages to a file using this special two characters of ampersand greater than so note the difference here the greater than sign shows only the normal output and saves that in a file any error messages are still is displayed on the screen I'm not sure if I can find an example let's try what about our find I'll scroll through before we searched in our file system for PDF files let's instead of display the output on the screen display into my file called I don't care what the name is so run this fine command and everyth everything find say that in this file X dot txt what happens it runs but the error messages these permission denied messages are shown on the screen so the normal output is shown in is saved to a file but the error messages are still shown on the terminal to the tilde character that I type the squiggly line is means my directory my home directory so the command is secured was saying I think it wasn't necessary there but what I executed was find save the output the standard output in my home directory a file called X dot txt if you look in X dot txt using less you should see the files which were the PDFs there the whole list of PDFs so no there's a difference between standard output the normal results and arrow results standard error if you want to redirect their errors to the file as well at the ampersand character I hope it works yes okay so now if you look in X dot txt you'll see the files found plus those error messages the error messages are caused because I don't have permissions to read some directories so redirection we can redirect different things there are many different ways to do it redirect the standard output to a file redirect both that standard output and the error messages to a file we can redirect our files to standard input I will not show an example now will see one maybe this afternoon that is if a command takes input instead of typing that input you can take the input from a file so it's as if you've already typed it into a file run that command using the what you typed into the file as input to that command so read from the file the input it will see one this afternoon of that next thing so this is redirection change where the input and output comes from one goes to piping normally when we run a command the output is text well what we can do is run a command it produces output and take that output and send it into another command and that second command executes using the output of the first command as input it's called a pipe and the character we use is the vertical bar take the comma first command executors the output is then used as input to the second command and the output of the second command is then in this case shown on the screen let's see an example or a few examples of a pipe when we do LS I see the list of files LS - EDL shows you the details I can pipe the output of the LS in this case the LS - L with all the details and filter it and send it to a new command and I'll use grep to search for a particular word so my aim lists all the files and of that list print just the lines which contain the word tiers and you see the two lines from the LS which contain the word tiers now we could have done the same thing just using LS in this case just an example of take the output of the first command and then pipe it into the input of the second command and then you can have actually multiple commands and using pipes and redirection is a very powerful to build up complex commands based upon very simple ones as an example from our introduction to the course for the the poster for the course this is a command don't try and run it sorry I've got the slide show on this is one long command actually the slash at the end just means it goes wraps around the line you'll see this actually multiple commands you'll see these vertical bars in here meaning run some command here run this tail command you don't have to understand all of it but run this tail command here and take the output of that and pipe it into this next command TR and there was a redirection there a redirection from a redirected into F so more so this afternoon we'll start to combine some combo commands using pipes let's give some more real examples remember your aim today is to understand this command understand why did it print those four or five lines on the screen now if you run the command today it may not print the exact same lines but it'll be close let's go back to our terminal for the last few examples what else can we do let's say there are other ways to do it but my LS minus L showed me all files in detailed long format grep tears shows me those lines which contain the word tears and we see all these strings there's one line note that each line is space separated these characters are spaced one space students pay student and so on and it's the same format let's say I just want to get the file size which field is it if it's space separator the first field the second field we'll see how we can do that think of a space separate fields the first field the second the third fourth the file size is the fifth field in that output let's pipe it into a program called cut which can cut an input when the delimiter between fields is a space - D means the delimiter between fields and we want the first field sorry its wrapped around - f space v cut cuts text according to the spec you give it and the spec I'm giving it is to say the fields are delimited by space I'll run it and then show it again is people cannot see and the fifth field it got one of them let's try again and it prints this big 38 380 megabytes what about the second line it doesn't print anything why why did it not work let's get say the first field instead of field five field one for each line at princefield one if the fields are separated by a space the first to the third field one two three the first three fields the first to the fifth field something didn't go as I hoped what went wrong what went wrong is that they're in fact two spaces here okay so cut separates fields by spaces there's a single space here meaning this is the fourth field this is the fifth field whereas student on this line is the fourth field there's a space the fifth field is non-existent there is no value the sixth field is 83 so on so card is only separated by a single space in this case so if our input has multiple spaces doesn't work very well we can cut I'd like to see it at the top we can even cut specific characters and pipe that into cut cut the first field from our first output and then pipe that into another cut that kacct cuts characters 2 to 4 so the first field was these 10 characters the next cut grabs just characters 2 3 & 4 which is RW - RW - so cut it's just a way to split up strings based upon field delimiters based upon characters and a few other things we'll make use of it in some further examples later what else redirection so this is pipes of course we could redirect all that to a file almost done some other shortcuts I think everyone should have two terminals open at least two terminals one you see what I type and one is what you type of course you can copy and paste if if you're lazy you don't want to type something copy and paste from one terminal to another in most Linux systems you can do a quick copy and paste by select use your mouse select don't right-click select so you're not be able to see it with me but I can select something if you want to select a word you can double click and now just move and don't right-click middle click pastes pace what was selected so just select as you would select anything with your left mouse button once you've selected what you want to copy it's already copied and then to paste it middle click on your mouse the middle scroll button click that that pastes what if you don't have a middle-click we were supposed to hold this lab in the Mack room and we're going to use VirtualBox but they don't have a middle click so you get a different Mouse but most Mouse is you can emulate a middle click with two buttons so my keyboard my laptop for example has two buttons only there's no middle button pressing both at the same time emulates a middle clip and is usually the case on normal Mouse's not on Mac Mouse's so middle click if there's no middle button double click left and right at the same time so let's just finish with a few more shortcuts now remember some you have a command you want to change the something at the start press ctrl-a moves your cursor to the start of the line control e to the end control a to the start of the line control e to the end if you need to edit a long line they are quite useful if you want to delete something you can use ctrl K ctrl K to cut everything beyond in front of the cursor control Y pastes I'll do it again and look where my cursor is it's at the start of the cut ctrl K deletes it it actually cuts it control Y Yanks or pastes them so we can do some basic things for copy and paste code cut and paste either using the keyboard or you can just use your mouse the mouse select and middle click is very powerful very very quick to copy and paste any commands that we've missed that we may need this afternoon any questions as we head into the lunch break the last five or ten minutes I think it may be a twelve o'clock we'll go have lunch so the next 10 minutes just try some of the things will come around answer any questions this afternoon once you know the basics from this morning we'll just do some other things like convert some videos manipulate some PDFs and create some scripts where we put a set of commands into a file and execute them all the ones which can be much more powerful than just running them individually so I'll stop talking now so the last ten minutes just try some different things so we'll come around spending questions and then we'll have a break

Channel: Steven Gordon

Views: 7,490

Rating: 4.9266057 out of 5

Keywords: Steven Gordon, Thammasat University (College/University), Sirindhorn International Institute Of Technology (College/University), GNU/Linux (Operating System), redirection, Pipes, Command-line Interface (Computing Platform)

Id: Cqanrev2jXw

Length: 22min 39sec (1359 seconds)

Published: Mon Jul 28 2014