Linux Essentials - awk

today's video is proudly sponsored by lenode the note has been doing cloud computing since 2003 which is actually before amazon web services was even a thing on the notes platform you can get your server up and running in minutes and they include all the popular distributions such as debian fedora ubuntu and get this even arch linux and let's be honest what could be better than a linux focus cloud server provider that lets you tell all of your friends i run arch the note has multiple server plans available to make any app scalable and flexible you could use it to host a blog a vpn server a minecraft server and much more in fact lenode is the platform of choice to host the entire web presence of learnlinux tv in addition the note offers 24x7 365 support regardless of plan size so you can get help from a live person when you need it new users can get started right now with one hundred dollars towards your new account and i highly recommend you check them out because lenode is awesome and now let's get started with today's video [Music] hello again everyone and welcome back to learn linux tv in today's video we're going to go over the awk command and i say command but it's more of a scripting language and it's a very useful scripting language that you can use to manipulate text and do all kinds of things and i'm going to give you the basics in this video but just the basics though because well this is the linux essentials series and i want to make sure that you guys have a grounded basis that you can use to build your future knowledge on so we're just going to go over the basics today but by the end of the video you will know how to use the awk command so let's go ahead and get started so let's talk a little bit more about how awk works you can use awk to create filters basically to write scripts to accept data from standard input change it in some way and then send that data back out via standard output in this changed form in general standard input is usually in the form of a text file and standard output is typically your screen but you can also chain commands directly into awk as well which you'll be seeing shortly so you don't have to have a text file to work with but working with a text file is how we're going to get started now offline i actually went ahead and prepared a text file that we're going to use for a number of examples i'm a big fan of teenage mutant ninja turtles so i figured i would use this opportunity to fit it into this video so i did and here's the output of that file now in this file i have four lines each one is representing one of the ninja turtles themselves so we have leonardo raphael michelangelo and donatello and we don't just have their names we have their bandana color on each line as well as their personality now if you're not familiar with the teenage mutant ninja turtles especially if you are watching this from an area where that's not popular it started off as a comic book it later became a television series and then movies but anyway you didn't come here to hear about the ninja turtles you came here specifically for awk so how does this fit in with awk by default the awk command sees spaces as delimiters for fields so if awk assumes that each space in between words is the delimiter for a field then what we can glean here is that we have three fields in each of these lines here so we have the name of the character right here we have their color and we have their personality or in this case their role so let's go ahead and see how this fits into awk so the way that awk works we type the awk command itself and then what we need to do is give it a command we need to tell it what we want it to do with that particular file now in this case what i'm going to have it do is print so we have the command in single quotes and the command itself is in brackets and then we give it a file name for the file that we want to work with so i'll press enter and there's no difference in the first command here i use the cat command which is one of several commands that you can use to print the contents of a file that's not all it's used for but that's what we're using it for here and then it spit out the contents of the file as we can see right here and then we decided to use awk we wanted to print and we wanted to print the contents of that file so essentially it does the same thing now in this case i would probably argue that the cat command is better because well i'm typing fewer characters to print out the contents of the file and now it's going to actually diverge from what we use the cat command for and we're about to see that awk is a lot more useful than we might think so let's try that one more time and show the contents of that tmnt file but this time let's show a specific field so by looking at this command right here you could probably guess what it's going to do and if you guessed that it's going to show the first field and the first field only then you guessed correctly and that's exactly what it did by comparison if i don't include the dollar sign one right here it's going to print each field of each line so what we can glean from this is that with the dollar sign one we were able to tell it that we wanted to print the first field for every line which is what dollar sign one represents the first field so for example go ahead and change this to two and also i could change it to three so now i hope you're starting to see the value that ock gives us it allows us to selectively print a specific field but i mentioned earlier though that awk is a scripting language and that there's so much you could do with it that is beyond the scope of a video to show you everything and that's true here we're printing something we're printing individual fields there's certainly more that we can use it for than just that i mentioned here that we are adding a command in the brackets technically that's a script we have a script in the brackets now you could get so advanced with awk that you can write entire scripts that'll span more than one line we're not going to do that in today's video but i just wanted to let you know that it gets orders of magnitude more advanced from here because awk is one of those tools that when you learn it you're going to keep learning new things and that's great because that keeps it fresh anyway another interesting thing that we can do is change the field number to zero and that's going to print everything because zero represents the entire file but we may as well have just not included that at all and the output is exactly the same but if nothing else now you know what zero means in addition you can also tell awk that you want to print more than one field so what i'm going to do right here is print field 1 and field 3. i separated the 2 via a comma dollar sign one for field number one and dollar sign three for field number three and that's exactly what it did the second field was the color it omitted that because we told awk that we only want the first field as well as the third and that's not all i mentioned earlier that even though you will commonly use awk with a file you don't have to use awk with the file you can also chain things into awk as well so for example i'll run ls-l as we already know that gives us a long listing of the files and directories that are in our current working directory and you could probably tell where i'm about to go with this we could pipe a command such as ls directly into the awk command and this output is completely useless it printed the permission string for each object inside this directory but we don't even know what the individual objects are we only have that one field but even if it's useless that was a valid command it did exactly what we told it to do so when we use a command that has spaces in between fields for example permission string the date and things like that awk is going to see these as individual fields so field one field two field three field four and so on because again by default it sees a space as well as a tab as a delimiter for a field so as you can probably guess i could change it to field number two or field number four five and so on and for the most part you could do whatever you want when it comes to chaining commands together that's one of the things that makes the command line so darn useful you could take a command that does a particular thing and then you can have the output of that command sent as the input to another command to produce your intended result so here i have a simple echo statement nothing too surprising there but let's have a bit of fun let's actually pipe that command directly into the awk command so i wasted all this time typing this entire sentence right here just to have awk truncate it down to the very first word the nerve of that command well actually that's exactly what i told it to do the individual words in this echo statement right here are going to be assumed to be separate fields by awk because again spaces are the delimiters by default so when i actually took the output of this command this echo command i piped it into the ah command which is actually wrapped a bit here but here it is it's going to take this sentence which is what echo does echo echo something it echoed this sentence echoed this sentence echoed this sentence and ock grabbed that and printed out the very first field which is going to be hello because it's the first thing to have a space next to it similarly just like before we can actually print more than one field so you get the idea now let's take a look at a few more examples of the awk command so in this case what i'm going to do is type awk i'm going to adjust the command a little bit because before i was printing individual fields field 1 field 2 field 3 and so on but instead of a field number i have nf and f stands for number of fields essentially whatever the last field happens to be is what's going to be printed so if you have 99 fields then the 99th field is what's going to be grabbed by this command and as you can see here that's exactly what it did the file tmnt had three fields so in this case nf is going to equal three it's going to equal whatever the final field number actually is so leader hot head party animal and so on and we see that's exactly what it did so what do you do in a situation where you have a file you want to print a specific field number from that file but the fields aren't actually separated by spaces and the perfect example of that is the etsy password file there's no spaces on any of the lines here so if i wanted to print the second field of etsy password it's going to have some unintended side effects we can see right here that it printed my last name and the reason why it did that is because we have a space right here so according to awk for this line in particular it thinks that there's two fields this one and this one so using awk with this particular file that didn't really work out too well but thankfully that's easy to fix so here near the beginning of the ah command we can use the dash capital f option and that allows us to set a different delimiter or field separator for awk to use to differentiate one field from another and what i'm going to do is set that equal to colon as you see here so this is the new option i'm telling it again that i want the field separator to be something else so now when i press enter we get a completely different result and this time we get the correct result the second field is legitimately an x so if i was to change that for example to a different field number we get a different result should get a different result every time we use a different field number in this case i am able to view this shell for each user right here but that's not very useful is it now let's make that a little bit more useful so if i wanted to find out which shell each user on my system is using and i didn't want to see any of the other fields then i could select field number one which should be the username and then field number seven which should be the shell let's see what happens and sure enough it worked check this out my user is using bin bash this user that i created in a different video is using bin sh so i was able to use a more practical example of awk by selectively looking for users and printing what their shells are and as you can see right here that was useful so there you go i've just taught you guys the basics of the awk command and by basics i mean the basics because there's so much more that you could do with the odd command than just this but it's really important that you memorize the basics because again that'll serve as the foundation for future learning so i hope you enjoyed this video and make sure you subscribe if you haven't already done so and i'll see you again very soon thanks for watching [Music] you

Channel: LearnLinuxTV

Views: 12,029

Keywords: Linux, LearnLinuxTV, LearnLinux.tv, awk command, awk, linux commands, learn linux, awk examples, command line, linux cli, linux command line tutorial, linux tutorial, linux tutorial for beginners, linux command line, gnu awk, linux awk, linux basic commands, awk command in linux, awk tutorial, command line tools, linux tutorial 2021, shell scripting, awk tutorial in linux, awk scripting, linux, gnu linux, how to use awk, command line basics, command line linux

Id: oPEnvuj9QrI

Length: 16min 7sec (967 seconds)

Published: Tue Aug 03 2021