Heroes in a Bash Shell: The Linux Command Line

um tim's going to be talking to us about the command line from the linux perspective now this is probably where i spend a good probably six to 12 hours of my day sometimes so really excited for him to present this but he brings us a great perspective from consulting as well as from working in the online gaming industry specifically not like league of legends think more like online casinos which you can imagine has some really interesting tech stacks and he's worked from a background of help desk and uh technical support from into security so i think that's important to note as well because we talked about that as a pretty ideal uh means of of entering into the security industry so great stuff to uh to show in here i have had a sneak pre-peek of your slides so i'm looking forward to it but without further delay uh tim i will turn it over to you sir for my favoritest title of the day heroes in a bad show thanks uh i think so welcome everyone i'm hoping you're enjoying this summit and all the great presentations uh we'll get through the initial starts fairly quickly um just so we can get onto the demo so uh let's go uh so i've been around it for about 25 years i did start in help desk it was i worked for an isp it's an internet service provider way back in the mid 90s so i was there when the internet was still very young and fresh and grew my skills as a unix system administrator dealing with solaris i moved and i think i started with my first linux distribution back in 95 and i have been a big fan of it ever since and seen it grow as phil said i've been working in a number of different organizations around the world um and i'm very familiar with fans and all the gx certs i have quite a few of them so i'd be happy to talk to anyone after about any of those if they have any questions but let's get going so uh human computer interfaces uh so way back in the 1940s uh they started with punch cards thankfully um that predates my era and the command line which is where we're going to focus today um is still used very heavily and it's one of my favorite interfaces to computers it is like a language in and of itself so it's really interesting that people who study languages like english and stuff like that do extremely well as system administrators when they start looking at command line because it's like its own grammar of course there's still text-based menus which are used today most of people who are getting into computer and stuff like that are used to graphical user interfaces whether it's windows or mac os um a web-based interface is still the same thing we do have voice recognition now with siri and alexa and so on and they've made made extreme progress with neural links now they have chimpanzees playing video games so until we get to the point where we can actually think in the computer can do what we say i still think the command line is by far the best interface to a computer so why would you want to learn the command line so the command line gives you a lot more control and freedom over what you're doing with your computer you can actually get your computer to do things uh that others can't when you're using a graphical application or something that someone else programmed you're very limited you're boxed into uh what they want you to do or what they think about the actual workflows you can create your own workflows your command lines which is really nice uh the other thing that's really nice is that it's it's repeatable and consistent so if i have a text file and we'll go through some text files in a little bit if i have a text file and i give a command to basically pull out information out of that text file i can give you that text law and have you give you the same command and you should be able to pull exactly the same information out this is great because once you get repeatability you can now start to program and automate things also you can look at parallelizing your work so if you know a number of steps have to happen on this text file you can split it into multiple pieces spread it across computers classical computing is just something i'm starting to get into now i have a nice little pi cluster to work with and the command line gives you a lot of flexibility with that in that you can have ship commands around and actually process them and gives you a lot more capabilities than just a single computer it also requires minimal resources a graphical user environment is very heavy on a system uh it actually takes up quite a bit of memory now that has been reduced over time but as new features the fancy transition effects and stuff like that that happen um that takes up memory and that takes up programming space on the computer programming or processing requirements so with the command line it's all text it doesn't require all these fancy transition effects and so on so you can actually get the computer to run much faster and when you start looking at stuff like container technology and some of the cloud native stuff um it's all done via command line and building out the the containers and stuff like that right it's not done in a graphical interface now there are some weaknesses to it um there is a learning curve right and the best thing to do for that is just get your hands on the keyboard and start working with it the other thing that uh is i would say a little bit of a weakness um is the computer is going to do exactly what you tell it to do and there's no undo right and you'll be planning for an undo and i've actually run across this in my past creating a massive uh file to scale out it sets them only to realize that it was searched literally and killed performance um and i did it during a peak uh production so learning from your own mistakes is pretty important um so today we're going to talk about bash and the supporting cast so bash is the shell that gives us the interface into uh linux and being able to send commands but it is not alone in and of itself it's got a supporting cast of a number of commands um that were made available through uh the new project and basically there's over 300 unique software projects in that and it's been around for a long long time gnu actually predates linux and linux started in 1991 so the barnegan shell is actually a riff off of the original shell which was called the bourne shell they added a lot of features and functionality to it even while it's open source it doesn't mean that it's always been reviewed there was a major vulnerability in the basha in 2014 that that was there since 89 which was shell shock so as you get around security and start talking the language and stuff like that you start talking about vulnerabilities and this was a very big one it affected a lot of major sites um and it it was interesting that it had been in place uh since 1989 but only discovered in 2014. there are some other ones that came out this year with sudo and some others that also had been around in the code base for quite a long time okay so this was a book that i read way back when i started my career um it's actually quite costly now it's close to 70 i think on amazon but it actually talks about the unix velocity um and the unix velocity got transitioned into the linux slot because the linux kernel and programs were modeled after unix now it's grown past that and has additional capabilities that your traditional unix systems like slayers hp ux aix don't have um but these are the really key points of this book right so small is beautiful so you want to have a program that does one thing and one thing well um by doing that uh you know it's going to do exactly what it needs to do and nothing more couple that with the ability to actually put it into a pipeline so everything is a filter uh which is the last one on here you can actually change chain these programs together and i'll show you that as we go through the command line to show you that um they also said to avoid captive interfaces so that's your web browsers that's your desktop environment uh because a captive interface is very difficult to automate the program um and everything is uh they wanted everything as a text file because text is extremely portable now there is some issues with text when you start talking about character character sets right unicode and other languages but text is actually a very universal format in that you can actually ship it around um you're not having inferior where there's delimiters in in the actual binary with coding and stuff like that um so let's get on to the demos and actually get some fun in here okay so let me just switch over here to okay so here is our command prompt um let me just increase that a little bit okay so when i first started um mentoring and teaching uh junior system administrators um i had a friend when i was in talking about he wanted to go from project manager into unix administration and he goes okay so how do i learn to be a unix administrator i said i want you to go through and look at every manual page on the system um and it took him two two to three weeks and he hated me for it he hated me for it until i got a call about three years later saying that it was the best thing i ever ever did because like learning how to read um and learning how to read a book eventually you learn what a table of contents is you learn what an index is so you actually learn how to find things in in the actual um in the printed material on a computer system you have manuals now back when i was uh helping him with that it was the late 90s and google was around but it wasn't anywhere near what it is now and you didn't have stack overflow or stack exchange and stuff like that so the man manual pages man for short um to get into them tells you basically basically to give you a help page on each of the commands that are there so going through the man pages there is a lot of commands so if we actually look at the actual bin directory right well it scrolls off the screen but let's count the number of commands that are in here right so there's approximately 1358 commands and that's just section one of the of the man pages that's the commands that are there there's other sections that deal with programming the interfaces of linux and and so on and i haven't read them all today i would actually say you probably want to start with a much smaller subset uh because this has grown over time uh it's actually reading all the man pages was something i did early in my career i used to do it once a year for the first couple years on the linux systems and unix systems i was dealing with to make sure i understood what was there but when you actually read a man page right so if i go and do a man of seymour uh it comes and broken down into a very specific section so you got your synopsis your description you can zip through here you can actually look at what your file that it's dealing with and a lot of times it'll have other sys other commands to see or examples now if i don't know what a command is and i say i want to find something about secure shell right there's another command called upper probe which basically will allow you to go and find all of your manual pages with regards to that search term so as you can see there's a lot of section one so the one in brackets is the section number and then you have section five which is a configuration file and then you have other things that are in there um so next up is um if i want to find out what something is as well as i can also ask what is ssh right and it'll just give me that very specific if there's a command for it right so if i do what is uh right it'll tell you the exact just for that one there's another command set or help pages that it's called info but it typically requires a package to be installed and it's not available on all linuxes the other thing that you're going to have is a with a lot of commands so let's do ssh you can actually add in dash help and it'll actually give you a mini help of what it expects different arguments that a command will take so if i want to do awk is a programming language that's on there and there's no help option right uh there's sort another one we'll look at right it gives you a lot more information so each of these programs are written by different authors and stuff like that and a lot of these commands that i've typed in so far are all part of that gnu toolset right if you go to gnu.org you can actually see it they have terabytes of going back to different versions of those files okay so we've gone through help um help is a very important thing to learn if you're learning powershell bash whatever going through the reading the help reading the manual is definitely very very helpful and important a lot of things that once people want to do on a linux system is to find a file right or to find something that they're looking for so there's a couple of different commands you can do with that so we can locate say i want to locate spell python3 right that returns a lot of information um and it's not finding it's finding python three but it's gone python3 in all the different things uh one of the more useful commands that we have is is called grip and that's global regular expression print right so and it's been shortened down to grep and it's used by a lot of people and in this case what i can do is i can actually look for three and then dollar sign means that it's the end it basically is looking for the last part of the actual um string so now i can just find the pipe things that are with python 3 at the end right if i want to be a little more specific i'd actually do python 3 and now i only find where python 3 is ending whether it's a directory or a file other commands that we can use is basically the find command now the find command takes can take quite some time to master because it's extremely powerful but i can actually do a fine and look through all the files in this directory and it'll actually go through subdirectories as well if i want to find a binary that's in my search path right so if i want to see if a program exists on my system i can actually do a witch and look for an actual program now this will actually go through and if i look at my path right my path is basically all the different directories it'll search for programs um and in this case the first path that found was in my home directory but that's not the only place where top lives i can actually do a where is and talk and it'll actually show me the man page for it and it'll show me if it's actually in different paths so this is extremely important from a security perspective if someone actually put something earlier in your path it'll get run when you actually run the command unless you actually use and specify the whole path for it so where is this something that i use on a lot of systems to make sure that it actually is running the plane i expect um another thing we can look at is ownerships because once you actually can find calls and stuff like that you may or may not have access access right file permissions is something that we deal with whether it's windows mac or linux in linux basically you have the structuring access controls which basically is broken down into two basically three things right there's the user who owns something there's a group which is associated with that object as well and then there is also um basically everyone else right it's very very um coarse as far as uh permissions goes and that's the default right but it's not the only area where you can get into because there are a couple other commands that we can actually use um to get additional detail so if i want to look at um you can actually put extended acls on on systems and on directories and files it's a command that's a lot of people don't realize so on a system if you're running docker whatever right so by trying to run doctors doctors running on the system i i get access to that because docker does all of its authentication sorry permissions uh through the docker socket which is under and if i do a get file icl there i can see that it basically has user and group and other now you can also set file permissions um to actually give me access to that now a lot of people will actually add the individual into uh into the group and that basically will give you right access as well but you can actually modify the acl on here so i can actually do modify user my user and then basically the permissions docker dot font all right no of course i have to do that as privilege because otherwise it is bad so i elevate my ownership into and that's not going to work because i copied that over so let's go in okay so now if i do i can actually see that i have read write in the docker socket so now if i do a docker yes i actually don't get an error message i can say document images and i can list it out um this is important because as an attacker if you're not looking for acls um and extended ecls if i've gotten on the box and i know that this is not something you're looking for i can actually add permissions into different commands through the file acl and it'll go unnoticed uh by a lot of people now adding in uh an account into the sd password for for another uh super user account or adding myself to a specific group might get noticed this oftentimes will go unnoticed we can actually then clear that out by just doing set file acl and we can actually remove all basically remove everything that's extended acls on the actual socket itself now again i have to do and because okay so let's go back to okay so that gives us through um some of those commands uh the slides that i have will be available um afterwards um next up we actually want to look at some files right so i i downloaded a file from um mock data so i will put it up there uh should come up yeah resource session so macaroon allows you to mock up data so um i do actually administer a number of websites for car enthusiasts uh for getting together and stuff like that so these are not from those individuals um but if i actually look at these so if i want to look at the file itself i can do cat which means concatenate and it'll print it all out now there's a 1983 um entries in here so i just want to look at the top 10. okay i can do the head and that gives me the top 10 lines like the same as i can do tail right of charge and that gives me the bottom 10 lines now this is all text and this is comma separated which is something that a lot of people are used to we get a lot of power with text files because we can actually use commands like cut okay and i know what the glimmer is a comma right and i can actually just print out fields let's say i want to do first name is the second field um followed by the third field and fields five through six five through seven and let's just look at the first 10 lines of that right so in this case i've actually removed um filtered out stuff right so and this gets us into seeing what the actual pipelines look like right so using things as filters right so we've gone from one command push information to another command which then filters and just gives us back specific information um we can actually change some things me or say we want to change instead of so common delimiter is um something that we've added in there but let's say we want to use cuts normal domain text number delimiter is a tab file so we can use tr to actually translate one character into another and like that i shouldn't need that and now [Music] so now tab separated i can now do cut i don't need to actually specify a delimiter anymore so now i can do two right so first and last names um another thing is we say we want to actually go through let's have a look at our data again so now we can see that there's a large number of countries that we've added here right so that is one two three four five six so it's a six field so we can actually do comma field six right so that gives us the number of countries now what we can do is if we want to count the number of people coming from each individual country we can actually sort um then we type it to u minus c and then we can actually sort by n so all right so this is a very common use um a series of of filters so the first one what it does is basically sorts everything um alpha alphanumerically right so uh lowercase a before lowercase b and so on um then we actually when everything is sorted the command unique will actually suppress repeating lines so if you have a whole bunch of candidates in a row and a whole bunch of ebs in a row it'll actually suppress it with the minus c it actually instead of just suppressing it it keeps count so we can actually count how many so in this file there's uh 1100 from the philippines 246 from canada um you can do this with ip addresses you can do this with other aspects so if i instead of instead of looking at that data let's say i go into logs and i go into my apache right and i look at my access log right well this is not as as pretty as what we've had in the past but we can actually pull information out of here as well right now it looks like it's space delimited but you're going to have additional spaces in here right one of the things that we can do is we can grab out our get and post so prep by itself will actually look for standard strings you can actually extend it by using a regular expression for your grep so if you do a minus e you can add a standard regular expression and i can look for posts or get so this will highlight basically all of our get some posts um now the gnu version of grep which is not available on windows and is not available on mac os unless you install it separate separately allows you to actually use 12 regular expressions which is actually much better when you're actually looking for numbers right so say i wanted to find three numbers together right in the access log so now it'll highlight the three numbers together right so i can actually look for specific things um the next thing you can do is well we have the ips in the beginning of the actual thing so well we actually see how many ips we have so we can do um akka is another fan that you would use on the command line right hawk is used when things are delimited by white space that is not consistent right so if i do right the white space is not consistent there's one space here one space here but then there's multiple spaces here because you have different sizes uh in this case awk is preferred over cut because if i do cut um and i do a denominator of space right i do field four right because of the additional space basically kai believes uh another space um so what we can do is we can actually change that and actually so that gives us the full numbers okay so um we've done sort unique uh we can actually count the number of lines in a file and access log which gives us quite a bit if we actually wanted to keep that information right say we're actually filtering stuff in this blog so let's have a look at the log again and say we wanted to grab out just the actual post from the logs so we want to see all the posts that come out so let's have make sure we're actually doing it correctly and then so this is a common pattern with building up long command lines is you build smaller bits and you build on top of it so in this case i want to make sure i had all the posts correct um now as you can see in this one it's not highlighting like it was in the past um if i want to do that you can always add color equals always and i should get that now let's say i wanted to actually look at all lines in context right i want to get all the posts but i want to leave all the guests in there as well is what i can do and this is the trick that i learned is you can actually look and just look for nothing but the end of the line so it'll match on every line if i have to use egret right so now i get the gets in between the posts this is very handy if you're trying to look in the context of where you're at in the fall you want to look at all the file but highlight specific things let's go through and so if i want to get all the information that's in that post and just the post information right there's another thing that you can do which is just minus o and what minus o does is it'll pull whatever you're matching and just pull that out right so it's just post doesn't make much sense but when i start combining this with just a regular expression and say i want to grab idea addresses so 1-3 b-2 yep hey um i just wanted to give you a heads up um i know you got a couple of more things to demo um and there's actually some questions in your hallway as well i'm wondering if we may be able to kind of shift over to some of those more specific questions in the hallway so we can we can bring jen in and make sure that we're getting hurt uh the the full time as well okay sure um yeah i can't bring up the hallway at the moment on the screens uh no no worries i just mean after we kind of shift over to jen i want to make sure people know that your hallway will still be open and then you'll be able to uh to answer the specific questions textually over there oh absolutely absolutely as we're going through yeah because there's a lot of commands that i haven't covered on the slides so if i go back to the slides if i get on here right so there's a bunch of commands that i'm like these are the important commands that i think you should learn um this one only uh so check on basically for security enhanced linux which really only works on red hat type distributions that actually use it um last tail so these are the primary commands if you're working with text files that you'd want to use um so getting familiar with these right and and just so everybody knows too don't worry about like scrolling down these notes you know immediately while this is screaming by because uh you'll get all those slides available to you in your summit access page and then you'll be able to check those out for reference as well um to make sure you can follow up on those and you know the one thing that i would like to say too is you know people are saying how do i test this stuff well setting up a virtual machine and something free like virtualbox great way to experiment with this regardless of what your operating system is um but i'd spend so much time in the command line i even learned a couple of things out of this which was pretty cool and and i certainly appreciate that so um well thanks very much for sharing tim really really good information and i know that a lot of people kind of had some eye-opening aha head-exploding moments which is always interesting to see as well uh what you can actually get done without even stepping into the graphical environment really really cool thank you very much

Channel: SANS Institute

Views: 807

Keywords: Timothy Brush, Tim Brush, Tim Brush SANS, Timothy Brush SANS, linux, linux command line, bashs hell, linux command line tutorial, linux cl tutorial, command line tutorial, command line, linux tutorial, linux command line for beginners, linux command line how to

Id: kRg_e88ikrk

Length: 33min 7sec (1987 seconds)

Published: Wed Oct 13 2021