Sockets and Pipes Look Like Files (Unix/fdopen)

today we're looking at how a lot of different things in a Unix style OS like Linux and especially sockets look a lot like files [Music] welcome back everybody especially all of you who helped me keep the camera running by supporting this channel on patreon today we're talking about sockets which I made a bunch of socket videos in the past I'll link to those in the description in case you are new to sockets because today I'm just going to jump right in the middle assuming that you've seen them at least a little bit so if you haven't checked out those videos it might make today's video go a little bit easier in this video I'm also assuming you have seen file i o so you know how to like open a file read from it and write to it and things like that again I've got videos on that as well check those out if this is new to you now if you've been hanging around Linux Mac OS or any Unix style operating system for a while you've probably heard people just casually say oh yeah and Unix everything's a file but more accurately everything looks like a file and behaves like a file and it's true and it's intentional Unix style file systems do try to make things as much as possible all look like files and the reason is is that this allows us to handle very different things in a very similar sort of way we can write one piece of code that can handle things from pipes and sockets and files and that's really cool so today we're going to look at how that works I want to show you how we can process data from a file from a pipe and from a socket specifically as always the source codes available through patreon so get it there if you don't want to type in as we go but now let's take a look at some code okay today we're starting off with a little bit of code more than usual actually usually I just start with kind of an empty Main and we go from there but today I didn't want to re go over all the socket stuff so I have a function here called HTTP get which is a really simple socket client in fact it's almost exactly the same thing that I did in my socket tutorial on writing a web client so check out that video If you haven't seen it before but all it really does is it's going to open a connection so it takes in an address which is just an IP address and a port and this function is then going to come in and open a socket connect to that server and it sends an HTTP request it looks like this so it's going to send that and then what it's going to do is it's going to return this socket file descriptor or the socket for the connection and that's going to allow us to actually read the response back okay so this is going to come in handy in a minute but that's here there's also this error and die function which I've used in the past all that does is allows us to print out an error message it has some nice variatic stuff so it acts like printf and mostly that's here because I use it throughout this HTTP get if ever there's an error I'm just going to exit you of course in your applications will handle errors in different ways depending on the needs of your program I also have a little make file here which is going to compile my program and this program is going to be called count lines because what I want to do is I want to read data from a bunch of different sources and just count the number of lines in the output okay so let's let's come back here and we'll just close this up see the way that I want I've got this print usage function the way that I want this to be run is you can run this in three different ways okay so if we come down here let's just it's just I mean it doesn't work yet but let's just compile it and so let's say that I run count lines this is just telling us what our intended usage is so I want to be able to say run count lines with standard in and that says I want to actually just read whatever's coming in from standard in so whatever's coming in from that pipe the second is I want to be able to specify a file so if I say accountlines file and then provide a file name then that's going to read from that file and that's the output we'll look at and then let's say that I want to actually make an HTTP request to a web server and download that then I want to grab the output from there so the point is is I want to be able to switch between these as always as a programmer I don't want to repeat myself I don't want to produce a whole lot of duplicate code that handles these sources separately so I want to just talk about how we can actually use all of these and process their output in the same exact way okay so what I'm going to do so let's come down here right now I'm just doing a quick argument check so this is just saying let's check to see if we have two or or three arguments which is what we need to make this all work and let's just come down here and we're going to add a quick variable I'm going to call it SRC it's character pointer and I'm going to set this equal to ARG V1 I could just use rv1 directly but this is probably going to make my code a little bit more readable and then I'm also going to make a file pointer I'm going to call this input source and let's start it out as null okay so this eventually is going to be the thing I'm going to read from now what I'd like to do is to have this one input source and then down here what I'd like to have some code that's like read all the lines from the source and so I really would like to do something like let's say let's make a buffer we'll call it you know we'll make it max line which is I Define that up at the top that's 4096 and let's make an integer which is called num lines we'll start that at zero and then I want to do something simple basically I just want to do what we would do if we were reading line by line from a file so let's just do something like well not feof input source so I'm assuming of course right now this isn't going to work because input source is just null but assuming that input source actually is connected to a file then I could just use something like you know check to see if we're at the end of the file for as long as we're not at the end of the file then we could come down here and say if F gets and let's read buffer we'll get max line that's the size of the buffer and let's read from input source and so as long as that gives me something that's not null then I'm going to come down here and say num lines plus plus okay and then down here at the end let's just close whatever our source is and I'm going to just print out we read percent D lines new line character and we'll just print out num lines okay so this is the code that I want to actually process my input whether it comes from standard in whether it comes from a file or whether it comes from a socket and now the only thing we've got to do is in here we need to actually set this up so that input source actually is connected to these things okay so how are we going to do this well let's start with the easiest case let's just say if I'm going to use string compare to compare our source to say standard in okay so if my source that I specified was standard in and if that's zero then we're going to come down here and let's just say input source equals standard in okay now standard in is actually a pipe but and I'm not sure why it's giving me a red line I'm sure it's going to work just fine but the point is it's a pipe coming in and that pipe is pointed to by a file pointer so it actually it's already already looks like a file so we can just assign it like this one pointer to another so that's our easiest case and then I can say else if and let's just do this same thing again but now we're checking for if it's a file Okay so if it's a file then we're going to come down here and let's say input source equals we'll just use f open and the file name we'll use is Arc V2 and we'll open it in read mode so that was almost as easy as the standard in case so really nothing nothing crazy there okay now let's get our socket case so same thing here let's say that our source here is HTTP now in this case we have our function up above we have this HTTP get function so we'll just use that so I'm going to say and socket equals HTTP get and then let's pass in our V2 that'll be our IP address and let's put server Port which I defined up above as 80 which is the standard port for HTTP obviously if this was on a different port we would use a different port number here but for now this will work just fine okay now the issue here is this is the the first place where things get a little bit interest interesting because standard in is already like the type of standard in it's already one of these file pointers and F open returns one of these file pointers but a socket is an INT and so at first glance you might go well how is that the same as a file that actually looks a lot different from a file it's a number rather than one of these pointers and the key Insight here is that this end is actually what we refer to as a file descriptor and inside of every one of these file pointers there is a file descriptor the file descriptor is actually what the operating system is using to keep track of these open files this file pointer here is just a convenience struct that helps with buffering and things like that so for this we actually have a helpful function that we're going to use that is a lot like f open but it's called FD open okay so in this case what I'm going to do is say input source equals FD open now FD open is just like f open but instead of giving it a file name we're going to give it an open file descriptor so in this case socket and then I just want to open that in read mode and that's because I want to read the data from the socket and so there was just a single extra line of code now we are actually able to take a socket and convert it into one of these file pointers so now we have a file pointer and now this can be used basically all of these can be used in the code Down Below in exactly the same way now let's just just to be careful let's add just a little bit let's add an else case here and so if something else happens let's just print out uh error invalid source and percent s and a new line and that'll help us if we accidentally type something in wrong and we can return exit failure so we don't continue with a source that doesn't make sense now one other little bit of error checking here is there's always the possibility that one of these things failed you know that I passed in a file name that didn't exist and so input source might be null so let's come down and do one more check and say if I keep putting a t in there if input Source equals null then we're going to print out error but not open source and then once again we will return exit failure and so now hopefully we have a working program so so what we're doing here we basically have a little bit of code here that takes each of these sources makes them all look the same basically gets this input source file pointer here and depending on what our input was it's either going to be a pipe coming from standard in an open file that we specify as an argument from the command line or it's going to be an open socket and then we can come down here and we're going to read line by line and then get the number of lines and print it out okay so let's just see if this works let's come down we can compile it it's actually kind of amazing that it compiles the first time went through that pretty fast now if we come down here let's just make sure we can do okay so we can do this now let's say standard in okay now it's going to wait for input so I can say something like hello my name is Jacob and I'm testing my program and then control D and you you can see we have one two three four five six lines in our input okay so that seemed to work just fine we're able to take standard in no problem okay so now let's try it with a file and we'll say let's do our make file okay just it's convenient okay so that says we read 10 lines let's double check if we come in here sure enough it is 10 lines long okay so far so good now let's run this with a network server so let's first let me get an IP address let's just ping google.com Okay so that gives me an IP address I can use so now let's come back here and do HTTP and specify that address and now you can see we read 91 lines in the output and so you can see our program works great so this is just a really quick example to show you both that a lot of things in Unix style operating systems do look like files and also how this can be helpful to you in making really not a lot of code like this is a pretty short program but it actually is quite flexible it can do a lot a lot of different things it can handle inputs from a lot of different sources without having to write socket specific code or pipe specific code or file specific code because we can make everything look the same so I hope that's helpful I hope you learned something new I hope this helps you in a future project if you found it helpful please like or subscribe click something on your way out and until next week I will see you later

Channel: Jacob Sorber

Views: 9,976

Keywords:

Id: il4N6KjVQ-s

Length: 12min 44sec (764 seconds)

Published: Tue Feb 28 2023