Unix system calls (2/2)

the four most basic system calls for dealing with files are open closed read and write read and write are quite self-evident read will copy bytes from the file to memory in your process whereas write does the opposite it copies bytes in your process to the file when reading and writing a file however that file has to be open so first to work with the file you invoke the open system call and you pass in the file path to that file and open will return what's called a file descriptor a file descriptor is basically just a number which in your process uniquely identifies an open file so in fact when you invoke the read and write system calls you don't pass in a file path you pass in the file descriptor when you're done with a file descriptor you should then pass it to the closed system call releasing file descriptors this way is not strictly critical but it is good practice because in particularly longer running programs if you just keep hoarding file descriptors and never giving them back you're going to run out eventually the each file descriptor does take up a bit of operating system resources or some memory associated with each one and so it's just good practice to always close files when you're done with them now note here that I've indicated the read and write system calls may block now exactly under which circumstances these system calls will block depends upon the type of the file being read or written and also some options that can be specified when that file was opened we won't get into too many of those details and the most common scenario though when a file is opened with the default options the read system Col will usually block whereas the right system Col will not so looking at the right system Col here's what happens in the default case if we are right into a file on a storage device like say a hard drive the data is not actually directly copied from the process to the hard drive instead the call to write will copy that data from the process to a buffer in memory outside the process and controlled by the operating system and then from there the operating system will write the data from the buffer to the actual stores to even though the scheme involves copying the data twice it still makes sense for performance reasons as we've discussed a bit before i/o devices like hard drives are relatively very very slow compared to the CPU so we wouldn't want our process to have to sit around and wait while data is being copied to the actual storage device so if the process only has to write to a buffer in the operating system and then it's left up to the operating system to actually copy the data from that buffer to the storage device then the write that system call can return after the data has been copied to the buffer it doesn't have to sit around and wait and the process can then continue and get some work done while that data is being copied out to the actual device so here's what the use of write might look like in code say we want to write to the file of slash Alice slash Tim well first we need to open it and that call to open returns a file descriptor here assigned to F then when we call write we specify the file descriptor so it knows which file we're actually writing to and then we also pass in the data to write in this case let's just say it's an ASCII string where each character represents one byte that's gonna be written to the file so here we call write twice and the first one writes blah blah to the file and then the second one writes blah blah blah well talk in a moment how exactly you specify where in a file you wish to read or write in this case what's happening is the first write just writes at the start of the file from bite zero and then the next write gets tacked on after that so we write five balls to the file and we're done with a file so we close it because that's good practice one thing to be clear about with the usual behavior of write is that your process can't really know when the data is actually written to disk or in fact if it ever actually is written to disk you never actually get verification that it actually happens in fact because it's left up to the operating system to do the actual writing to disk the process which wrote the data doesn't even have to stick around it's quite possible that the process can terminate before the data is ever actually written to disk this is problematic for programs which need with a high degree of certainty to know that their data has been written properly to disk a database program for example makes data integrity a very high priority so a database just can't accept letting the operating system handle the actual writing out of its data without getting any verification that the data was actually properly written even if databases can't get a hundred percent guarantee that nothing will ever go wrong with the data they at the very least need to know when something has gone wrong so although we won't get into the details of how exactly this is done there are ways in UNIX of writing to a file such that you do get verification that the data was actually written just understand that by default you don't get such an assurance now as for the read system Cole like with write there's an intermediary buffer in the operating system when a process invokes read the data is not copied directly from the storage device to the process first it is copied into a buffer in the operating system and then from there copied to the process again this is done mainly for performance reasons io devices like hard drives are typically very very slow relative to the CPU so it would take a long time if the process had to sit around and wait for the data to actually be read off the hard drive what makes us different from calls to write however is that when you're trying to read data presumably you really can't do anything until you actually get the data you can't sit around and wait for the operating system to then get the data later we need the data now so the purpose of the read buffer is really quite different the most common usage pattern when reading from a file is that we wish not to read just a small portion of it but also the portions in the file which follow so given the typical performance characteristics of storage devices like hard drives it makes sense when reading from a file to actually read more than is requested and then to store it in a read buffer so that subsequent calls to read can read directly from the buffer without having to wait again for data to be read from the hard drive so typically the first time we read from a file the read buffer is empty so our process actually has to block while data is read from the hard drive to the buffer and then once the data is ready in the buffer our process is unblocked at which point the read system call finishes its business by copying the data from the buffer to the actual process if our process then invokes read again on the same file it's quite like that the data is already in the buffer in which case the read system call doesn't need the block so to put it succinctly the read system call works by default by first checking the buffer and seeing if the data it wants is already there if not the process will block while that data plus some amount of extra data most likely is read into the buffer and then the process will unblock once the data is there in the buffer at which point the data can then actually be copied into the process what the use of read might look like in code is this first we open the file say slash Alice slash Tim that returns the file descriptor which we assigned to F and then we invoke the read system call specifying which file we are reading from and the read Col whether it blocks or not will eventually return the data read from the file and if we're done with that file descriptor good practice is to release it by passing it to the closed system call now the interesting question here is how much data does the call to read returned well in fact how much data gets returned is not up to you is left up to read itself to decide how much data to return when invoking read you can actually specify the max amount of data you want to return the maximum number of bytes but it's left up to read to decide whether or not to return that much or actually less and again read works this way basically for performance reasons consider for example if you invoke read and request 4,000 bytes well if only 2,000 bytes are available in the read buffer that means your process would have to block to wait for the remaining two thousand bytes read we'll probably in this case decide rather than have your process block it'll just return those two thousand bytes it's then left up to your program to check how much data was actually returned by read and then call read again to get the rest while it is left up to read to decide how much data to return read is always guaranteed to return some amount of data when there's data in the file left to be read so this means that read will only return no data in the special case when you are attempting to read at the very end of the file so in the cases where you invoke read and there's data left in the file but there's nothing in the buffer read will not just return nothing it'll block your process and wait for at least something to be read into the buffer if read were to return nothing when they're still data left be read that would falsely indicate to the program that there's no more data left that the end of file has been reached consider now this code which reads in a whole file and prints all of it first of course we open the file / hello / Tim and we assigned a file descriptor to F and then we read from the file with read F this call most likely blocks but eventually it's going to turn some amount of data unless of course the file is actually totally empty in which case it will return an empty string and so for our loop we test whether the return data the string is not empty that is whether its length is not equal to 0 assuming it's not we then print that string and then we want to read more from the file so we invoke read again and assign it began to data and unless we've already reached the end of file that read will return some amount of data so they condition will once again be true and we'll go through the loop again and so we'll keep doing this you know every chunk of data we read we then print and then eventually we'll reach the end of the file at which point read will return an empty string so we'll then leave the loop and then close the file because we're done with it we haven't yet actually explained how exactly read and write nowhere in a file you wish to read from or write to well quite simply when you open a file with each file descriptor is associated a marker that is basically just a numeric index keeping track of where your next read or next write is going to take place by default when you open a file the marker starts out at the first byte and each time we do a read or write with that file descriptor the marker will advance that much into the file so say if you read 5 bytes then the marker will advance by 5 bytes now generally if you're both reading and writing a file it probably makes most sense so you have separate file descriptors with separate markers so you can use one file descriptor to read and the other to write and they don't interfere with each other now you may be wondering what if the marker is at the very end of the file and you wish to write or what if it's near the end of the file and the amount of data you're going to write is gonna go past the end what happens well in that case the file simply expands to accommodate however much data you're writing so in fact if you wish to create a file in UNIX all you do is simply open a file which doesn't yet exist and then start writing to it the file starts out empty but each write just adds more data now if you actually wish to shrink a file that's a different story we have a system call for that called truncate to which you pass the new size of the file so if your file is 5,000 bytes in size if you call truncate with the argument 3000 that will effectively lop off the last 2,000 bytes leaving you with the file of those first three thousand bytes and actually even though the name truncate implies it's merely for shrinking files you can also expand files with truncate and that effectively adds bytes to the end of the file and all those bytes will be no bytes they will be zeroed out until of course we actually write something else in their place now of course in many cases you don't necessarily want to start out reading or writing at the very side of the file or for whatever reason you just want to move the marker so we can do so with the system called called L seek the seek part of the name makes sense you're effectively seeking to some part of the file what the L stands for is sort of lost to history probably stands for long as in a long sized integer but we can't say for sure in any case the name is stuck and it's called L seek so when you call L seek you simply passing the byte to which you wish to move the marker if you wish to move the marker to the very end of the file like say you're gonna write to the end of the file to append more stuff there's a special value you pass in to make it go to the very last byte also it's actually possible to move the marker past the end of the last byte in which case if you then write you'll be effectively expanding the size of the file and all of the bytes in between past the former end of the file and where you're beginning that new write those bytes get filled in they become null bytes zeroed out bytes now when dealing with file descriptors it's actually quite important to understand that a file descriptor itself is really just a number in the process used to uniquely identify an underlying data structure what's called a description and the description is the thing which actually represents the open file and which contains the marker this distinction between descriptors and descriptions is important because they're actually circumstances in which the descriptor can get copied and you end up with two separate descriptors which both point to the same description so you'd have two separate file descriptors but they'd share the same underlying file marker when we call open however open always returns a new file descriptor with a new underlying file description so here when we open the file slash a Lesage 10 twice the two file descriptors F and F 2 both have separate file markers so when we write here with F and then read with F 2 the read and write are both done at the start of the file because F enough to have their own markers looking at this code may raise a question and that is when the read call is invoked does the data returned reflect the change in the file as written by the previous write that is is the data returned going to start with a blah blah well to answer this question first we have to look again at how descriptors are connected to the actual files stored on disk and that is you have a descriptor which points to a description and when we read or write via a description we're actually writing to a buffer in the operating system not directly to or from a file on disk the key thing to understand is that in most unix's including Linux there's only one buffer no matter how many descriptions are open on the same file so consider say a scenario in which a single file is opened and being used by 4 different processes these processes very well may end up reading and writing from overlapping portions of the file but what happens in each case is that each write ends up just overriding what already was in the buffer and each read simply reads whatever happens to be in the buffer at that time and very important to understand is that reads and writes are by default not atomic meaning when you invoke read or write it may get interrupted in the middle of copying the data to or from the buffer and so say here when we have two writes to the same portion of a file the data being written there may end up getting interweaved that is portions of the data from one call to the right may get overwritten by data from the other call - right and vice versa so what we end up with here is not necessarily herp derp overridden by blah blah or blah blah overridden by herp derp but possibly something resulting from them taking turns overwriting each other in the case of simultaneous reads there's generally not a problem because as long as the data is not changing then the reads aren't affected by each other however if a reed is intermingled with the writes then the change is made to the buffer by the writes may end up changing what data gets read so the important takeaway here is that by default in UNIX the file buffers are read and written with no coordination now some programs like say databases do require exclusive control over a file and for that purpose there are special mechanisms which we won't get into in this unit as previously discussed every file on UNIX has an owner and it has a set of permission bits that is it has a read write and execute permission for the user for the group and for what's called the other class so when we create a file how do we set its permissions but we can do so with a strangely named umask system call which sets the permissions for any file we henceforth create in that process the reason it's called umask is because there is a so called mask a set of bits each one of which corresponds to the permission bits and so we configure this bit mask by invoking you mask and passing in the new mask we want what you mask returns is the old mask the one with the old permissions in case we're curious what the permissions formerly were and henceforth any new file we open and write will be created with the permissions as set and that mask we passed to you masks and these permissions also apply to directories which we'll talk about creating in a moment now if you wish to modify the permissions of an existing file the system call for that is called chmod as in change mode we invoke it by simply passing in the path of either the file or the directory and then also the mask of the new permissions we want now of course it would totally defeat the whole purpose of permissions if any process could change the permissions of any file so for a call to chmod to be successful the invoking process effective user ID must be zero the superuser or it must match the owner of that file or directory' to change the owner of a file or directory we can invoke CH own as in change owner and of course we pass in the path to the file or directory and we specify the new user and the new group which will own this file and again of course for security purposes we can't allow just anyone to modify the ownership of the file or directory so for a call to CH own to succeed the effective user ID of the invoking process must be zero that is the super user or it must be the same as the user which owns that file or directory though in that case CH own is only allowed to change the group so the user passed to CH own must match the user which already owns that file or directory before discussing how to work with directories recall that each storage device is divided into possibly one or more partitions it's within these partitions that files and directories are stored and within each partition each file and directory is known simply by what's called an inode number a number which uniquely identify as a file or directory within that partition always within a partition you have at least one directory called the root directory which is always designated inode - why i know'd - well i node 1 is used specially for a list that keeps track of all the bad sections in the partition the parts that are unusable and I know 0 is used especially like a null pointer it indicates the absence of any file or directory now to be clear this diagram here is a bit misleading it seems to imply that directories and files are always stored contiguously that is that their bites are always written in order in adjacent parts of the partition but in truth a file can actually be scattered all throughout a partition in pieces in any case to create and remove directories we have the system calls MK there as in make directory and RM door as in remove directory and so here for example MK der with / house / Tim creates a directory called Tim in the directory Alice and then if we invoke RM der with the same file path it then removes that directory now to add files to a directory you simply create the file that is you open the file that doesn't yet exist and start writing to it and that creates it what that does exactly is it creates a new file in the partition with its own inode and then in the directory it creates an association between an inode and the name of the file and be clear that files only have names insofar as they are listed in directories stored with the file itself is no name whatsoever files don't have a concept of names really the inode has no name just a number holy and directories is there a name associated with an inode so in fact what we can do is we can actually place a single file in multiple directories once the files been created we can then add it to other directories with the system call linked so here for example we're taking the existing file slash Alice slash Ian's and we're adding that file as another entry in the directory slash been giving it the name Jill so now we have the single file with this one inode that is found with the name Ian in the directory Alice but also found with the name Jill in the directory been as far as the file is concerned neither one of these is its true name they are both equally valid they're just different listings of the same thing now to remove files from directories we have the system call unlink so here say slash Alice slash n this will remove the Ian entry in the directory Alice now stored with each I note is a count of how many times that inode is listed in directories and so every time an inode gets unlinked from a directory it's linked account is decremented and when that count reaches zero the filesystem knows that file can then be discarded effectively that means that the storage area which that file takes up on disk there's now free to be overridden for the use of other files and directories so it'd be clear that unlinking the last link to a file doesn't really delete the file if for privacy or security reasons you want to make sure that a file is really gone you should overwrite its entire content with random garbage to read the contents of a directory we have the system call get d ents as in get directory entries first off we open the directory just like with the file that returns a file descriptor which actually points to a directory and then when we invoke get the ence with that file descriptor it returns some number of entries from that directory now for basically the same reasons as the read system call we don't know how many entries exactly get the ents will return but it's always guaranteed to return at least one so when get the ence returns zero entries we know there are no more entries to read now each returned entry is a particular data structure containing most obviously and inode any associated file name but also the length of that file or directory and also something indicating the type that is whether this is a file or directory or possibly one of a few different file types which we haven't yet discussed in Microsoft Windows which are probably familiar with each partition of every storage device on the system is given a drive letter like say the main hard drive is usually one big partition which is known as C so the path to the root directory on that partition is C colon slash and in Windows the convention is to use back slashes rather than slashes the windows doesn't actually care it doesn't matter which one you use in UNIX things work quite differently rather than assigning each partition its own Drive letter each partition is mounted to a directory so first off in any UNIX system when it boots one partition must get mounted to the root partition designated with just a single slash once mounted the root directory in that partition is now synonymous with slash so the file path consisting of just a single forward slash that is synonymous with the root directory of the partition mounted at slash once the partition has been mounted to slash we can mount additional partitions to directories found on other partitions which are already mounted so here for example we have partition two mounted at root and if we have a directory slash jessyca a directory which itself is in the root directory of partition 2 then we can mount partition 4 to slash Jessica henceforth / Jessica now refers to the root directory on partition for not the directory in partition to the director Jessica and partition 2 is still there is just now effectively hidden we would have to unmount partition 4 if we wanted to get at the underlying directory again in practice we generally just don't mount partitions to directories which have stuff in them usually we just create empty directories for the purpose of a so-called a mount point in any case continuing here if we wish to mount partition 3 to slash Jessica / Vincent well first there must be a directory there so inside partition 4 inside its root directory which is now mounted at slash Jessica we create a directory named Vincent and then we can mount partition 3 there finally to mount partition one at slash Alice / Tim well then in partition 2 we're gonna have to create a directory Alice and then inside that a directory Tim and then we can now partition 1 there to mount and unmount partitions we use the mount and you mount system calls now when specifying the directory to which we wish to mount a partition and from which we wish to unmount a partition it's fairly obvious you provide that as a file path but as for specifying the partition itself that's actually specified with a special kind of file which we won't get into in detail in this unit anytime a system call expects a file path we can express that file path as either what's called an absolute path or as what's called a relative path in an absolute path we specify the whole path starting from the root of the system that is the root directory of the partition mounted at slash a relative path in contrast does not begin with a slash a relative path is automatically translated into a full absolute path by tacking on in front what's called the current working directory the current working directory is a directory associated with a process and is set in that process with the CH door as in change directory system Col so here for example I am setting the current working directory for this process to slash bin slash in when I then call open with an absolute path slash Alice slash Tim the current working directory is irrelevant this simply opens slash Alice slash Tim in the second call to open here though the path specified is relative so it is actually opening Alice slash Tim inside the current working directory which at the moment is slash bin slash in so this opens slash bin slash en slash Alice slash Tim what's the point of this well simply in some contexts it's more convenient if we don't have to write out full absolute paths so far we've only discussed what you might call regular files that is files with data on a storage device and we've discussed directories which are basically just listings of files and other directories but UNIX also has a number of other things which it also considers to be so called files in fact in the broadest use of the term directories are considered to be files these other file things include what are called symbolic links and also what are called device files or sometimes special files and those come in two types character device files and block device files and then also we have what are called pipes and sockets very briefly a symbolic link is a file which is written to disk like any other file but it doesn't have any real content it just has a link that is a file path pointing somewhere else and it's specially marked as a special kind of file it's not a regular file it's a symbolic link file such that any system call which opens a symbolic link will not open the symbolic link itself but actually the file pointed to by the symbolic link in practice symbolic links are very much like shortcut files and windows device files as we'll discuss in a moment represent hardware devices and they are effectively a clever way for our processes to send and receive data from devices using the same system calls we use to read and write files these device files don't really represent stored data like a regular file rather they're more like convenient fictions which appear to act like files pipes and sockets are both means for inter-process communication that is sending data between processes the difference being that pipes can only communicate between two processes on the same system whereas sockets can connect processes on different machines across a network again pipes and sockets like device files don't really represent any kind of stored data on a storage device they really are just sort of a fiction that allows us to do this communication using the same set of system calls we use to read and write files that's the sense in which these things are considered files so to create a symbolic link we have the system Cole sim link to which we supply the path to the file or directory to which we wish to create a symbolic link and then we also provide the path of the symbolic link we wish to create so here this creates a symbolic link of slash Jill slash Ken which points to the file or directory slash Alice slash Tim if we then open that symbolic link what actually gets opened is the file to which it points not the symbolic link itself that's pretty much all there is to symbolic links they're really not that complicated now as for device files first recall the basic relationship between the CPU and an i/o device devices typically have some number of registers which the CPU can read and write and that is how they communicate there actually isn't any way for the device to force the CPU to read its registers though some devices may send an interrupt signal to the CPU which then triggers it to go and execute a pre-designated chunk of code which then will tell the CPU to hey go read these registers other than that the CPU is basically in control so ultimately talking to a device means reading and writing its registers but at the level of processes we don't want to have to deal with such details we want to work at a higher level of abstraction so this level of abstraction is what device files provide for us a device file is not really a file but when we open a device file and read from it we are getting data from the device and let me write to it we are sending data to the device for this to work though we need a distinction between what are called block devices and what are called character devices very broadly block devices are generally devices with large storage areas like today a hard drive is naturally a block device character devices in contrast on the sorts of devices that don't really store much data data flows in and data flows out but the data is not really retained by the device it's more like acted upon as it flows in and out or to think of it another way with block devices it makes sense somehow to read and write to specific locations whereas with character devices the data simply goes in and out in sequence you are not specifying where in the device you are reading from or writing to so looking first at block devices storage partitions are divided into units called blocks like the bytes of RAM these blocks are numbered from 0 and they're all of a uniform size usually the block size on the partition is something which is a multiple of 512 so like say 4096 is a typical block size also like the bytes of RAM these blocks have to be each read in their entirety so if you want to read something in a block you have to read the whole block you can't just read part of it and even if you want to write to just a single byte in a block you actually have to rewrite the whole block so that usually means you'll have to read out the block to a buffer modify the portion you want to change and then copy back the buffer to the block so when an inode that is a file or directory' is stored on a partition it's stored on some number of blocks and those blocks actually don't have to be contiguous as I previously mentioned so say here if we have an inode 86 which is presumably some file or some directory it might be stored on blocks here one four and six so one consideration when deciding how big the blocks should be on a partition is if you make them too big then very small files like files taking up only a few bytes are gonna end up wasting a lot of space because files always get stored in whole blocks you can't have a block that's shared between multiple files so every file no matter how small always takes up at least one block now when it comes to reading and writing files on a block device as previously mentioned that there's always a buffer involved and the way it works in most UNIX is including Linux is that there's a buffer for each block and as we previously mentioned no matter how many overlapping reads and writes you have from however many different processes they're always just reading and writing to the same single buffer so there's always just one buffer per block anytime the buffer gets written to that block is marked as so-called dirty meaning that the content in the buffer no longer matches what's actually on the disk so it needs to be copied to the actual storage device so that explains what happens when you write files to block devices but what though is a block device file well a block device file is a file which effectively represents the whole storage area of that block device such that the first byte of the first block is by 0 and then the last byte of the last block is the last byte in the file when we write to a partition this way we are actually circumventing the whole system of files and directories on that partition this way we're actually reading and writing the raw bytes themselves and so in fact when we write to a block device like this we can very easily screw up the files and directories on that partition because of course when files and directories there has to be some extra information written that keeps track of how the files and directories are actually written on disk namely which blocks they occupy and in what order so reading and writing a block device through a block device file is not something we normally do but the capability to do so is usually provided because well some programs simply have very special needs like for example databases as you mentioned have special needs when it comes to storing and retrieving data so a database may in fact have a partition like this set inside which it reads and writes in this manner again though this is not the usual thing to do and it probably occurs to you that this is an obvious security hole block device files pretty obviously should be given restrictive permissions you don't want just anyone reading and writing arbitrarily tiny partition character devices differ from block devices and that first of all there's no concept of blocks so instead of having a buffer for each block there are actually just two buffers one for input and one for output when a process writes to a character device file it is writing to that device files output buffer and when a process reads from a character device file it is reading from its input buffer these input and output buffers are usually so called because the character device file usually represents some kind of actual Hardware device and so the input which the device receives it writes to the input buffer of the device file and the data written to the output buffer is sent as output to the actual device so the other big difference here is that not only do character devices have no concept of blocks they also use a separate buffer for input and output the explanation for this difference is at the input and output buffers of a character device file are both what are called FIFO they are first-in-first-out buffers a FIFO is effectively like a line a queue of people waiting for something when people join a line they join at the end of the line and it's the people at the front of line that get served next likewise in a FIFO buffer the bytes that are written to the buffer are appended at the end and the bytes which are read from the buffer are always read from the front so to make this absolutely clear in the case of the input buffer of a character device file the device will send data at the input buffer which then gets appended to the end and then when a process invokes the read system call on that character device file it reads the bytes at the front of the buffer once read those bytes are then removed from the buffer conversely with the output buffer data is added when a process rights to the character device file and the data at the front of the buffer is read by the actual device and again once read the data is removed from the buffer now this is just a logical picture of how a buffer is actually supposed to work I like to picture it as if when data is read from a buffer the romanian data slides forward to the front but of course that's not what actually happens because in practice what that would involve is actually copying all the data byte by byte each time something's read and that would be very inefficient so really what goes on in the character device file there's something which keeps track of where the next read in the buffer should occur and where the next append of new data should occur but those are details handled by the operating system we don't really have to concern ourselves with them on the other hand these buffers are generally kept in size they of course can't hold an infinite amount of data so what these caps can mean is that when you write to a buffer the write and they fail because there's not enough space left in the buffer aside from that you really shouldn't have to concern yourself with exactly what's going on in these buffers let me reiterate the core difference between block devices and character devices block devices are for devices that have some large kind of storage space and so each buffer for a block device is actually backed by storage space but character devices generally aren't backed by any storage space with character devices typically what's happening is that there's these two buffers in the operating system which are serving as a channel of communication between processes and some actual device a device which typically has a small number of registers but maybe no other storage so again character device files are usually for this sort of device which immediately acts upon the data which it receives as output as we'll see in the next unit primary examples of character devices include terminals how exactly character devices get created differs greatly from one UNIX to the next generally is something handled specially by the operating system during boot time by convention though device files should be found in a directory placed at slash dev dev of course short for word device so here for example I'm opening to device files one is slash dev slash SDA one and SDA one assuming the system follows the usual convention should refer to the first partition of the first storage device on the system SD here actually originally stood for scuzzy device but some people have read Conda and now say that it stands for storage device in any case a device file representing a partition will of course be a block device file and the second line we're opening a character device file LP here standing for the archaic term line printer so this should be a character device file which I can send data to to actually print something out on my printer assuming I have one don't ask me why line printers are numbered starting from zero but partitions are numbered starting from one now that I've opened these two files I can now work with them such as say reading from them and notice that I can invoke L seek on the block device file but I couldn't do the same with a character device file character devices have no concept of file markers so calls to L seek on a character device file won't do anything it should be fairly obvious what we get when we read from the block device fowl here we'll get back whatever data is stored starting at byte 100 on that device but in the case of a character device as less obvious in the case of a printer it's obvious why you would want to send data the device because you want to print stuff but understand that you may also need to read from the device to say check its status if something's gone wrong with the printer and it's not ready to print that's something your program would probably want to know in the dev directory they'll also find a number of what are called pseudo device files these are mostly character device files but they don't represent actual hardware devices they're simply abstractions provided by the operating system for example the sudo device file slash dev slash zero is a character device to win red will simply return an infinite string of Nold bytes of zero doubt bytes similarly the pseudo device slash dev slash random will return an endless stream of random bytes and lastly another commonly used pseudo device file slash dev slash null doesn't return any bytes at all instead it simply exists for when we need a file to which we can write data and have the data just to go nowhere and disappear it sounds a bit odd but strangely enough that does actually come in useful sometimes honestly I find the name a bit confusing zero and nulls sound like they're kind of the same thing though they're not null should have probably been called discard or something like that now I think I've been clear that device files whether they're block files or character device files or whether they're pseudo device files they don't necessarily represent stored data but on the other hand there has to be some kind of representation for these so-called files that is in the dev directory for every device file there has to be an entry of this name corresponds to such-and-such inode and there has to actually be an inode representing the device file in the inode stored on the partition is a special indicator that this is not a regular file this is a device file so in that sense device files are stored like other files it's just as no stored data associated with their I nodes what we call in UNIX a pipe is basically just a single FIFO buffer used as a channel of communication between processes when a process rights to the pipe data is appended at the end and when a process reads in the pipe data is taken off the front so when say process B wishes to send data to process a it writes to a pipe and process a then reads from that pipe generally it makes most sense to treat a pipe as unidirectional so one process rights and the other one reads if you want to send data in the other direction you should use a separate pipe to create pipes or in fact to create device files and even regular files we have a system called called MK no doesn't make note here note is used sort of like a generic term for file when we invoke make node we specify the name of the file we wish to create and we specify its type and in the case of block and character devices we have to specify the device number as will explain in a later unit each device in UNIX is given a unique device number now again be clear that these files though stored as eye nodes on disk themselves do not have any data stored with them so when we write data to a pipe that data only exists in memory it is not rent to the partition on which we have this pipe file now when you create a pipe in this manner as we do here with the file Ryan slash Tina this is what UNIX calls a named pipe because it actually has a name it has a name in some directory you can also though in UNIX create an anonymous pipe or what's just simply called a pipe and you do so with the pipe system call what pipe does is create a new anonymous pipe and returns to file descriptors each point in two separate file descriptions though both of those descriptions are opened on the same pipe the first one open for reading the second one opened for writing the idea is that we create a pipe in and fork off another process then one of the two processes are that the child or parent can write to the pipe using one descriptor while the other process uses the other descriptor to read from the pipe and as mentioned a pipe is usually for one-way communication so if you want communication in both directions you should open two pipes to be clear when you create pipes in this way it's only useful for communication between related processes that is two processes where one is a descendant of the other because open file descriptors are passed on to child processes they can both see the same pipe created in this manner for unrelated processes you would have to use a named pipe another feature of modern unix's is the ability to map a file in whole or in part two pages of memory what this effectively means is that pages in a processes address space get specially marked such that reads and writes to those pages actually trigger reads and writes to some underlying file so here for example we're opening a file slash Brad slash Mike assigning it to a file descriptor F and then when we invoke M map to map pages of our address space we pass in first the number of bytes we wish to allocate but now we're also passing in a file descriptor and then specifying with a file offset which part of that file we wish to map to memory so what this called the ED map will do is map the 500 bytes starting at byte 200 in the file to pages in the address space such that when we now read and write from those addresses we're actually reading and writing from that part of the file when done with this work we should then release the pages of memory and also close the file actually it would have been fine if we'd closed the file after we did the M map we don't have to keep the file descriptor around for the sake of the mapped memory you're probably wondering what is the point of memory mapped files well it tends to make sense when your reads and writes are not contiguous and clumped together they're just spread everywhere when this is the case it can be considerably easier to write code with a memory mapped file than with regular reads and writes and also it may end up being more efficient in fact depending upon your UNIX system and depending upon some options when you invoke M map the memory map this way into your process are generally the very same buffers that are usually used when we read and write from files so this avoids the usual extra work of copying data from your process to the buffer or from the buffer to your process I would say that memory mapping files is used much less commonly to read and write files than simply using the read and write system calls but it is something that comes in handy for the programs which can make good use of it what eunuchs calls signals are sometimes called software interrupts because there are somewhat analogous to the hardware interrupts sent by hardware devices to the CPU a so called software interrupts however is sent by the operating system to a process which then must somehow deal with a signal either by performing a default action associated with that type of signal or by invoking a handler function registered for that signal or possibly by blocking the signal or ignoring it blocking here meaning that the signal is queued up until the process then unblocks that type of signal and ignoring here meaning that the signal is just discarded and totally forgotten now in most modern unix's are between about 30 or 40 different types of signals and the reasons for why these signals are sent varies from type to type many of the signals are sent in the event of some hardware event especially some kind of error so for example for signals found on all unix's include sig sig V where sig of course stands for signal and seg means segment as in segmentation fault don't ask me what the V stands for actually and then sig FPE where sig stands again for a signal and FPE means floating point error and sig stop is signal meaning stop and say containing signal continue sig seg V is the signal usually sent in the event of some memory error such as when the process attempts to use an address which isn't currently allocated in its address space the name seg v as in segmentation fault is actually a bit of a misnomer it's an archaic holdover from when memory systems were based around the scheme of segmentation rather than a system based around pages more appropriately it would be named something like sig page f as in page fault instead we're stuck with site V for historical reasons in any case when a process is sent the sig seg v signal the default action is to simply abort the process however if a process registers a handler for that signal that is it registers a function to invoke when that signal is received then that function will run instead in the case of a memory error a process probably should quit because it seems something has gone quite wrong but by registering a handler for this signal we can at least run some cleanup code before exiting say giving us a chance to preserve some important data or something the cig FPE signal the floating point error signal is usually sent to a process because it processes a template to divide by zero when your program tries to have the CPU divide something by zero the CPU is going to throw a hardware exception and that's going to trigger the operating system to then send your process the cig FPE signal and again for the signal as in fact for most signals the default action is simply to abort your process to terminate it again however if you've registered a handler for this signal then instead that handler will run when the signal is received and if appropriate your handler very may well choose to terminate the process though nothing is forcing it to do so if you just let the handler return the process will continue from where it got interrupted the sig stop signal as the name suggests is sent to a process to stop it the default action when a process receives six-top is to go into a block state the process will then only unblock when it receives the signal in the case of six-top and sig can't these signals are not sent in the event of an error condition they are sent explicitly from one process to another using the kill system call kill allows us to send any signal to a specified process the name kill very misleadingly implies that signals always kill the processes which receive them but really that's not the case it just happens that the default action for most signals is to actually kill the process to terminate it still kill would have been much more sensibly named something like send signal it's just one example of a long UNIX tradition of stupid crazy names that we've been stuck with for 30 years anyway the two most basic system calls associated with signals are first kill and also signal which is used to register signal handlers so here for example we're invoking kill to send the sake stop signal to the process 35 and the invocation of signal here is registering for the current process a handler for the signal sig FPE and the handler is a function which we pass in here called func last thing we'll note about signals is that of course for security reasons you wouldn't want any process to be able to send any signal to any other process consequently unless the process has super user privileges it can only send signals to processes owned by the same user

Channel: Brian Will

Views: 82,201

Rating: 4.9645777 out of 5

Keywords: programming, unix

Id: 2DrjQBL5FMU

Length: 51min 17sec (3077 seconds)

Published: Wed Mar 23 2011