Go Class: 32 Error Handling

hi my name is matt holiday and welcome back to my class on programming ago so in this section i want to talk about error handling first i want to talk about how we can use the more advanced features of go error handling to do a better job okay and that really means instead of just making a string can we do some other things with errors and i'll show how we can do that but the second part is more about a general concept of how go approaches error handling compared to other languages and why it's a good model in a lot of cases an error is just a string in a lot of programs we just call a call like bump.rf and we create a string value now that gets wrapped in some internal object in the standard library that represents an error but it's not very sophisticated it's a string and we can print it out and look at it and that's good for just debugging things but sometimes we need to do a bit more okay so we recognize that the error object is actually interface it's anything that has an error method so we can create other concrete types that can represent this right you know i've got again my phys gig joke here but we can create an object with an error message it returns a string value from error but it can have other data in it and that's what i want to talk about now is how can we create a more complicated error type to do more useful things so i'm going to build out a custom error type and this came from doing some wav file processing so i'm going to create a wave error now in order to define that i'm going to create an enumerated type for the kinds of errors the underlying things and i'm going to use those as tags inside a struct okay so my wave error type is a struct and one of the things it has is the kind of error and the kind of error comes from these tags over here okay right based on some enumerated type right and i may also have some other value i can carry along and we'll see how i can use that okay now wave error is going to be an error because it's going to implement the error method and in this case it's a little more sophisticated because i've got different kinds of errors and some of them may have an underlying error or some of them may have this value that i'm going to use in different ways and so my error method is actually a switch statement that looks at the different tags and does different things right it could be a case where it's just a plain message it could be a case where i use both of those values or maybe just one of the values that's embedded in my error structure type the concrete wave error that represents an error right and i'm going to show you a couple helper methods i built right and i'll explain why i have those but one of them each one of them actually takes a value of my type adds some information to it and returns a new value so it's copying right and basically it depends on whether i want to stick one thing in it or two when i copy right pretty straightforward these are just methods they're private methods on my type i'm also going to define some error variables and each of these variables is just an instance of my struct with a particular tag in it now in the past i didn't used to export the type or the variables there wasn't any use right i create the type it becomes an error interface thing which somebody prints out but there wasn't any reason for them to look at the variables or the type there is now and i'll explain why in just a second so now i'm exporting these variables but how do i use them so here's an example piece of code i'm going to start decoding a header of a wav file so i start looking at my bytes and if i don't have enough bytes well i can return one error and i'm just returning in this case the variable it doesn't have any other information other than well you don't have a header but then i start trying to decode parts of it right and maybe i get an error that's a little more complicated and what i'm doing here it's an interesting technique i'm taking my original error variable as a prototype i'm modifying it by making a copy and putting into it particular details of this error in this spot and then i return that new value so my error variable doesn't change which is good because i might need to use it somewhere else but i'm returning a copy of it with particular details now there's other ways i could do this i could create a big constructor in line here i think this is clearer i think it helps understand okay exactly what i'm doing right is taking this kind of error prototype giving it some information and then returning it so i think it's a neat paradigm why are we exporting the types and variables well there's a new capability starting at go 113. it's called wrapped errors okay now the way you may commonly see that is in like thump dot error f using a new format percent w which says take this error and wrap it so now what pump.rf is returning is a little more complicated it's still an internal thing but it allows us to chain these errors together okay it's not just one big string with you know and again the way you see this typically is top level error colon second level error colon third level error all right but in many cases that's just a string and the problem with having it just be a string is sometimes i need to ask the question well did i get an error of a particular kind so i do some complicated logic i get back an error and then i do a string search to see if it has the words not found and say oh i got a 404 let me deal with that right the problem is the string search method of looking at errors is fragile right is there a better way and the answer is yes so the notion of wrapping errors takes us to this concept of an error chain because instead of just having a string where my wrapping notion is well thing colon thing colon thing i now have a structure under the covers where my top level error actually has a reference to another error which has no reference maybe to another error the actual original error okay and now i can walk this chain and do do some things and i'm going to show a couple examples now the notion of wrapping errors means we can unwrap and in fact i can now take my custom error type and give it an unwrap method and if it has that then it's unwrappable and we'll see that in action right now all i'm doing is returning the underlying error it might be nil in which case you know the functions that deal with unwrapping errors will say oh well it's nil that means there wasn't any other error i'm at the bottom of the chain okay it's kind of like a a list a linked list if you will and i get to the end if it's nil right so i don't have to do any thinking here i can just return the embedded error and i wrapped it in the case of if i go back a couple slides right right here i created a header read from error and i wrapped the other error inside of it just by putting a reference to it in my error structure into my concrete type that represents the error right and so it became you know one of these maybe right the header read error is the top level it wrapped this and that might also wrap something else some underlying problem beyond that right and again so i can unwrap them that brings us to a couple of useful utilities and the first one is called errors.is and again this is new with go 113. okay i want to know if a particular error is part of this error response and again in the past i might have tried doing a string search guessing what i think is going to be in the string produced by calling the error method again that's fragile instead what i can do is i can use errors.is to say hey does this error value show up in the chain okay and it's based on a variable which is why i exported the variables from my custom error type because now there's something you can actually use in a type of comparison and is is a fancy way of doing the comparison it starts with whatever error you have and it walks down the chain trying to see is this error match the error in the chain and if so it will return true so here's an example right if i have a decode wav file and it needs to open a file i gave it a file name right and if you look at the code in os.open it returns a path error but a path error can usually wrap an underlying problem okay so i probably have a chain of three errors i have my top level error from the my wave pile package which is wrapping a path error which is wrapping an actual underlying error and the one i'm looking for here for example is os dot error permission now os dot air permission is a variable of error type okay and i'm just saying hey is this error have one of those somewhere in the chain because that's a way of saying well this is that kind of error right the underlying cause of my failure was a permissions error and maybe for example i want to log that for security reasons so i look and say hey did i get in a permissions error okay deal with it and we do that by using errors dot is okay it's a much safer way than trying to parse the string or do a string search in the string to see if certain words show up now i can make advantage i can take advantage of this right with my custom type because in addition to doing an unwrap method i can add is to my custom error type and what i'm doing is i'm saying hey you gave me this error is it me and what it's going to be called on okay remember when we called error.is we gave it a variable so you can imagine that this is being called where the receiver is one of my error variables like header header missing i can't remember what i called it header missing or you know header read from error or whatever okay one of my error variables is the receiver so this method is being called on one of my variables given some other error okay is it the same error well the first thing i've got to do is i'm going to use reflection and again i'm going to punt i'll describe reflection a little bit later but i'm asking first is it my type of error is it a wave error and if it is that means i can go and get the tag and then say okay it's one of my wave errors does it have the same tag and my logic is if the tag is the same then it's that kind of error the fact that it has a different underlying error or the fact that it has a different position value or something doesn't matter i'm just going to say if it has the same tag then it's that kind of error right and that's i think good logic so i can not only provide the unwrapped method i can provide the is method on my custom type and that allows people to go figure out is did a particular error happen without trying to guess at what string they should search for okay go has one other method okay so is again it operated on a variable not a type okay as operates on a type not a variable and what it says is can i go into the error chain and extract an error of this type okay you can almost think of it as a kind of down casting right given an error can i downcast it to an error of this type now the error might not be the top level error it might be in the chain okay so if i look again at my decode wav file right it may return an error because it couldn't open the file and i said os dot open is going to return a path error and the path error might wrap something else but let's not go there okay path error in this case is not a variable path error is a type if we go look at os.path error it's a structure which has among other things the file name or the full file path and the underlying error okay so if i'm using errors dot as what i'm actually saying is attempt to extract a path error from this error okay and if it works okay so i have to give it a variable to copy into i'm going to give it an the address of an os dot path error that i've declared okay so i pass that in here and i pass in the error and if it returns true okay that means my os dot path error actually has a path error data in it so it was a path error in this chain and the information of that path there has been copied to my local path error variable that i declared it's been decoded into it great and that means i can do something for example maybe i don't want to pass back the wrap the wrapping error maybe i just want to pass back the underlying error or i want to do something else with it okay so again this is a way of finding the error by type rather than by name where name was a an error variable so again errors dot is and errors dot as help explain why now i actually not only have a custom error type but i export the custom error type in the variables because this gives people a powerful way to figure out what is the underlying error and doing it without using string comparisons in the last couple three years there's been some controversy about how go does error handling and there been some suggestions about changing it which didn't happen so now i kind of want to explain why okay and it has to do with these you know we keep writing if error not nil return error things and some people find that obnoxious okay i think it's actually practical and i want to explain why so i want to break down errors into two kinds and i'm going to start with what i call normal errors and a normal error is an edge condition your program caused by either something you got as input right you got input from a user of the program or you got input by doing a network query and get a response or the external conditions the program is running in like is the network available are you out of memory okay now to my mind these edge conditions these error cases they're things you should expect to happen as you write your program if you're writing your program to use a network then it's possible that the thing you're trying to talk to isn't there or the network isn't working at all if you're dealing with files it's possible that somebody's going to give you a file name that's mistyped or the file's been moved or the disk is out of space and you can't write any more to it and these are perfectly reasonable things that you should deal with when you're writing programs that use files or networks or whatever okay they're just part of your program logic now here's an example from open and i've simplified this but it shows how we do this right we go into open we try to open the file if it doesn't work we create an error type okay in this case path error is a type of course it implements error so it fits in the it satisfies the error interface and so that path error comes up here and gets returned as the error from the function okay and so we return errors as values they're just values getting returned from a function you assign them to something you check them right just like this all right great what's the other kind of error abnormal errors and i'm going to put them all in one bucket the abnormal errors are not edge conditions caused by other things there are edge conditions caused by you the programmer they're your bugs in the program okay and they're abnormal because we don't write programs to have bugs those are not foreseeable okay i mean obviously every program has some bugs but we don't write a program with the knowledge of oh i'm going to have these bugs in it okay and so an example of that is a nil pointer or reading past the end of an array and go handles that by doing a panic which normally crashes your program here's an example from the go standard library right we're calculating the final checksum of like a secure hash right so we've processed all the bytes and now we're doing this little check that says well there shouldn't be any more and if there are panic because if there are we had like an off by one bug okay it has nothing to do with the bytes that were passed in it's not the fault of the bite slice that we're processing it's the fault of the logic that's doing the processing okay that's an abnormal thing and there's not really a way of handling that in the program other than go fix your bug rebuild the bug redeploy the you know rebuild the program and deploy the program okay there's no other way to handle that really and i'm going to explain why here in a second okay so hold on to your seats because i'm going to give you an idea that's one from experience hard fought right if your program has a logic bug crash it fail hard fail fast okay now i'll put a little asterisk next to that and say unless it's a safety critical program but people don't do safety critical software and go for obvious reasons okay so why take this approach i'm going to give you three reasons pardon me if this slide is a little complicated but i'm going to walk through all of these so the first thing has to do with how we develop software okay and this is again this is from hard experience you have some bug in the program you throw an exception it gets logged and the program goes on okay maybe the program just treats it like okay i'm going to respond with a 404 and log the exception but the exception might have said hey i'm getting ready to destroy your database okay when people test your program they don't go looking through the log file to look for exceptions because the log file frankly is almost always unreadable in most programs i've worked with okay but if your program crashes the system testers are going to write you a bug report and that means that bug gets surfaced earlier looked at and then somebody can say oh my god this is almost going to be a disaster if we don't fix this okay the second one has to do with debugging i've also worked in systems where you have different processes in a distributed system so i send an rpc to a process it has an exception and what it does actually is it wraps up the exception and sends it in the response rpc back to me okay that makes it hard to debug because what we'd like to do is get the evidence of the problem close in time and space to where the defect in the code was okay close in time meaning as soon as the defect happens we want to get some evidence and close in space like ideally right in that function or very close to it not in some other part of the program or even in another distributed process in our system now both of these first two reasons are good valid by themselves but now i'm going to give you the biggest reason and it's really serious okay because we typically use go to build distributed systems and in a distributed system the easiest and safest failure to deal with is what we call a crash failure the process just dies it goes poof it's gone and in most well-designed distributed systems that's okay because typically you have more than one instance and you have it behind a load balancer and so you know one goes away the other ones can handle the traffic the system will create another instance why is that better well here are the alternatives the alternatives and what i mean by where i'm going with this and maybe i should have said this already if your program has a logic bug it's corrupt and maybe now it's in a corrupt state internally okay if we let it run the possibility is it's going to continue in that corrupt state and corrupt other things okay it could turn into a zombie and simply consume resources okay and become part of a dos attack eventually or it could start babbling bad messages on the network and corrupting the other processes and that gets a cascading failure your whole system goes down okay been there done that or it could silently corrupt your database as it continues until the problem is surface somewhere else bad enough that then something finally happens but it's too late now your database is trashed shut the system down try to find a backup okay if we don't have crash failures we may end up with what's called a byzantine failure and i'm just going to say they're bad there isn't time here to describe how bad but byzantine failures are very hard to deal with crash failures are safe and easy so the best reason to just panic and die when your program becomes corrupt internally meaning it has a logic bug is that's the safest thing to do crash get out of the system we'll pick up the pieces and look at it from a debugging perspective and the rest of the system hopefully will continue on safely when should we use panic right we should use panic when our assumptions about our code are wrong if we get a nil pointer error we must have assumed that the pointer wasn't nil and so we dereferenced it and we were wrong okay our assumption that the pointer wasn't nil was wrong okay if we look at the other piece of code about doing the hash the assumption was we've gone through all the bytes oops there's a byte left over how did that happen it's an off by one bug okay so we panic when our assumptions about our program are wrong great here's an example a b tree is a complicated data structure often used to build databases okay it's like a tree it's got a bunch of rules and i've listed them here just for the heck of it okay if any of these rules don't apply if they're not true before or after an operation on the b tree then we've actually corrupted the b tree it no longer operates or behaves the way a b tree is supposed to and if we keep going down this road we're going to end up with a corrupt database and so my argument would be you know if we detect these conditions as we're working in this b tree we should panic we don't get here because the user gave us bad data we get here because our logic of processing the b tree made a mistake okay it put the wrong thing in the wrong place and now our data structure is corrupt because our program is simply wrong so an alternative approach is exception handling it was developed in the 60s popularized in the 70s with the programming language ada for safety critical systems now remember i said fail hard fail fast is not a good strategy in a safety critical system instead we have this notion of what's called graceful degradation and imagine i'm flying an airplane with two engines one of them fails i would like to continue flying on the other engine as long as i can it's better than crashing because in this case crashing means people get hurt okay now ironically people don't use exceptions building safety critical software and the reason for that is what it does to the complexity of the software okay we have a notion of measuring complexity and there's a tool called cyclomatic complexity i'll talk about when i talk about linting okay it just measures how many paths there are say in a function how many different control pads or logic paths when we add exceptions we just balloon that number out of control okay if i look at an ordinary function maybe it has two return statements so there's two ways to return from the function but if i have exceptions every line of code that's not a comment could throw an exception and take me out of my function and analyzing those with static analysis tools turns out to be hard all right and if it's hard to deal with a tool that means it's hard to do it as an individual person doing a code review it's just hard to try to find all those invisible control pads looking at the source code because they're invisible they're not written in the code really not much right now in theory go doesn't have exception handling in practice it sort of does i'm going to show you what it is and then i'm going to tell you why not to do it right we have panic all right there's also another thing called recover where you can recover from a panic and so i'm going to give you a very simple example of that right it's a little bit of code and this thing is just going to print recover colon omg right my main function calls abc abc panics and inside this defer and the only place we can do recover is inside a defer we have a call to recover and if the call to recover returns a pointer that's not nil that means it's capturing the results of a panic some piece of data which in this case is just the string came back and we're just going to print it okay so in a sense we've done exception handling okay the model here is probably closer to what in the c language was called set junk set jump and long jump all right but i'm not going to go there what i really want to get at is well i have this little snarky comment i'm printing out the results of capturing this panic in the recover block what else can i do and the answer is in most cases i don't know what i can safely do okay if i'm getting a panic it's because my program has a logic bug and has entered a corrupt state well for all i know it's still in some corrupt state but i don't really know at this point exactly what the corruption is okay now this might make sense and there's a technique that's been used in safety critical programs you enter some region where you can do a panic and a recover and you start by doing a checkpoint of the program state and that way if you hit the panic and recover you can actually return to a known state by taking that checkpoint and putting all the variables in back in the program where they were and people don't do it very often because the way i've described it it's very expensive it's kind of like in a database where you do a transaction and a rollback which is actually kind of expensive but it's just it's part of the cost we assume in a database in your program it's harder because your entire program state is the transaction every variable the stack conceivably everything it's very hard to know what that state is and put it back to a known good state and if you don't well then you are the possibility that you don't really know where you are okay so there are a few uses of recover it's actually used effectively sometimes in running unit tests where i can actually run a test and if the test creates a panic i can capture the panic and produce an error message perhaps in a better way than just crashing the whole unit test or the whole set of unit tests okay that's not a bad use i've seen one or two other uses that are okay but in general i would say calling recover is a code smell it's just not a good thing to do we do not want to bring the exception handling model of other languages into go okay now i want to finish this with a couple thoughts about better ways to handle errors and the best single thing we can do about errors is just not to have them okay an error represents an edge case in the program and the more edge cases we have the more complicated the program becomes the more logic it needs if then else statements and so on okay or we can try to do this very useful idea of defining the error cases out of existence where we can that reduces the number of edge cases and reduces the complexity of the program and there are some examples for example built into go did i just say that yeah okay you know it's okay to append to a null slice it's okay to read from a nail map it's okay to take the length of an initialized string if we think about other languages in c an uninitialized string is just a character pointer with random data in java it's a pointer that probably is nil okay that's safer but in both cases i've got to do some logic before i can calculate the length of a string to make sure i haven't blown myself up right in go an initialized string is just the empty string and if you ask for the length you get 0 because that's very logical and so i just removed a whole bunch of if statements from my program that would show up in other programs in other languages okay i've simplified things i've reduced the edge cases i've reduced the complexity all right i've handled the errors by simply defining them out of existence okay that's one thing the other one is about how to write a program now this is the kind of very simple you know logic that used to be written into introductory programming textbooks 40 years ago and for whatever reason now we don't go there a lot of the modern textbooks they're about this neat new language these neat new language features this neat new framework or vague ideas like computational thinking great but we also need to think about the practical aspects of building programs so here are some rules that again are going to keep you out of trouble right every piece of data should start in a valid state every transformation should take data from a valid state to another valid state and we should never accept any kind of input at face value we should validate it let me start walking from the third one okay for the last 40 years about half of the security problems on the internet have come from violating rule number three some clever programmer said oh no one's ever going to type more a thousand characters on the command line so i'll create a buffer in my program i'll make it a thousand characters long and i'll take the input string and just copy it to that buffer without checking it because nobody's going to type a thousand character string and the buffer lives on the stack and so the wiley hacker figures i can write this 1200 byte string smash the stack and take over the program okay because the data wasn't validated on the way into the program right let's look at the first one every piece of data should start in a valid state one of the worst system failures i've seen in my career started with an uninitialized pointer value and because of the type of programming language it was it was a random value from the stack at least in go if you have an uninitialized pointer it's nil and you'll get a nil panic and that's pretty clean compared to what happened in this particular case i'm thinking of where that random value on the stack actually caused a chain of errors that brought the whole system down okay all right that leaves us this middle piece and i'm going to tell you the middle piece is hard that's where the rubber meets the road that's where practical programming skills discipline of programming matter you really have to stop and think about what's going on and what are you doing and how to deal with it okay and most importantly you need to avoid being clever the clever guy is the one who said nobody's going to write a thousand characters into my buffer i don't need to check the input okay the clever programmer is the one who assumed i don't need to make sure i set my pointer values to nil if the language doesn't initialize them to nil on its own and there's a bunch of other things here and i'll really get down to the last two are get never ignore errors again how many c programs on the internet call some system called don't check the error and when it's wrong then they start failing in weird ways and the last one of course is just the absolute most important i'm going to talk about this more test and test and test again you can't test enough okay so this is just the kind of basic logic that we need to think about when we write software and if we do that it's going to put us in a much better position okay why does this matter to go so there's a philosophy and i pulled out a quote from dave cheney's blog about why we do this right and again people think that this little section of programming that's verbose to have all these if they're not nil things okay but dave chaney's point is we handle errors we don't just kick the can down the road and i want to follow that with one other thing and that is what you get out of this is visibility right the logic in the program is explicit it may be verbose but it's explicit it's right there in front of you what are the control paths what happens in an error you'll see it you'll see it when you look at the code and when you do a code review right as opposed to having it magically happen behind the scenes right so this notion of making the errors visible and handling them is why go takes this approach and not the exp approach of exception handling okay so that's a little bit more on error handling and go i talked about how we can make the error system a little more fancier by creating custom errors and providing some additional methods like unwrap and is okay and then i talked a bit about the logic of why we do errors the way we do and go and why go prefers this type of error handling to exceptions

Channel: Matt KØDVB

Views: 9,323

Keywords: Go, golang

Id: oIxXp0OgK_0

Length: 33min 55sec (2035 seconds)

Published: Sat Jan 16 2021