ElixirConf 2021 - Greg Vaughn - Fumbling with Exceptions

[Music] [Music] and i'll start by thanks for coming to elixir comp 2021 it's great to see people in person uh it's really cool that this conference center is where it began the first two elixir conferences were held here so it's kind of a sense of nostalgia here that's kind of fun coming back my name is greg vaughn and i'll be talking to you today about fumbling with exceptions i picked this topic uh kind of in a way that made it easy on myself because just in case i even fumble during this talk you're still guaranteed to have an exceptional talk good crowd good crowd okay but yeah i've set the bar and the level of humor it doesn't get better than that but all right is the mic on am i not close enough maybe i should be close enough okay i'll do that um this is how you can contact me online uh various elixir communities i've been in elixir community since the first elixir conference and i want to do a shout out to my new company dscout i started with them in march they're sponsoring my trip here and they're sponsoring the conference also well we're all in on elixir we do qualitative research which means like academic and market research user experience use our platform to connect to panelists in many different ways including live interviews media rich sort of options for generating qualitative research which we then consolidate and we're growing we're hiring and if anyone is interested to talk about the company or openings please find me during the conference i love meeting new folks so let's jump in uh you've probably heard uh in the elixir uh erlang communities to let it crash you know we don't handle errors we let it crash and the first time i heard that my jaw dropped it's like that's not a resilient system but the important thing to understand it's not let the whole system crash it's each independent little process can crash and not affect the whole because we have supervisors and that's really important in the large scale design of your system super supervision trees should be probably 75 80 percent of your error handling but i want to focus in on some of the smaller parts here in this talk kind of instead of the concurrent code which is how the different processes communicate with one another i want to focus in on the sequential code that's inside of one process because each process is one thing at a time it doesn't share memory which is part of you know what i love about elixir so i'm going to be focusing on in on that section of the error handling we're going to go over some of how erlang uses errors and exits and their throw catch keywords the data data structures that they use elixir adds its own richer exception data structure on top of that and elixir also will even wrap those erlang errors in many cases uh some of you have been with elixir for a while may have gotten some of those errors that got thrown up from the erlang code that was just like nested nested tuples and lists and tuples of lists and it was really hard to parse and so elixir has done a great job working on developer experience and how we can understand uh exceptions so let's jump into code i pulled this off of the um the github repo for for the elixir standard library and i've simplified things to fit on the slide took a lot of comments but this is the core idea of what is in the exception.ex file uh i encourage everyone to read the the source code um to the core libraries it's most of it is very readable unless you're into some really complex parts so the first thing it's doing is it defines a type which is a data structure that's a map with some required underscore struct and exception fields and anything else you want to add to it but at its core it's a behavior it defines three callback functions uh two of them are required and one of them is optional the first callback is called exception which i think of as you know the constructor this is how you take some term and you output the exception data structure then you have the message where it takes the exception data structure and spits out a string it's kind of like the two-string version and then there's the optional blame which we'll get to later um since it's optional it's not a kind of required understanding uh and then i put a little comment at the bottom here there's another 1500 lines in this file now once upon a time i would have you know wagged my finger and maybe judge somebody for having that long of a source code file uh but with elixir i've learned to to come to peace with that um it's not object oriented we don't have instance variables that are coupling various methods to one another it's just a bunch of independent functions sharing a namespace and so we can reason and understand about each piece of it in isolation so the size of the files i've i've learned to be okay with it just becomes an issue of kind of navigating your editor which is kind of an orthogonal uh concern so uh moving on some more source code this time from the kernel module the death exception macro it's not really that complicated and again i've simplified it but what this does is it sets the module attribute behavior to this exception so we are implementing the exception behavior it's calling the deaf struct macro behind the scenes so every exception is destruct and it adds that required underscore exception field to it so in a certain sense an exception is that struck with this extra true field on it and then the death exception macro will implement the message callback for you if you have a field called message it kind of gives that back to you if you don't it's up to you to do it and it's overrideable if it does imp if it doesn't implement it and then also the exception itself another interesting little tangent i know there's like just a few things this the core team has uh decided that has to change or requires elixir 2.0 uh to implement because it's a backwards incompatible change for semantic versioning reasons they've identified a few of them and i think they're waiting to bundle up enough of them before they really go through that hassle and but this is one of them the default exception function deals with some backwards compatibility issues but really moving forward it says in the comments there that this kernel.struct bang is really what this should do in its default implementation and so if you're writing code today you should make sure that your exceptions are not exceptional in needing that backwards compatibility but it will work with this moving forward and of course the uh deaf overrideable is is set on that so you can change it yourself in any of your implementations but you'll notice that this macro in ninety percent of the cases will get you to both both required callbacks implemented so that's really cool so now hopefully everyone's really clear on what acceptance are all about right well except i don't know i i think i know what some of you are are thinking right now there's a there's a question in your mind greg you said their behavior and then you said they're struck you know make up your mind well the way i look at it is if i if i look at exceptions and squint a little and and just tilt my head they remind me a little bit of objects okay wait like right rush don't rush for the doors you know hear me out um objects have kind of gotten a bad rap um really conceptually at the core they're the idea of of of data and behavior kind of being bundled up together and kind of packaged together and used together and even though some of us have had some bad experiences with that in the past let's don't forget that it's a it's a valuable abstraction and tool that we should we should keep around for potential use i don't recommend that you like i don't know orient your entire thinking about your software system around objects but keep it and keep it in your programming tool belt because it's a valuable thing and i think exceptions here are a great use of that concept so i'll step off my soapbox now let's go into a little bit of iex exploration at a prior talk i gave i got some really interesting feedback that that surprised me it wasn't about the topic of the talk but about the way that i explored in iex it was something some people hadn't seen that comes naturally to me so i want to do more of that and encourage people to explore uh i like to think that iex stands for i explore you know it makes it all about me you guys have to figure out your own you know acronym of what it stands for um and the idea of understanding one layer of abstraction below the level you're working helps you gain intuition about your system um not because you can then couple to implementation details you know actually knowing them kind of helps you avoid that it lets you know this data is available here so this thing should be possible and then you can look through the public apis and figure out how to make it happen or if things go sideways and things aren't happening right you have ideas of how to introspect and how to understand better so that's something i encourage everyone to do i have an iex window open all the time so let's start out oh yeah the other thing i love about iex mixed formatter is not all up in my face so i can play fast and loose with syntax this is a production code but this is you know it saves you save you some keystrokes in iex you can define a module in one line it's valid elixir if anyone wants to follow along they're welcome to but this is just defining a module that just simply calls the def exception real simple but now i have this and i can use some introspection functions the compiler adds module info for example and you pass in attributes as the parameter it shows you that these are kind of the persisted module attributes and we see that there is a version which is kind of a kind of a hash of uh that's used by the erlang code server for hot code loading but that's a whole separate topic but we see the the behavior listed there too that the death exception macro did add that behavior to our module as we saw we can verify that we can also just go ahead and create one of these since we know their their structs we can use our regular struct construction constructor syntax but that looks like a plain old struct uh it's also important in iex to know that you can pass parameters to the to inspect and turn off that inspects protocol so now we can see that this is really a raw map and down the erlang layer and the beam code really all structs and therefore exceptions are really just maps the fact that they have some special keys that mean special things to the elixir layer is kind of not important down at the lower layers of the stack and we see again that the exception field and the struct field are both added in so let's move on a little bit more with uh iex it's a lot of code the bold faced pieces are the ones that are really important it's different i have three anonymous functions that raise exceptions in different ways and it catches it and inspects it in the first one i'm just raising the raw struct second one it's a it's a two parameter call to the raise macro and the third one i'm calling the callback function on the macro on the module so just digging into that a little bit more raise is a macro defined in kernel again if you were curious um in hex docs there's a little kind of greater than less than with the slash to the right side of all function and macro definitions you click those you go straight to the source code it's a really great navigation tool that if anyone doesn't know that so raise if it receives an exception that's a basic pass through it's got what it needs it verifies it's an exception because of that double underscore field in the struct raise airity 2 takes the module name and the keyword list of field values and that's going to call that exception function defined in the callback uh module now the third time i called that explicitly uh which was unnecessary because that second that two two area version does the same thing now there's two other sort of convenience cases that the raise macro supports is you can pass just an atom to it and it expects that to be a module that implements the exception behavior and it calls the exception function with an empty keyword list so you get just all the default values that you might define in there it has another one that takes a string and it uses that in the built-in runtime error exception and just sets that as the message so those are two kind of convenient convenience clauses in the raised macro but all of them end up at a lower layer layer down at the erlang.error and this is kind of you know where it all comes from down at this layer where we're dealing with tuples and atoms and the docs for that they take any term as the reason for the error and in this case we're giving them a map and as far as they're concerned it's a map even though in elixir we consider it an exception struct and it's only at this point when erlang error is called that the stack trace gets generated so you can generate these exception structs all you want to and you're paying no penalty for creating stack traces or anything until it gets raised or at least goes through the erlang error so let's look back into iex a little bit more i mentioned the blame optional callback earlier and now we can we can revisit that that takes two parameters and it returns a two-element tuple of those two parameters so it's allowed to potentially update uh one or both of those now this is a really stupid implementation that just to fit on a slide in iex really doesn't do much but you can do computationally intensive things to help the developer understand what happened to in their in their situation but what's going to happen here is when you raise the exception which here i was lazy i just used the module name so i used that where it passed the empty empty keyword list and so now we see that this prints out the message so it went through the blame step and this is used by iex and xunit in particular for when developers are interacting with the code this is not production code so you can do all sorts of computationally intensive things and not pay any production performance penalties but you can make raving fans of the people who use your library because hey you really helped me out with these great error messages um i want to show one one example from lecture standard library this is in the key error exception and this is actually in that same exception file that 1500 lines that i left out earlier they define all these exceptions in there and i want to highlight again the the bold face parts parts there not not all the code on the slide since key error implements blame optional callback iex and x unit checks to see is this available okay let's call it before we do our final printout and we see that under the right conditions it will call this did you mean helper function and that's going to call the string.jarrowdistance to try to find typographic errors that you looked for this field name in some map or keyword list or struct and can i say did you mean one of these things that there's typographically similar to what you what you typed um it's just purely nice to have it it doesn't make your code better in production but it makes the developer experience really great and i really applaud uh elixir core team for implementing this and it makes looks for a joy to work with but you can make take advantage of this in your own libraries and any sort of error handling that you do with your custom exceptions so let's jump down a little bit into the lower level of erlang errors they have what they call three classes of errors and the first two the throws and exits came first and then the the error class of error yeah it's funny naming but you can just deal with uh with history here uh came later uh throws is very i don't know primeval uh i guess i i might say it's very much like a a go-to or a long jump you're you are you are throwing just a plain atom and you are expecting some other code to catch that it's mostly used in performance critical situations where you're like in a deeply nested loop and you suddenly need to kind of jump out just you know ignore the stack and jump out this other code because hey we found we're looking for and don't need to process the rest of this you're probably not going to use it very often in your day-to-day coding but know that it's there the exits these are kind of hey the process is is dying you know it's it can't handle what's going on and this is what it's referred to in the erlang literature as their their dying breath and this is how links and monitors and the inter-process communication uh operates they they're they're dealing with exits and exit is a is a tuple with exit atom as the first element an error is a little bit more kind of ambiguous because it's like something bad happened and maybe someone can catch it but this is where stack traces actually get generated is when it's in the error class of erlang error and this is represented in erlang as a two element or not two not two element it's multiple elements but it's a tuple with the error atom as the first entry and you'll rarely probably need to dig down this deep but i'm just kind of trying to to to to go over things um in erlang they use the keywords try catch and after are are what they support and catch can pattern match on any of those tuples or bear atoms that i mentioned you can still use catch in elixir code but you're dealing with the raw erlang error data structure in that case sometimes that might be what you need to do but i don't think that would be very often elixir further adds rescue and else and rescue is where elixir puts in their wrappers around the erlang errors it allows you to pattern match on struct names and it will wrap corresponding elixir errors in corresponding erlang errors in the elixir exceptions so for example if the erlang code raises an argument exception or elixir has an argument error and it can wrap that and present it to you in a more familiar form else i include here for completeness it is a real sort of corner case when you're doing like tail call optimization optimization it's there and it pattern matches the result of the tri body when there is no exception raised i really recommend if you ever need to use it look it up at the time because i always have to have to re-familiarize myself with things like that so i want to talk a little bit more about different types of codes you may be writing and the different ways you should be approaching error conditions a lot of people use these some of these terms interchangeably i'm going to try to define what i mean by them here and the way that i look at the errors differently libraries are passive code they're sitting there waiting to be called they're not starting processes they're not active and they should not make assumptions about what is truly exceptional they really should be dealing with error tubals at least primarily they can offer bang uh bang variants for convenience but for the most part uh things that you may think in your library that oh a config file is missing nothing can happen well maybe some embedded logic knows that on the first boot up it's not going to be there it prompts you for it creates it and then restarts the supervision tree so you shouldn't assume anything about what's exceptional to the application level when you're writing these passive libraries applications i almost just kind of defined by in that last step they're the caller of those libraries they're the ones that are that are managing the interaction with with the end user of the application and they're the ones that decide hey you know this is put an error in front of the user a crash uh or try to work around it that's where that sort of logic should be handled and then there's the frameworks which are these weird sort of corner cases um i like to think of them as distinct from libraries by this hollywood principle i heard the chuckle don't call us we'll call you like a hollywood agent so the framework is good is is going we're managing the the interaction with the user but we give you hook points for your logic a great example everyone's familiar with is phoenix phoenix is a framework phoenix with help of cowboy and ranch are managing those processes that are the request response cycle and you plug in by writing your endpoint in your router and your controllers and you're telling phoenix under these conditions you call this business logic of mine so in this case it's more of a cooperative idea of what's truly exceptional phoenix has its own opinions about what is an error and what's not it tries to convert certain things to um to status codes and present those uh nicely to the end user but you can also plug in with your uh with your fallback handler and customize that yourself so you're kind of playing together as a team because of this frameworks are kind of the hardest code to really write well in my opinion so i would you know urge people think about libraries first and then maybe frameworks later uh talking more about writing libraries i want to mention uh what i consider you know kind of exceptional libraries wait a way to do them properly and i'll mention this first idea uh i first heard i first learned from um brooklyn zelenka's oh i've got your name right um her library early on in my elixir career and she's giving a keynote tomorrow afternoon so so much of the elixir ecosystem uses an error and an atom reason and i really like the idea of having an exception in there because it's an entire structure of data it's richer data that you can provide more information to the application so they can reason more about how to handle these different error situations when you do this it's really straightforward to write bang versions because you do a case match and you can just raise the exception there you're not paying for a stack trace penalty by creating the exception until you raise it so it just gives gives the caller of your code more more information to handle their errors more uh intelligently and when you're when you use the tuple uh the unbang functions i called them the callers can do these case or with else expressions and they can just catch error exception and they can treat any exception the same if they want to or they could pattern match on the different struct types of the exceptions or you could include a reason field in your exception and they can use that to further subdivide how they want to handle different situations so now i'm down to the point this is the whole reason i kind of got me into giving this talk and at my prior job um i used this as a refactoring technique and i want to share this and i'd love to hear some feedback if other people have used it or if in the future you do use it and like it but to give you a little background at the very kind of simplified level we were doing order processing as a service so our customers were hiring us to handle their orders and they had third-party inventory systems that we were integrating with on their behalf and there were multiple of those supported and there were third party public facing marketplaces that were also supported and they would give us api tokens on their behalf and we would handle these orders and there were lots of you know distributed systems you know issues timeouts could happen each one of these had their own little idiosyncrasies uh and so we had multiple steps involved in our in our order processing uh at the high level it was a big with expression that had like i don't know six or seven steps and then maybe a dozen sort of else clauses for all the things that could go wrong but then if you dig in some of those main steps in another function maybe in another module in some cases what's another four or five uh step with expression with its own half dozen error uh error claw or else clauses and it was grown organically and it was getting kind of to the point where there were corner cases and we were having trouble diagnosing what went wrong in some cases and when money is changing hands our customers are going what went wrong and when are you going to fix it so it doesn't happen again so we really couldn't just let it crash and so we were trying to handle those errors uh pretty uh carefully so what i came to when i understood exceptions really are strucks they're rich data structures they're modules we can add our own logic to i created what i call the order fumble now i don't expect everybody to be a sports fan but the idea of a fumble is you lose possession of the ball and the other team might catch it and so sad for you or you might be able to recover it and maybe even advance your position so that's why i called it a fumble instead of an exception even though it is an exception i used a couple of things knowing that it's going to use death struct i could use the enforced keys which is something that the defstruct will use so i can make sure that we're always creating it properly in our code i found the the exception callback name a little bit kind of long and i like i like calling it new it seems like a constructor and i did a little pattern match here so that what we have in our context field is always this particular type of struct um which was our was our our context of the order it included like customer ids it included which of these third-party services they were using like the raw json responses from those and even our database record so we had a lot of context of what was going on when we were processing an order when something went wrong and we needed this order context to consolidate our logic here i also mentioned we have a step in there so we actually named the step of those high level ones i mentioned because we might want to react to the same error in different steps different ways and since i didn't have a message field in there i had to implement that myself pretty simple it shows us our reason and our step and so this is like the rest of it this is kind of really kind of the big payoff here that i had this mitigate function is what i decided to call it and this uses these module attributes that give us different categories of how we want to handle uh problems before the process crashes we need to you know be able to store store state in the database the status of the order uh in some cases we have ones that are called manual so again very simplified example and there were a lot more of these in real life this just to fit on the slide and get the idea across if we're in the inventory step and we get mismatched counts right that's something that we as the programmers we can't fix something's wrong somewhere someone's trying to buy too much of you know more than what you said you had and we have to notify our customers and so we can go through a case statement in there and have a helper function handle a notification and that can look up hey this customer likes email or likes push notification or texting or whatever and can handle all of that um so we're contacting our our customer as quickly as possible notifying and other ones uh retriable if we got a timeout while we're talking to a payment processor you know let's wait a few seconds try again before we you know exhaust uh retries and then it becomes a manual notification and then finally if we don't know we didn't expect any of this step or error to happen we raise and so our exception handling service we were using app signal so that's going to notify us and be loud in our face and we're going we missed a case in our code and made it clear to us that that needs to be fixed so this was the core idea that that the module is a target for refactoring we were able to handle and consolidate how we're handling these error conditions into one place and it was able to simplify a lot of a lot of our logic and that was the core sort of you know design idea that i wanted to get across here and that is hopefully not too anti-anticlimactic so the main takeaways if you can handle the truth exceptions are objects if you can't they're behaviors and structs okay it's this quantum superposition okay um use blame in your own libraries and you'll have raving fans of you know the developers that use your libraries i really consider strongly i'd love to see our ecosystem returning error and exception a lot more than just the atom reason and you could have one exception for your whole library with different different reason fields inside of it or you could have separate exception structs which is whatever works best for your situation and an exception module they're modules they're they're a refactoring target they're a way to to put common behavior dealing with this sort of my order fumble something went wrong with the order and let's try to recover before we die so thanks for coming this is how you can contact me and we're hiring and happy to talk with anybody love meeting you folks and uh thanks for coming we have any questions do we have anyone who's brokering questions or this is up to me okay okay the question was exceptions and errors are they the same thing uh uh for the for the purposes of this talk errors are situations that you don't want to happen exceptions are complicated uh when you raise them they become something different uh they are blowing the stack trace they're ready to kill processes and and get supervision trees involved um so raising exception is one way of responding to an error but errors are also the terminology used down in the erlang layer and exceptions are the terminology used in the elixir layer so it's it's complicated and you can create the exception struct without raising it so there's ambiguity there and i'm sorry i didn't clarify that okay here so uh earlier in this uh several slides ago you had kind of a quick summation of like uh try rescue these sorts of things there was an after keyboard that's another sort of performance optimization thing if you are in a try block the um the lower lower levels in the beam have to store that stack frame they they can't just use tail call optimization because an exception may come and they need to remember that position in the code when you use it after it's kind of in the code in there you know you're safe to reuse the stack frames so it's a very kind of low-level technical thing that probably you'll rarely interact with but that's what that's for anyone we're here and then you went to the order fumble is that ortho like this if i call the initial with clause workflows that work humble basically over the entire that workflow and then you'd have a i don't know call a purchase only you'd have different so the question was was the order fumble shared for the whole workflow or were there other kind of sub fumbles for different steps in it you could certainly design it that way and i chose to use that step field in the order fumble so that they're not separate and it was it was more of a consolidating effort and that worked for our case well what um i showed a little example there uh here that the keyword list under manual like inventory was the step name or payment was the step name in some cases we could put things on hold so hold with a step name or fulfillment for kind of the various high level steps of our order processing flow well it was it was based upon the business logic he needs those steps to to handle an order and i don't we didn't spend a lot of time thinking about the naming of it it really just kind of came out we just you know had casually talked about well during the fulfillment part of our this order it failed or while we were checking inventory i'm sorry known terms to the team to the business domain yes over here okay so the question is looking for like background research computer science language agnostic error handling advice and i can tell you confidently that yes there have been people writing about that i don't have anything in particular i can recommend specifically but i know that has been an area of lots of lots of research in computer science um i've picked up pieces here and there but i don't have anything fresh that i could try to recommend on top of my head sorry over here i like the air tubal exception thing i was wondering if you have any packages or examples of that in use to kind of look at i'm trying to remember i i have not seen a lot of it there have been some blogs that uh mihao um i'm probably missing missing his name muscula has used that i've seen andrea leopardy also talking and add on that so there's there's uh quite some people thinking about it but as far as as a major package i don't have one off the top of my head [Music] question like things to watch because you're looking for advice for for handling the erlang errors and wrapping them yourself um the most common theme i i have is the elixir core team has done a great job i haven't seen an erlang error in quite some time because they've they've done this wrapping and and put this around it i don't spend a lot of time calling the erlang libraries directly there's something we should all keep in mind but i i don't have any particular advice on that area sorry now one more question let me let me let him go since you asked one already well considering open source is what anybody wants to write in their spare time nothing's universal but the idea of returning an okay and a value or an error and some reason is actually well um well used in the elixir community and you know it's not too far to go from a reason atom to a reason exception and you know that's kind of the adv advocacy that i'm speaking of here to give richer data to the callers and we are just about you know we're out of time and um i'm gonna i'm gonna call it with no proctor in the room so thanks for coming uh we should talk later with the other question [Applause]

Channel: ElixirConf

Views: 490

Keywords: elixir

Id: HZZCl1ClYb4

Length: 40min 55sec (2455 seconds)

Published: Sun Oct 24 2021