Intro to OTP in Elixir

all right so intro to OTP in elixir who has heard of elixir here alright good most of the people here who's actually used programmed in elixir whew we got one in the back alright so I'm gonna guess has anyone here heard of OTP okay we got a couple good so OTP is Earling was built in 1986 and then OTP was kind of like combined with it and it was released open source in 1998 and so Earling an OTP can I came out it's been around for like 20 30 years out on 98 so whatever that is and it's kind of kind of has a lot of parts to it so here you can see a list Erlang the programming language and the compiler that's that's part of OTP it's got a bunch of tools for like in-memory data storage there's a database server analysis tool we're not going to cover any of the tools today there's a bunch of libraries like there's the supervisor library there's generic server and a bunch of other libraries we're going to cover some of those today and then there's design principles and those are part of how you use a generic server and supervisors and then there's other stuff because there's just a ton of stuff - OTP and so there's some stuff like how to release applications and some stuff like that and OTP is huge so we're not going to cover that stuff today so the main thing in OTP is your processes everything in OTP runs and processes it's like your basic building block for OTP and it's not like an operating system process so when you spin up an OTP process it's just like super light like you can like on a laptop you can like spin up just like thousands of them because they're just like an internal process that's a super lightweight doesn't have to deal with all the stuff that like an operating system one does so let's go ahead and well I'm not gonna live code it but I'll show you what it's like to spin to spawn on real quick so here we are in IX which is if you use Ruby it's kind of like IRB it's the repple for a lick sir that lets you kind of just run some live code mmm so to spawn the server spawn the process we're going to run spawn and then we're going to pass in a function so our function is all it's doing is it puts you know it's going to print out hello full full sack so it's real simple so we run it and there we go we got our hello full stack and then we've got this weird-lookin hashtag paid 0.6 4.0 so what that is is the process ID it's like the mailing address for that process so that you can track it so what we really want to do is track that from the beginning so we'll run that again this time we'll assign it to the variable pid' so that we have that process ID and we can keep an eye on it and then we see we run process alive and it says false so what happened the process runs it prints out our hello full stack and then it dies so process just kills itself as soon as soon as it's done doing its task so if you're doing a you know one like printing hello full stack you know it's like pretty much instantly dead does its thing and then it's totally done so what if you're doing a longer process like a you're doing a longer function that's going to take a while and it fails partway through so that's where we run into error handling one of the common phrases that you'll hear when you look into elixir and Erlang and OTP is let it crash which sounds terrifying because letting it crash usually means well you have to kill the whole app that is that's doesn't make any sense but with OTP you're running all these like hundreds or thousands of little processes and all you're doing is killing just crashing that tiny little one so you don't have to restart your entire app it's just this one little piece that gets or gets to restart and the reason this works really well is when you think about like any sort of like if you think about a computer or your app and like something goes wrong and can't figure out what's going wrong the easiest way to fix it is to just restart it and it's the same with like everything like my kids electronic toy or your TV your video game console your phone you get to a point where it's just like I don't know how to fix this but if I restart it that'll fix it so this is taking that principle and applying it to our code so if something goes wrong and are in a process or some functions running and if and it fails somehow we just can restart it so the reason this works is because you're clearing out the state because what happened is while it was running some combination of stuff happened and the state got messed up and so that's why restarting works because you're just giving it the fresh State started over and give it clean clean slate or a clean state so how this works in OTP is with supervisors so there's a supervision tree that's kind of based on this principle so in this example these are all processes some of them is are marked as supervisors and the supervisors just have two simple jobs they start up other processes the ones underneath them and then after they start them up they track them they just keep an eye on it and if one of those worker processes crash and then it just restarts it and it and that's like this is a really simple tree to look at but this is freaking awesome like anything goes wrong in your app and you have supervisors watching everything and it can just kick up and restart that process like that it's it's pretty it's pretty rad to see like when you're when you're working an app so you just can run something kill it and just restarts like instantly it's it's pretty awesome and so just like that you have the supervision tree and now you never have to do any error handling again pretty awesome it's a lie you do have to do error handling give a question well sir so that one is watching so in this example say that the far-right worker crashes and the supervisor restarts it the supervisors right above it restarts it if it restarts it a couple of times and that doesn't fix the problem then it'll crash itself so it's keeping an eye and it's kind of restarting it's like this didn't fix it I'm going to kill myself and then the supervisor above it will restart it and that will cascade down so it can kind of cascade up if you have like a really bad problem that's kind of infected in there yeah well then you're in trouble that's the whole like you know who watches the Watchmen at some point you get to the top and like all the whole things just Tet with just toast and actually so there's a thing I was joking that you don't have to do air handling what you do and the reason like when I first heard about this I was like okay it sounds great but what if I did what if it's just broken like if I like what if my work over here is just bad like restarting it isn't going to fix that worker it's just going to restart and running again it's not going to work even if it goes all the way up that workers still broken so this doesn't fix that you still have to do that hopefully you catch that before it goes to production just like you normally have to do I mean that's the sort of thing you should catch in testing and there's a there's a great article called the Zen of Erlang by Fred Hebert and he kind of talks about this so there's he says there's basically four kinds of bugs that you have in your code there's the kind that are really easy to repeat so you can see how it's breaking and try to fix it and sometimes that's on a core feature and sometimes it's on a secondary feature and then there's the transient bugs those are the ones where you get the bug report and you're looking at it and you're like I can't I can't get it to do it how I don't know how this person made the app do this I'm trying to get it to happen again and it won't happen and those are the kind of bugs that are really hard to find before you push to production because like you don't have this whatever weird combo of stuff that messed up the state you can't like you usually can't do that and so it goes for production and because of that yeah but this slide sort of says that those kind of bugs happen all the time because they're really hard to catch the really great thing about those kind of bugs is restarting the process fixes it like the restarting is not going to fix the easily repeatable bugs but it's going to fix the ones that you can't catch because as soon as as soon as it crashes like that supervisors there and sees that it's crash and just restarts it and now it's got a brand new fresh state it's / it's pretty magical I've I love this the more the more like researching OTP the more like this just amazing so yeah it's awesome all right moving on so processes in elixir are there completely memory independent and they're completely isolated but obviously they have to be able to communicate to each other so they do that with messages so every process has the ability to send and receive messages and that the process ID we pass around earlier that's how we track the processes and that's basically the mailing address like I said earlier every process has that mailing address and they also have a mailbox where messages that are sent to it you just go into that mailbox and then the process has a way to handle those and we're going to go over that so let's see how this works so we're going to build a real quick basic fantasy football app I'm a huge famous football fan I'm a little bit bummed that I did not see that the first regular-season game starts like today so I'm missing it right now but that's okay I'm really excited to be here but we're going to build a quick app that just lets you build your fantasy team add players to it remove it and track track your team that way so and we're going to I over the summer I wrote like a quick elixir wrapper for fantasy football nerd API so we're going to use that kind of in the background to get player data as we do this alright so one thing with this so elixir is immutable you might have heard the data and elixir is immutable which makes state a little bit tricky to store because you can't just like mutate the state and normally you get to this point like okay so we got to save the state and you're kind of traditional object-oriented programmer might say alright time to make a database got to get it to the database somehow that's really kind of counter to how a lot of people especially in Erlang but elixir to but sorry the communities are kind of like coexist for their little bit different you know it totally sidenote um but they they they try to hold off going to a database until like lot like that's the last possible thing because the process like you can a process is running for years and we're going to show I'm going to show you right now how to store state in a process instead of it instead of a database okay so here here we got some code can you guys I'll see that okay go ahead because I can't actually change the size so I'm glad you can see it um so this is that basic process we ran like this is it ready to receive messages so we're going to I know it's it's a lot we're going to go through it line by line to kind of understand what's going on here so we start off and we have the function that's called start link so that starts up the process and start links just like a common name that's used and so that function does spawn link which is similar to the spawn function we use before to kick up a process and then it's got three arguments so the first one is this module all caps with double underscores on both sides all that does is it references the current current module that you're in so basically at compile time it's going to change that to fantasy team basic which is the module name at the top and so that's just like a reference back to this module on the second argument we have here is loop that's actually going to tell the process so it's going to tell the process to look back at this module that we gave it the name of and then run the loop function which is what we're going to go over next and then the third argument here is uh this is the list of arguments that's going to go to the loop function in this case there's only one argument loop just takes one argument and so we've got inside of that this is called a map which is kind of like a library you know like a key value store is what a map is in elixir so let's go look at the loops loop function so we have loop state here so States the that empty map that we before that we send to the process so now the process starts to run this function takes that in that empty map and then it starts this receive block so receive do sets it up to receive a message and now it just sits here so remember before we have the process it printed out hello full sack and then it died so this this process isn't going to die because it's sitting there waiting for a message so we have defined here the first message that it can receive which is a tuple which if you don't know that it's basically like a small list so it's got two items in it the first one is an atom which is like a symbol if you use Ruby up anyway so that is just identifying what kind of message this is so the message the message here is we want to add a player so the first thing is ads so that we tell the process what which function we are yeah which function we want to do here and then the name is going to be like Russell Wilson in quotes so then it takes that name it uses our like external thing that goes to the API and gets the data for that person and then it applies it or yeah applies it to the player variable so now now we got player which is the all the information like the you know team Seahawks position quarterback all that sort of stuff is now signed up player and then we go to the next line here and this is defining the new state so we take we take that name Russell Wilson we take the player info which is the all the stats quarterback team all that and then we're applying it to the current state which is an empty map so basically once we run this line we're going to have a new map that has one key in it and it's Russell Wilson and the value for that is all of his stats so if we just stopped here though that's where the process dies because it's processed a message and now it's done doing its thing so instead we call the loop function which goes back to the top and starts the whole thing over again so the loop here takes that new state so it takes that map that's got Russell Wilson in it with this with the excuse me with his stats in it and then it carries that back up to the top so then you rerun loop function the state is now that map with with Russell Wilson and now you waiting to receive another message so next we have the remove message and that's pretty much the same thing you don't have the line to get the stats because that doesn't matter but you're taking that state and now you're removing that name from the state and then you're looping it again with the adjusted map and then the last function we have or the last message sorry that we have is the team so this is we're asking for like we want the team who's on the team right now and so that actually the first one you see is the team atom and then the second argument is the pid' so we're actually when you call this message you have to give it your pid' so that you can get it back so like if we were in IE x ie X is a process so if I ax sent out the message team it would also send out its own pid' so that the so this process would know where to send the team info to so it does that with this send it sends to the pin and it sends the state to that pit and then loops again don't note the state doesn't change it all so it's just loop state so let's see this in action real quick okay so we're in ie X and we're going to go ahead and do that first start link and we're going to assign it to a pig so now we have so we spun up the process and we have the ID for that process now we're going to send it a message we're going to send to the paid add Russell Wilson we get the return that just tells us what was sent and then we're going to go ahead and add another player we're going to add Doug Baldwin so now we've got two players on our team and then just a tester will remove Doug Baldwin and then we'll call the team and see when we send it to send the message calling for team we don't get the team back because what happened is we got a message sent back and IX isn't it IX we haven't built a received lock for it so right now that message is just sitting in the mailbox but with with IX we can just flush the mailbox so we flush it and then we can see ok there's that's the team so we get the information Russell Wilson position quarterback team Seattle so this is the process that did that is there any questions at this I mean does this like in general make sense or is there any use specific questions about this um I believe so I believe so yeah anything else okay I'm going to move on assuming you understand everything so all right so now that we've done this basic process this is a super super common pattern in OTP like this because this is how you basically build a process that can send and receive messages it's so common that they created the generic library to cover it which they call gen server and like this is great right here but it's actually missing a lot of tiny little edge cases stuff like tail call optimization avoiding deadlocks message ordering a bunch of other things like that that they've kind of discovered over the past 20 years of using Erlang and OTP and so because that they've fixed all those and then they hide it away in this thing called gen server so we're going to we're going to take our process and we're going to convert it to gen server now all right so um so here on the right we have the process that we just went through the code for that process and then on the left this is what it looks like in gen sir you see it's some great idea I can't even see it at all there's a little bit of extra code right there we grayed out because that's more like a convenience function so you can sort of see that it's about the same amount of code outside of the convenience functions and we'll cover those too so we'll go down the line first you start off and you put a use gen server that's what gives you kind of all the magic for this module and then now we'll compare the stuff that's all basically kind of the same so first we have start link and on the left now instead of doing the spawn link to spawn the process we do gen server dot start link and we pass in the module and then we don't have to pass in loop because genzler is using kind of its own loop function that it doesn't need us to provide one so it's kind of doing its own stuff behind the scenes so all so we're passing module a second things okay you don't have to worry about that one that doesn't really matter and then the third thing is arguments and we don't have any arguments in our situation here so after that we have callbacks oh you have a question are you talking on the right column oh yes so if you do not match it which like we just sent like I don't know what some random word whatever it would just sit in the mailbox and usually what you can do I don't have it on here usually you can add a like a final thing that's just like an underscore and that's like a catch-all you can just like throw up an error message because if you don't do that you could run into problems where it's like you're the mailbox for process gets flooded and there's I don't know what is at some point that could like be a real big issue yes uh actually I'm not sure you might have to do that manually I'm not sure so we're just gonna ignore that for now all right yeah mm-hmm so actually when when I say like restart the process it actually killed like the prot the process itself dies and you like code in a way that when something goes wrong it kills itself so you're actually starting a brand new process that just gets the same has the same starting state so it's not actually restarting that brought so those messages would just be gone yes yeah there you can work around that like if you needed something to store that you could like duplicate those messages elsewhere but the general vanilla case we have here is they would just they would be gone because you're starting a brand new process it just happens to be in the same place with the same state right um possibly I don't know the answer so yeah yeah so I don't know the answer to be able to help with that okay so we're going to go back so the first thing that happens when you the first line the start line can you get the Jen server and that kicks off the process the first thing that it's going to do is go back to the module that you gave it and say okay where's your an it function and that's we have right here and so that matches the okay that we provided in the first one and then Jen server is expecting this tuple that starts with okay and then the second argument there is the state which again is our empty map so we want to start the process with an empty map so now we're going to handle our first message so in this case we don't have a giant receive loop each message is handled and like its own function so in there's there's two kinds of messages actually in Jen server there's casts and then there's calls so a cast is like our add remove or we're sending something to it but we don't need a reply and then a call is you send something like for Team you want to hear back like what's the team that's the point you're sending a message in expecting a reply with a call so here we're doing a cast so you handle casts with a handle cast function and then you can compare over to the right so the first the first argument in handle cast is the message we're expecting to receive just how we listed over there on the right and then the second one is the state so previously we had that loop function that kind of held all the message messages and that kind of is where the state came from in this case the state is in the handle cast function and then these two lines there are the exactly the same as before and then it's instead of calling loops since there is no loop you just return a tuple that starts with no reply because it's a cast it doesn't need to reply to anybody and then the new adjusted state and this is something that Jen server is expecting this returned and so it takes that state and saves it same thing as before the remove is basically like add just without the the line that gets the player data so that just adjust the state and does the no reply tuple and then we have handle Hannah calls slightly different it's got three arguments instead of two the first one is the message which in this case is just team the second argument is the pit of the sending process in this case we don't need it so we just put an underscore before it so it'll just ignore that but we put from there so that just so we know what it is and then the third one is the state and handle call is expecting a reply so you get a tuple with reply at the beginning and then you have state twice and the reason you have state twice is the first one is what is going to return to the calling process and the second one is going is what's going to save as the state so like for example if if this instead was a candle call quarterbacks and you have a whole team of bunch of different players the first state there would be just the quarterbacks and the second one would be the entire team because you want to save the entire team as your state so yeah so that's how that part works okay so how do we actually make these calls it's a little bit different with Jen sir which is why we have those convenience functions that I show before so at the beginning they have the start length that we already went over and then we're just defining a really simple function add with the pit number of the process and the name and that's going to do yep it process in the name and so that's going to do Jen server cast and then it's going to send that to the right paid with that message so all you would have to do is call the module name fantasy team my Jen server dot add the process paid number and the Russell Wilson and then it's going to do go through that whole message thing you don't even out you shouldn't be worrying about the messages that's kind of happening behind the scenes and then team is basically the same but it doesn't have a name just the pid' and then it does Jen server call and that's basically the same so now we have this is our Jen server version which is a little bit longer to look at than the basic one but it's better because you don't have to actually manually send messages to use it you just have to call the function add remove or team cool so we can do one thing a little bit better here in this case I'm not using this to like let a whole button I don't want a whole bunch of teams I just want one so I can make this a single server or like a singleton kind of well it's a little bit different and so on the right is the code that we just had the gen server code and then the left is just very slight adjustment we're going to add this name at the top so we're naming the process or we're just creating a module attribute that we can use later which has the name of the module fantasy team single server and so we're going to place that in here in the arguments so then now when you spin up the process with your start link it knows that it has a name so you don't need the process ID because it has a name to call directly and then because of that we can take the pit out of these Jen server casts because it doesn't need to call the pit anymore it can call the name directly and that's great because now people don't have to have the pit as an argument which is great because it can be that in this case if you're really dumb to have to remember the pit every time when you're just trying to add a player to your one server so that's great so Brent makes add only a single argument just the name remove just as the name and then team doesn't even have any arguments anymore so yeah so let's see that real quick and IX so we're going to call us up we'll do fantasy team single server start link we don't need to capture the pit this time because it just created that process and we already know the name of it because it's the same name as the as the module we're going to add Russell Wilson so we just call fantasy team that single server dot add Russell Wilson we get an OK response just a firm that it went through then we do the same at Doug Baldwin now we're going to remove Doug ball down we get a bunch of okay's and then we're going to call the team and this time we don't have to flush the mailbox because it's you it's giving us the team as the return rather than having to send a message back so it makes it really easy because NC then you can just do fantasy teams single server team equals team and now you have team or team equals that so now you have it in a variable so it makes it a lot nicer than having to send a message and receive that okay so quick recap so we built we built a gen server and this allows us to keep the state and functionality in a process which is great because now we have the benefits of the supervision tree and it's like fault-tolerant for the transient bugs that we were talking about and it's just because the processes man is their own state it's a lot easier to reason about and it's got a bunch of other really cool OTP things that I just don't have time to tell you about but it's yeah it's awesome real quick a little bit over time so on a wrap up real quick elixir has a couple of extra things called agents and tasks so our gen server Argent's R is just basically holding state you can add and remove from it and then you just holding the state we're not really doing anything any like complex like operations or anything so task but we could because we could just add another message that did a thing like that tasks and agents are kind of like splitting of a gen server so gen server can store state and it can do all the complex operations whatever tasks is like a great way to do parallel or like concurrent tasks you want to send it just a whole bunch of work and it's going to automatically spin up processes and use all the different cores on your machine and everything and it's like like two lines it takes like no work to do that and then agents is the opposite it doesn't do any of that stuff but it's it's really simple and basic storing state so I'm not going to go over it but this is the same as as the gen server that we wrote just in an agent so agent just has like a start link get update get an update it's like a really kind of basic version of gen server so if you know that a process is just going to be storing state like currently we do then this would be in much easier way to do that mmm okay that's all I got um there's a lot of awesome resources to learn more right elixir that the community is really great they've got an elixir slack group and I highly recommend going there everyone there is like incredibly friendly and super helpful and there's a there's a really great book that I've been reading called the little elixir and OTP guidebook it's by Benjamin Tanduay Howe I think it's pronounced um and it's fantastic highly that and I have a link bitly full-stack OTP with some other links to videos and other learning sites and books that if you want to learn any more about this that's a great place to do that

Channel: Full Stack Talks

Views: 25,597

Rating: 4.9771862 out of 5

Keywords: iMovie

Id: CJT8wPnmjTM

Length: 31min 1sec (1861 seconds)

Published: Fri Sep 09 2016

Reddit Comments

[u/smurfkiller013]

RemindMe! 3 hours