How Elixir Made Us Better Programmers - Ari Lerner and Anna Neyzberg

me well hey sorry all rights books on code that's ginger or he's dog she doesn't write books or code but she's cute so so we're here to talk a little bit today about how elixir helps us write better software so we're gonna talk about a we were talking about an app that we developed called elixir bot elixir bought is a semi real-time system where we pull data from Twitter and we're gonna we're going to talk specifically about what actually went into elixir bot and you'll see why when we couldn't do it okay pretty quickly took us about a day and a half to get to the real-time system we're using Twitter library sockets and a little bit of react but no framework really just a lick sir and how did we accomplish that what did we do and why is this beneficial so elixir really gives us a lot of there's a lot of power that we get from the language of elixir a lot of that comes from Erlang a lot of that comes from specifically from elixir so one of those features that we embrace is the umbrella app design so when we're doing when we're building any application regardless of the language the way that the way that in my opinion that good apps are developed are done through design driven development that is you you take your what you are building and you break it up into small modules and you follow that design as you're building elixir the ability to build an umbrella app which is a single monolithic app broken up into little parts kind of forces that you in order in order for you to actually build a quality application it for it you're forced to look at the divisions of the functionality of your application so the umbrella app gives us the ability to break up our functionality in the little parts it also allows the management of dependencies to be independent of each other which so you might have one part of your application that uses a specific dependency or a specific version of a dependency and another uses a completely different version of that dependency breaking up into into an umbrella app allows you to work with both of those pretty easily the dependency resolution in dependency conflicts are minimized and also Wow this is a run-on sentence and also and and I'm gonna finish that sentence and go to the next one okay umbrella apps also make test-driven development and behavior driven development pretty easy so I like to say that you get the you get some of the benefits of building a monolithic app while still getting some of the benefits of building on micro-services in the same app decided how are we gonna do this in a way we could harness the power of elixir so we decided that we're gonna do we need something that was gonna pull from Twitter we needed a web interface to show the tweets that we pulled from Twitter and somewhere to store them its database so let's get started so we use the mix build tool to create our app it's really a powerful tool that makes it really easy if you look at our current app it basically passed a new command to mix with the name of our app and the umbrella flag you can see our three child sub apps the DB app a Twitter app in the web app so let's take a look at how we set up our DB f it's gonna save tweets from the database and then update the vote count or when people have favored at a tweet you'll update that so we can see how many times it's been favorited so to start we need to grab our dependencies we're using Postgres and ecto ecto is kind of like Oh RM similar to active record for those of you that are familiar so we pulled those into our app we run mixed ups get to retrieve those and mix as I mentioned it makes it really really easy to manage dependencies and that's important right it means conflict is a lot less likely and when there is conflict it makes it a lot easier to resolve and smarter tools allow us build smarter apps hashtag smarter tools Twitter themed so what else comes with smarter apps so we talked about we're talking about smarter applications we're all we're one of the things that we're talking specifically about is applications that come with both that come with stability so one of the ways that we appreciate stability inside of an application regardless of the language that we're talking about is doing test driven or at least test behind to so as you're writing your functionality you're writing your tests that go along with that so you can have the stability and confidence that your application is going to work the way that you expected to just ask Dave he's here you could ask him you could ask him so in elixir one of the really awesome things about elixir is the is testing is it core to the actual language itself so unlike other unlike some other languages you where you have to pull in your testing frameworks when you use a lick sir you get X unit for free because it's built-in so let's take a look at what that might look like or let's actually take a look at what that did look like as we were building our application one of the features of our application that we want to ensure works as we expect is that the vote count is set accurately based on what we get back from Twitter so we can ensure that that is the behavior that we expect because our through writing a test so let's write the test we'll write the test first this not only when we're writing a unit tests are we writing the the test to ensure that that functionality works but we're also writing the the interface that we want our application to to adhere to so for that test we're writing a function that does first sorry that's okay I'm actually done with that one now so we'll write that we'll write that test we're we'll run that test although we don't need to because we know it's going to fail because we haven't written the functionality yet so then having knowing that our interface and the expected output of that test matches are our expectations then we can go ahead and write that functionality and get that test to turn from red to green so we implement it then we run the test again because we want to see if we implemented it correctly and it looks like we did and this gives us the confidence to know that our code is working do I do I need this okay great so before we move on to the Twitter app I want to talk about look it's a pattern matching which is a term you hear a lot in elixir and it's powerful but why take a look at an example you saw that we were saving tweets to the database and if you look here we have two versions of this one function with the same name but if you look closely you'll notice that they each have different arguments being passed in right so we have two cases here either we have a tweet that's already been created and we just need to return it or we haven't and we do need to create it and save it to the database so the function that gets executed is the one whose arguments match the data that's being passed in or the data matches the arguments that are set in the function so why is this important makes it really easy to understand what's going on you look at each function it's really clear what you're declaring again as I said our codes declarative right you're being really intentional and you're focusing on what data you're inputting into the function it prevents complicated conditionals right we're able to decompose our functions you're not looking at long strains of logic to figure out what's happening aware and it allows you to deal with again as I said data in really small pieces we can write really small and focused functions which makes our programs overall a lot more clear so let's take a look at our Twitter app we have a high level diagram we have Twitter we're streaming from Twitter we're saving to database and then we're sending it to our web UI so let's implement the Twitter streamer and again somebody mentioned this morning there was a talk about Twitter and rate-limiting it's a thing so we're gonna try not to get banned so we're gonna pull from Twitter elixir somebody mentioned again in the talk this morning it was really challenging to do this in Erlang somebody wrote a pretty great library called X Twitter that's easy to use at parity whoever you are if you're here love to meet to you so thank you so it's really straightforward we pass our parameters in in this case hashtag it looks about then we're going to you can also pass in options how many tweets you want to pull at one time we're going to save them and then we're going to send those tweets off to our web UI so we could do this we could do this using one function write our our semi complex application could be could work in the flow function and we could pull our tweets and do our database manipulation and then throw those up to the web well there is a there are some problems with that one that's really complex to test it's really hard to isolate those features it's it also mixes a bunch of concerns like when you're dealing with the database risks when you're dealing with Twitter and what happens if your database fails how do you know that it's your database that's failing and not Twitter because you have everything written in that single function so Anna is there a better way to do this some of you may have heard recently about Jen stage which a behavior it's a behavior that allows us to do a pipeline of work essentially and allow independent steps to handle just to happen and independent processes so again let's look at our design it seems like it would be great for this to happen in our app right we have a process where we're doing something with Twitter we have a process reduce them in the database I mean process so we're doing something with the web you know what what happened if we flip the direction of the flow and why would we want to do that so what happens when there's more coming into your system than your sister can handle right we want to rate limit Twitter so eh we don't get banned but B we're not constantly consuming and maybe our consumer can't handle everything that we're putting into our system anybody like I Love Lucy just me okay so Jen stage actually allows you to go backwards what does that mean well they're just concept of back pressure which allows us to have control I'll explain that in a second again looking at our diagram we're going to have three processes Twitter our database and our web process and how is this backwards system set up so we have I said we have three processes and Jen stage has three roles that you can assign to a process a producer which is acts as a source a producer consumer which knocks the source or a sink and a consumer connected to the sink and it'll receive processes at the very end we'll get the very end of your pipeline so to speak so if we look at our diagram the way we set it up is we do have twitter pulling we have a database and we have the consumer but the interesting thing about jet stage is it actually allows the consumer the things sitting at the end to tell the producer I only want you to pull in this much I only want you pulling this many tweets and since we want to controller desires okay manage demand pressure we have a callback once or started that we can tell our Twitter app hey only do this much this often and then Twitter will pull some tweets will save them impress them in database and then we'll send them off to our UI our DB producer consumer sits on top of the web app or in the middle so it'll receive tweets it'll save them and it'll send them off again and it comes with a really nice callback once it's done process once it receives it it's there's its thing and then it sends them off this handle events callback at the end we'll send them off to the process that subscribe to it which is our web process now back to the tweets so we get to the top and we have this handle demand callback from our producer and you notice the first arguments demand that's actually going to come from the process at the very end that's saying oh no I only want you to pull in this much stuff but there's one piece missing okay so in elixir BOTS we it's it's kind of only useful if we have a web front-end at least this version of the app is so in order to do that we're going to use the fantastic cowboy library which is written in Erlang you may or may not have used cowboy if you have used cowboy it's awesome in either elixir or early we'll go ahead and we'll pull that dependency again thank you mix for making it really easy and awesome then we'll set it up in this in our implementation we have two endpoints we have the root end point which we'll talk about and we have a WebSocket end point which we'll also talk about that's superfluous all right so let's talk real quick about the router the router is the part of the application that serves all the index dot HTML and the JavaScript this is actually the entire implementation of the router one thing that's really cool about about this is because we're building a single page app where you don't have any other we don't have any extra routes that we have to support so in a single page app we're taking our where any request that comes in that does not match a request or a file that's on disk we're sending the index.html everything else we're just sending whatever file matches on disk thank you to Erlang and he looks for for making the eex library come with elixir because edx if you're unfamiliar is a templating is a templating library that allows us to write templates that's silly it's a templating library one of the cool features about that is the fact that we can in a different environment in elixir we can send back different pieces we can send back different pieces of HTML so if you look at this code we're under development circumstances we're sending back a web pack to served application which allows us to do hot reloading inside of our app as we're developing because we're doing hot reloading that makes development really easy and really fast under production circumstances we can we'll send back to the built version of the app and notice we don't have to change anything there's no extra there's no extra web pack files which if you've ever used web pack that's kind of annoying and there's no extra index HTML files so we're using reacts and web pack - this is how hot reloading is set up we're also using Redux because all the cool kids are using Redux raise your hand if you know what Redux is so Redux is a data library that I'll that makes handling data on the front end really easy and really quick ok so moving on to the other interface of our application we are we have WebSockets built into our library on the client will connect to our web sockets at /ws stands for web sockets and when we're connected then we'll listen for we'll listen for any data packets that come from our web server and this also allows us to send it's bi-directional so what are we connecting to this is the cowboy WebSocket behavior handler that we're using the cowboy a WebSocket behavior handler is really nice it allows us to treat our web sockets as though they are elixir or Erlang processes we can kind of treat it just like it's gen server so we'll implement the callbacks and we'll get those callbacks when we get something from the applications front end so sending data from the client to the server is really easy it's we're just using socket the native socket send this is a native HTTP web socket and we'll on that web talking handle side we'll listen for messages that come in presumably you'd want to use something like pattern matching here but for brevity and simplicity you could listen to your you could listen to your messages just like this however broadcasting to our connected clients is a little bit more tricky in order to do that we need to keep track of all the connected clients that are connected to our application I just said it was tricky but I kind of lied because it's actually really easy to do using elixir and the elixir registry the elixir registry if you're familiar with G proc it's like the elixir version of G proc if you're not familiar with G pockets it's basically kind of a shared dictionary this is really high-level it's a shared dictionary that allows us to register keys in a state multiple keys in the state so we can register our different processes under four different keys that we want to look up so for instance we have a registry here that is named WS registry for WebSocket registry when we connect when our WebSocket client connects we'll subscribe to that registry with that process ID and then all we need to do is send all we need to do is send along our broadcasts to every client that is connected under a specific name in the registry and then we handle everything using the normal redux web flow so for instance we are listening for received tweets that's the only message that are our only message that our application is dealing with our new tweets will receive those tweets and then we'll do something on the front end in this case we're just storing that in a list and the rest of our react application works the rest of the rest of the way that a react application works okay any questions yeah to do in Erlang I'll let Anna ask that question or incident well we did use we yeah well that's for one you don't get ecto you don't get Phoenix you don't get the mix build tool although I I think you can use mix with Erlang I've never done it I it's I don't know how you would so you you lose out on mix you lose out on the that those function that function only you still have to build a testing library with Erlang how powerful this would be to do an elixir as opposed to maybe other languages that we've worked it in the past and how the language itself because of its this because of how elixir works and a lot of things it does receive from Erlang why it helps you develop good practices as you're building an application or forces you to so how did it make us better software developers well as we kind of mentioned through the talk elixir gives us practice over good development processes such as test-driven development it makes it really easy to do that so rather than having to spend a bunch of time setting up your JavaScript testing library so that it works against so you're setting up while you're moss and test library setup harness in your test harnesses all nice you just can you can do that pretty much for free without any extra work so the pressure of having to write a test is a lot it's a lot less pressure reading that's not what I meant to say the the bar for writing a test is a lot lower design driven development I don't know how you would do that I mean I think a lot of it and a little bit even more of a higher level and I have not been working with I mean I'm really interested in elixir and a really interested in learning more about Erlang elixir that long but it's interesting with some of the constructs in the language at least to me kind of really forced you to be intentional forced you to think about how you're setting up your application they really forced you to think about how you're composing your functions how you're working with your data and I feel like sometimes even though we are intentional we will always want to be intentional in other languages it's easy to sometimes lose sight of that and I think elixir helps reinforce think one more thought just higher level is that we didn't touch upon this as much we won't touch about how the language makes us better developers but the community is really active the open source communities really active there's a lot of support people are really willing to answer questions and there's a lot of information and resources out there to help speaking of which if you do want to be a participate in that we have there's two opportunities that are there any more questions will be around if there are yeah so there's two opportunities oh that's a really good question how do you sell good development practices up the chain to people who don't either to people who don't use the language don't currently use the language or people that are hiring people to use language is that yeah yeah but I think yes I totally agree with you that's how that's like the business model of Dropbox right it's really easy to use Dropbox so it from this bottom up but you do have like there are also cases for when you want to bring something into a company because it's eat if you want to bring it into a company you do have to do some selling so to speak and you do that I think I think you're right to augment or to add to what you just said you make it super quick to write tests and super easy to divide up your app in pieces and that's what elixir I mean that was actually the core function of this talk so the way that you sell it is you like the way that you sell it is you make it really easy and you make that perfect the prototyping process really flexible okay okay so we're totally gonna devolve into this discussion and I'm very happy to do it but I want to make sure that I share the two opportunities that we have we do elixir bridge where you we help facilitate learning elixir and Erlang and Phoenix we have a couple different tracks already bridge series of workshops they're free day-long workshops and the goal is to make it easy for people who are interested to have a space where they can learn if you're interested come talk to us about it and then we're also writing a book on elixir a community driven book on elixir called full stack elixir tweet at me if you want more information or you would like to get involved in the book we we like I said it's we like to have a bunch of different people collaborate I wouldn't say maybe and I also like as I do more potentially I wouldn't say bad but to think about Joe's point like it does make things really easy in certain ways but it does took me some time to understand some of the concepts right so like I had to be okay with being a little bit frustrated about how Supervision trees work or some of the other or gen stage or how some of these concepts work to really understand how powerful they are and so took a little bit of time of like being frustrated and being okay with that because I was trying to understand this tool that allows you to do like it's really powerful work I don't know what your experience I don't have a good answer for that yeah yet but I know you Ryan so we'll hang out later and I'll answer it for you later and for you on the live stream or looking at this later that's my twitter handle so feel free to tweet at me thank you [Applause]

Channel: Erlang Solutions

Views: 1,151

Rating: 5 out of 5

Keywords:

Id: Trhc5bygwFc

Length: 29min 52sec (1792 seconds)

Published: Fri Mar 24 2017