okay awesome so we're talking about the new web socket layer in Phoenix today I just landed this weekend in master so you probably have not played with it yet and basically the goal is or the vision is doing real-time stuff in your web app should be as simple and as straightforward as riding like a restful endpoint so we've built in some abstractions to make things like doing a pub sub and channels and something you would otherwise done in like socket IO a node you're familiar with that we build in kind of some similar mechanisms but going a little bit step further making it almost like a restful routing layer but for web sockets so it should be almost as straightforward is routing HTTP requests to a controller action we should be able to route socket messages about particular channels to a channel handler so we're going to basically show this off just by building a chat app and then maybe if we have a little bit of time I'll go into some of the fault tolerance that I've built in to kind of get some ideas on the approach I took and whether or not it's a good way to go but to get started basically we'll use Phoenix's new tasks so there's a Phoenix has a basically project bootstrap task to get your project going so if you clone Phoenix you'll be able to run mix Phoenix at noon in the Phoenix directory we're going to create an app called chatty and we'll give it a location to live this is basically just going to generate me a project hierarchy with all the necessary conventions set up in place but you could roll your own mix app yourself and pull Phoenix in if you needed to oops so if we go into that directory here we can see Phoenix basically just seated out a directory hierarchy there's a config for configuration different environments added a single pages controller so we get like a hello world if we launch the app and it gave us a router and then it also pulled in a static directory with jas and that's the two very small javascript dependency for the WebSocket layer so Phoenix Jas is going to give you the channel pub/sub support on the client and it's also go to multiplex single WebSocket connection so we can basically send multiple events and channels on a single connections so we don't have to basic we want to worry about starting multiple web socket connections because most browsers are limited to a certain number so if you open this up and explore it actually first before you forget we need to run depth get and compile after we bootstrap our project and then we open that up in vim so it looks like there we go still compiling should be about done here right all right so we can start this app up that's basically a bare-bones app we haven't written any code yet but Phoenix gives you a start task so now we have our app running on port 4000 so if I fire that up I get hello world so we're running a Phoenix app and we want to convert this hello world app into a chat application so to do that we need to do a couple things one we need more than just being able to render a hello world text to the screen so we'll make like a static index.html page now so again there's no view layer and X currently but it should be in soon I have a branch that has some experimental eex compiled tempting but at the moment there's no queue layer so this process should be a little bit more elegant pretty soon hopefully before electric op we'll have a better story but for now I'm just creative use file or views directory with an index.html that we can render and just so you don't have to see me copy a bunch of HTML I have already had an event buffer here so this HTML you see here it's nothing special it's using bootstrap and it's just rendering a input for a chat messages and as well as a user name input and that's it so there's nothing not a whole lot to it now the other thing we need is some CSS so it's not it doesn't look like junk so we'll create that and then in our pages controller that Phoenix added to our projects instead of rendering text hello world we actually need to render that file so again this will be a lot more elegant in the near future but for now we can just read that file directly what's up what's up you okay H you know I have a K good ah Turkey okay now we need to restart our server there's code reloading in place but just since we added new assets I just wanna make sure it's good to go theoretically oh this looks this looked a lot more pretty on my non shrunken resolution so we have a username input then beneath that bootstrap has collapsed our a message that's off the screen that no one can see that's it's not very nice oh there we go okay so I should be able to say my user name is Chris type a message and it should show up in the white space but this is just a static page that we've rendered with two containers so we want to make this into a real-time chat app now but to do that we have to do a couple things so Phoenix router anyone's not familiar that bootstrap task seated out a router already for us added a static plug middleware and then we have a single root get request that rendered that index page so what we need to do for a web socket layer support is use Phoenix router socket so it's an optional opt-in layer and what this is going to bring in is like a web socket handler for us and we have to mount that at a particular location so the way that web sockets work is they send a get request to the server and then it's upgraded to a web socket connection so that mount says what you want to serve as that get request that's going to have clients request an upgraded connection and the only other piece of that is Phoenix has a concept of channels so instead of dealing with straight web sockets we abstract a little bit so channels can almost be thought of like you have a controller for like HTTP requests channels are going to field a WebSocket connection so it's a place to group application behavior around a particular thing so if I had like a user's controller I could have a user's channel where I want to broadcast events about a particular user so we'll use the channel DSO so for a chat app we're going to have rooms channel so if you're happy you're modeling your application you're gonna have mini chat rooms so it makes sense for me at least I have a room channel and that's going to be handled by a channel that we define so I can find chatty that channels rooms so we haven't written that rooms channel yet but it's going to only be several lines of code so in our app we have like a chatty controllers folder we want to create a chatty channels directory and then we'll make a room's channel then we use Phoenix channel and we're going to do one more thing so all channels are required to define a join function and this is going to basically serve as your authorization for joining a particular channel so if you don't define this it's going to crash and you have to basically abide by very restricted level of conventions here so there's only two return types that this can return and this is going to serve as your authentication layer so when you're joining you get passed a socket that's going to be multiplexed and then you're joining not only a channel but you're joining a channel and a topic and this is confused a lot of people so hopefully I'll be able to describe the channel and topics and and what those mean so for now we can just think topic can be anything and then you do in channel the browser can pass up any kind of additional authentication data and it's some kind of message but for now all the channel all that the joy needs to do is return to okay socket so if the socket is authenticated it's only job in life is to either return okay socket or it can return air socket and then reason if they were not authenticated so we can say like unauthorized so those are the only two return types but for now we're not worrying about authorization there's no database layer we're just gonna have a global chat room so for now we just return okay socket and everybody's happy but you could have returned something like you know defined a authorized function passing the topic maybe the topic was like a shod version of some database room for your company maybe there's a room stable in your database in a room ID concatenated with some room key and that's what the topic is and then the browser client could have passed in some kind of a topic up to be authenticated with but we're not worrying about authorization at this point so we're saying everyone can join and that's that for now so that's our channel at the moment and we need to write a little bit of JavaScript to actually have this news item now so our HTML file I'm just gonna write the JavaScript directly in here because that's how everyone does it right no it's a it's just a little simpler here and it actually allows me to show you how simple it is to write a real-time app in Phoenix so you're shipping this in production you should priority AAB script outside of index.html but for now it's easiest thing for us to do so the first thing we want to do is say on doc ready we're using jQuery here we're going to create a phoenix socket so I can say VAR sockets equals new Phoenix socket and this is that Phoenix Jas dependency that you saw earlier so we're going to connect at that endpoint that we defined earlier so say whatever the current window location host is and then we mounted it at /ws so that's what this represents this is going to create a multiplexed WebSocket connection for us and Phoenix is going to handle all that and reconnecting on failure it actually will continue to try to reconnect till the end of time so it's kind of fun that I can shut the server down and hi leave a browser tab open and I fire my app back up I get a bunch of connect events because it's just trying forever so now that we have a socket we should be able to try to join a channel so I can say socket join and give it a channel name so we had a room channel in our router and then I give it the topic I want to join on and this the topic concept has confused a lot of people so I'm hopefully I can clear that up here so you can think of channels like controllers so there's just a place to group behavior application behavior and then a topic is almost like a resource so if I had like a room if I had a HTTP controller and I had like a rooms controller that I posted and to get requests to I would have some room ID and that would be the topic so usually the topics got to represent some kind of resource it could be an ID it could be a token it could be anything but in our case we have a global chat room so it's not really scoped to a database record so I'm just gonna call it Lobby so if we're building like a chat app we can say that you could have a lot of private chat rooms for like a bunch of companies maybe a bunch of organizations have team members that can subscribe to like a general topic on a particular company ID but for our case we just have a global chat room that everyone can subscribe to so we end up with a nice like tragedy of the Commons scenario with a global chat room but in our case we give it the channel name topic and then we can pass up any kind of authorization data that's required it's going to pass a empty JavaScript object because we don't care about any authorization in this case and this is what's going to call that joining it on the channel so then the callback we get past is a connection to that channel I'm sorry callback we get past it just is literally a channel so if I do nothing else here we should at least be able to see the joint events come into the server I go back to our rooms I'm just going to put a little bit of debug data here we're going to say the socket try to join socket that channel and the topic so we should at least be able to get here and have it not crash refresh this page it works we've got two joint events trying to join the room channel the topic was Lobby so WebSocket connection is currently connected to that channel it's active and it will keep reconnecting on failure so what I like to do here if I have a hard-coded channel like Lobby I'll just hard code it here so we can say anyone that tries to join I'm sorry hard-coded topic anyone tries to join the lobby Channel we just return to okay socket as we know that it's public everyone can subscribe to it and then none of that would that would let you later say okay we added a database we have companies with private chat rooms now and you could pass up some kind of a you know private topic to be anything to be on the scope of this so for now our implementation is just going to be you try to join any other topic for now we're just say you're unauthorised ok so we have a WebSocket connection we need to actually send some events so back in our JavaScript someone asked a question David I heard a noise I don't know if someone posted a question or what now okay okay so we actually want to send some events on this channel so we create a WebSocket connection it's not very exciting let's actually do something with it so we can say if we hit this callback you join successfully let's send in the vent out so before returning okay socket in the controller I can say okay broadcasts to anyone currently listening on this multiplex the socket and channel we want to broadcast the event maybe user entered to someone into the chatroom and the content of our message is going to be anything we want so I can say maybe the username was whatever the user name they passed up and their message they didn't give us one let's just say that their user name is a non for anonymous for now and then back in my JavaScript layer I could say okay I have this channel I'm going to say Chan dot on an event user entered I get past some kind of message from that channel and then I can say maybe I have some jQuery object messages dot append trying to stay basic right now so I'll use some nice BR tags that's maybe message user name entered maybe a blog to the channel that someone entered whatever that user name was they passed up so theoretically I felt so to find that container so I could say var messages it's going to be equal to I have a div ID messages here that's just the main chat message list theoretically this should work okay well good I'm showing off the Phoenix gives you nice error messages so I something blew up and I get a formatted error page currently that's handy let's see what happened oh I'm using the variable topic here but it was hard coded as Lobby so that's why I do okay so that it's actually good this popped up so Phoenix has code reloading but sometimes it's a little bit touchy so if you ever seen air that you can't define or module because it's currently being defined I apologize you have to restart your web server and restarted I'm basically recompile I'm running a mix compile in process to reload the code it's a pretty naive it works like 90% of the time sometimes it doesn't so till we have a better way I'm trying to fight my battles right now currently so that should work so now we notice nothing happened here but if I load up a different browser tab go back here I get anonymous entered so that actually works doesn't mean many tabs as I open here I shall just refresh this one so I refresh your browser every time it connects it's going to rejoin the channel and it's going to broadcast out that someone has entered so if I go back to my active original tab I see a bunch of a non entered flags here so now I'm going to open up by port locally and see if everyone can connect just so as we build this we can share what's going on let's see if this works it was like a there's a global chat in the hangouts right okay I just pasted a link into my infinite recursed background here I should see a bunch of people enter hey there's one baby okay so it is working I think awesome cool okay so we can see everyone is currently joining you're tunneling to my local River that I'm currently riding so don't do nefarious things to my computer I don't know what forward opens up so let's actually make us a chat app so now we have events that we can post JavaScript so I can receive and send events let's add a little bit of a real behavior so to do that on our JavaScript side we need to be able to post messages so we can just say use a little bit jQuery and we're good to do here okay we're going to say that your I have a couple inputs here so I have a message inputs and username input so I'm going to create those as jQuery objects quickly okay create a couple of jQuery objects and they're not going to bind to an event so if you type user name inputs on the first I'm going to unbind keypress so every time the website connection drops due to connected connection issues or for any reason let's go to refire this join haulback so any binding we do in here we want to make sure that we reset it so we can say if anyone remember what enter is 13 anyone key code for enter 13 I think so so if you type key press where it's enter we will actually want to send a message on the channel so we can say we want to say you hit enter let's H and sind so we had chan dot on we could say chants in to send a message on subscribe to amass Benetint so we want to send an event saying maybe we're to call it a new message and then we can give some jason to say here's some the actual payload so maybe we're to say that content of the message is a message input value whatever you typed in and I need to use brackets here you so writing CoffeeScript we're going to send up a content and then also the username of the person that sent the message and that's going to be the username input about you okay and the last thing you probably ought to clear the message input so you don't have to backspace it yourself the amiracle all this Java scripts correct the first time cuz I don't write vanilla JavaScript very often so we've sent the new message event here we actually need to handle it not going to do much it's going to ignore it if we send this message up so we actually have to handle it on the Phoenix channel side so back on our Phoenix channel in addition to my joint events I can corrupt I can define event functions so we can say when we get into Vince for a new message and this is arbitrary or whatever the whatever you want to pattern match on for particular sockets and pass up as a particular message payload and I'm thinking I pull probably switch around these two arguments here so if you pull Phoenix down tomorrow and it airs out it's probably gonna end up being a socket as a first argument always only because that's where I you sock as a first argument everywhere else so just heads up so any client sends us a new message event we don't have any database later all we need to do is a simple thing and say we want to rebroadcast that message to all everyone connected so I can broadcast out the event new message I'm just going to forward it to all active clients and I can just pass the message in here directly I'll be a little bit more explicit though instead and I'll say that the content is a message content that they passed up and the username was whatever they told us it was and then important thing to not for good is we have to return our socket from the event if you forget this you're going to crash and you'll get like a really nasty error message I'm going to try to improve that a little bit but since elixir is immutable we could change the socket state we might unsubscribe a user we might add some other subscription to the socket we could end up there's a way to store stay within the socket so we can be storing on basically an ephemeral state on our multiplex connections while the socket is alive so we always have to return the socket from each event you don't it's going to die and I'll try to improve the error messages that's just something to remember so basically all of our new message event does is say we're going to broadcast back out to anyone currently connected that same a bit message and that lets us go back on the client and just define another event saying okay anyone that receives a new message let's do something with it so on the new message events we're going to have some message payload and again we're just going to do a really quick and dirty append of a message so we know that that message is going to have a user name like before but instead of saying you entered let's just append the message content it's might be off the screen huh it works so theoretically we should have a chat app if everyone refreshes maybe enter your user name though and nothing it's worked see what happy no javascript error did we crash no oh my keypress should have been for the message input not the username input through try this again so if anyone is still connected to that forward URL we should actually have a working chat app right now I don't know if anyone can you have to refresh your browser though because it all the code changed so we'll see maybe like kill I'm have to restart forward I have to forward crashed or maybe because I killed the server all right give it another shot here so Ford is running oh the Europe yeah the URL change sorry about that alright new URL let's try it one more time that work that work David and then you should be able in or a username and a message and we should have a working chat up there is no cross-site scripting and sanitization so don't push Java scripts so this is a lick search a tap so we really it only took a few lines of code our JavaScript outside of the curly brace indentation if we were running copper script this would be this will all fit on a single my single screen here so it's a very basic API we can join rooms and broadcast on them we can send events but the neat thing is that we can do all of this outside of the web requests or the socket requests so if I have browsers up and running if you leave your browser's up and running I'm going to fire up a IX session here and I'm actually gonna have to kill my server again so the URL is gonna probably likely change again let me try this yeah forward dot so I'm gonna have to generate another link this be last time we have to do this because I wanted to start up a an IX session and could have connected another node and had it work but I didn't start up a named elixir node there so so new link D FY dot chrism instead of qgc if get some people hopping in there we'll do the next last bit of magic here so now I have an IX session up and running of the app I'm gonna get a bunch of log data here which is actually going to be problematic should have started a node here I could yeah I'm going to do that yeah oh my gosh well wait till everyone joins cuz I might be able to type here and not refresh so basically I can push a broadcast directly for my X if I can end up typing here I'm going to aliasing exchange of connections that keep dropping so basically I can say a channel broadcasts but I have a bunch of debug data oh I have live code reloading I can stop this we can make it better I'm going to delete that I outputs because it's making me not be able type I should get code reloading when i refresh here see what happens okay now I can type yeah I think everyone is going to reconnect as the server had to restart I'm still logging get requests which is a problem that's alright I'll just type so basically I'd say Phoenix channel broadcast and I have to give it the fully qualified channel and topic so I can say let's broadcast to the rooms channel lobby topic now give it some kind of keyword list payload so like I said the content is going to be a new line here Phoenix from my ex username is going to be IX it should air out oh I have to give it an event so I gave it the rooms channel lobby topic but I need a broadcast on a particular event so the event is going to be a new message just like our controller or our channel content Phoenix for my ex username IX and we should have gotten yeah everyone should see that in their browser so basically we're not limited to Phoenix channels in handling just WebSocket connections we could have this running and any node on any part of our cluster and we could be pushing out updates to the browser or any other listening WebSocket connection so my goal is this my goal for Phoenix isn't just to be just like a standard web application framework you should be able to do all kinds of stuff you could have some kind of node running a particular like analytic service and this thing could be pushing data to any active Channel and topic from anywhere it just kind of neat push another really excited message so that's basically building a chat app in Phoenix that's where I'm at with it currently but I think the current implementation is a good to go I encourage everyone to check it out and play with it I plan to try to build some kind of conventions on top of it or maybe pushing rendered partials down and doing some of the neat things with so we got a little bit of time the other thing I want to talk about was how we implemented the topic server and doing failover for the topics because I'm not an expert in OTP yet so I'd like to get anyone's familiar with doing a globally registered processes is please chime in or chime in later but I'll go into that momentarily so try to go into the Phoenix master repo I'll open this up here so this is just the Phoenix codebase and I'm going to just throw in a couple debug statements to show you what's going on here but basically I have someone raised an issue earlier with the initial topic implementation where there was a race condition and a node failed on the cluster we could end up with a bunch of ghost process or ghost topics that we're never cleaned up so if someone we had a lobby topic if you imagine if we had a topic per room in our chat application we could have hundreds of thousands of companies they could also the each company could have dozens of actual chat rooms so we could have millions of these topics hanging out there waiting for connections so we want to clean these up if there are no active web sockets for a given topic maybe everyone's logged off we want to go ahead and clean that up out of the process pool so we have that implemented in place and it currently does failover as well as race condition protection and the way it does that is it I'm using a topic supervisor so every Phoenix know that you start up starts up a topic supervisor and each of those is going to start up a topic server but there can only ever be a single global topic server on the cluster so the global topic server is just a Djinn server on its in it it's going to basically have all nodes compete to register their server as under the global registry so only one person can win Erlang will enforce only you can only register a single pin to a name on the cluster so only one person could possibly win if they win they're going to go ahead and start the garbage collection check and start the role as leader I fail OU's they're going to link to over whoever ones whoever the current leader is they're going to do a process link and the start is slave and what this gives us is if that node goes down that was the active leader since we've linked here it's going to crash our slave processes all those slave nodes are going to restart and it's going to be recalled and then the global competition restarts and only one person can win that whoever wins that everyone else links to their process and so on and this thing basically continues on forever always failing over r-e competing for the global registry and whoever wins serves as that topic server and the reason why we only have one currently is we only want to broker all WebSocket connections and topic join create and garbage collection events we want to make that synchronous so we're almost creating we're creating an intentional bottleneck from all these concurrent nodes running because we don't want someone to have their topic like Lobby garbage collected out from underneath them because the garbage collector is going to say is anyone actively listening it's gonna get no back but after it gets no back before it calls delete there could be a another node that actually had registered Lobby out from under them they could then have their topic deleted so by forcing this all into a bottleneck through a jint server Jen servers always process only one request at a time so we're basically forcing an attention to bottleneck to broker all copic connections in garbage collection and if the node dies it's basically going to restart our all garbage collection events so that's essentially the approach I took I'm going to spin up a bunch of topics to show you that how it actually works right go into my topic server I have a randomized time that the topics have to be garbage collected because if a node fails it's going to go through every single Phoenix topic maybe a million and restart a bunch of garbage collection timers for sets of like a couple hundred at a time if we had those all timers fire at the same time we'll end up like Dena was servicing attack being Origin server because all those topics restart at the same time so I give it a range it doesn't have to be exact because we just eventually want to check on these things every so often so right now it's like one to five minutes I'm going to lower that to show you how it works though so let's say 110 seconds to 20 seconds for now is it's good enough that should work we can say whoever fails to register as leader we're going to say starting as slave whoever one gets to start as leader alright so now I'm going to fire up a couple nodes in Phoenix let's say I X really really a secure password there and start it with mix so now I have this node running we can see that it started as later so we have a leader globally registered topic server and if I start up a second node name it node 2 it's hard as leader as well because they're not connected yet so we have them both running on their own it looks or nodes as leader what happens with we connect them together and say in one connects it and we can see immediately it crashed and started as slave to go back to our first node we can see that erling detected that there were there was a global name conflict and it killed one of them but since it's supervised that restarted and just said oops I lost the global name race now I'm slave I do nothing but link to the other node to monitor in case it dies so on the first node we can spin up oh man we spin up like ten thousand topics so if I ask Phoenix how many come out your topic we can see there's no topics it's nothing's happening so if I do like a you know meet maybe one 20,000 it creates 20,000 topics so Phoenix topic create is being called under the hood under Phoenix channels it's abstract all this from you so just to show you what's going on will create topic I so we're going to create 20,000 topics and these topics eventually we want these to be garbage collected if we don't have any subscribing processes because there's no reason for them to live we don't them to fill up memory eventually I think that should be it so that's going to create 20,000 topics and also start timers to garbage collect them and if we hop back to our other node we can see that this thing should be able to look up all the active topics and it should be a bunch of them so now there are 20,000 topics running and they should all start to be garbage collected between 10 and 20 seconds from now see that they're all counting down on I waited too long so they're going to be okay so we want me topics left because no one was subscribing so I'm going to restart 20,000 again and show you the failover show the current node this is in one that we're looking at this is the one that's being brokering all create join leave and garbage collection events so we have 20,000 topics created I check on this node there they are so if I kill this node any one just drop down we can see that no to actually just restarted as leader and it should continue garbage collecting and there goes so these topics are going to start counting down so node two just took over as leader after node one died and we just started garbage collecting all of the topics that don't have any subscribers so there's how I'm handling node failover and garbage collection fault tolerance the whole shebang I'd love to have feedback on the approach I think it should work really well like I said it's a forced bottleneck but I think that's the way these things make sense we have to make things synchronous we're possible when it using a gen server a single gene server global register for this for me seems to work perfectly you notice we did we had 153 topics I didn't get cleaned up it's because we're buffering these we're not creating 20,000 timers we're buffering there them up in sets of several hundred so we're not creating much unnecessary timers um so yeah so that's the Phoenix real time web socket layer I don't know if we have any questions or whatnot but hopefully that gives you a little bit of magic on where I want to go with this drop it in the easiest way is drop it in the chat let's get dangerous over there I do have a quick question yeah and Chris I can is this in master now yes as of this morning yeah so in this is also Phoenix is on hex now so I should mention anyone that's not on using their you're writing an open-source slickster project you should put it on hex hex is the package manager does a dependency version resolution and everything I just shipped like a couple weeks ago so get your project on hex all of Phoenix's dependencies other than Kowboy are now on hex which is nice so basically get on hex if you're not on there yet and you can use Phoenix and hex we open up that bootstrap project you go to that mix file with hex it's as simple as adding your Phenix dependency as just the Adam Phoenix and in the version and a hex hex is going to manage that install for you which is super nice but yeah this is a master nouns is on all on Phoenix 0 to 2 latest and great mentally ah there's a handful of conferences coming up if you have one you want me to mention right now feel free to drop a chat on the mic chris is going to be electric awning Jude what's your July July July so in July what's your countdown Austin's we're down in Texas Austin Austin is elixir code you can be talking about Phoenix and where that's going to hopefully we'll see a few favorite are a few later at that time I will be up in Michigan Detroit Michigan in the end of this month doing a quickie one-hour session on and started with the lecture as well as August I'll be out in Wisconsin that conference it's called that conference and then also James Smith who's on the sub session right now he is going to be at electric op as well talking about building command-line tools like screw so I think there's a lot of cool stuff coming up this year to be fun if anybody else has anything they wanna share feel free to either doing in a Google hangout or the Google event I've also dropped in both the chance as well as the Google+ event RSVP or June we don't have a topic yet but when we if you have one let me know either ping me on Twitter or something like that I'm a Cromwell Ryan or Crisco we'll figure something out you did it sure if you have like a 15 minute topic you want to talk about we might be doing we're doing a bunch of other videos with lightning talks this month so we also do vanilla the elixir one has a Miss lightning talks hopefully this will be recorded Google+ has been for public hangout recording these things yeah I'm running I'm running ScreenFlow locally so this would be up on YouTube in coolant and that's Molly on YouTube public hangout which it was last time it'll also be Chris Kyle's publish about there this without may be available so we'll put it out there and I can tell you anything else Oh Richard said he's going to be talking about growing in a DA a growing adoption of language and learning alex caruso also hot you should get tickets to elixir cops in Austin it's going to be it's gonna be good awesome alright well thanks everybody thanks Ryan for hosting yeah thanks everybody talk to you later guys yep yep thanks everybody
