A Beginner's Guide to WebSockets

okay so Deon here today is going to give us a beginner's guide to websites he said he will introduce himself so I'll get off the stand so he could get going and depending on how he goes is whether how much question time we'll get at the end okay so hi everyone welcome to my talk and thank you for coming today as you were saying I'm gonna be talking about WebSockets and we're gonna be talking about it from sort of a beginners standpoint so someone who's never actually really used it more has sort of like heard of what WebSocket is but has never actually gone to the effort of actually using it in one of their projects so a bit about me I'm doing music that's my name I'm a full-time associate developer at Y state currently I study at UTSA doing the vit co-op scholarship so that's the Bachelor of information technology I spoke at PyCon last year some of you might have been at my talk and I did a bit of data science work at the ANU last year as well and so I'm 18 years old and I'm obviously from Canberra and have recently moved up here to study and to give you guys a bit of structure about what we're going to be going through today we're gonna sort of start off with a basic low-level understanding of some of the protocols which we're gonna be talking about today sort of to get an idea of where everything sits in the whole IT world we're gonna go over some of the use cases and the applications which you might find using WebSockets and I'm gonna go over some of the previous technologies which might be used in place of WebSockets since it's still quite relatively new and will go a bit about the browser support and the characteristics of WebSockets as a protocol and how fast it is in comparison to other implementations and of course what you'll hear for is the Python aspect of it so we're going to go over how python has a role to play with it and how easy it is to do things with WebSockets with Python and just a quick note at the end we're going to go quickly over how deploying a WebSockets works over the web so to get into it we're gonna start off with a pretty contrived example and so anyone who's done any sort of networking or taking any networking courses would have seen something similar to this you probably would have seen something called the OSI networking layers so this is the Internet Protocol suite and this is sort of just a good idea of where everything sits relative to each other so I'm gonna start from the bottom you'll see the transport layer so these are things which define the way that data packets are transferred over the web right so these are things such as TCP and UDP and so TCP guarantees that any packet that you send over the web will reach his destination in the same order which there was sent right and it does this by acknowledging the packet once it's reached its destination so say I want to send a packet to a web server TCP guarantees that all of my packets will get there and I want to mention that UDP the less important one to TCP is a bit more careless and it doesn't really care about this guarantee of order and it's generally used for things like streaming video data where we don't care if we lose some of the packets on the way and so TCP is sort of the underlying transport layer behind a lot of the things I'm gonna be talking about today and so then you move up to the next stage which is the internet layer so this is things like ipv4 and ipv6 which define the addresses which packets are sent right so everyone has an IP address you've probably heard of this in any regards it's it's pretty much where data is sent on the web and so we then get to the application layer which is sort of the top-level layer and this is where HTTP WebSockets and the rest of them all sit this is the sort of stuff that you guys might interface with when actually developing things now again I want to admit that this is a very contrived example so take this layers in with a like a grain of salt I would suggest if you want to be further reading into where everything sits to either Google the OSI layer model or just looking into any networking courses that you can now I'm in a sort of basic just to gave one sort of up to speed and you probably all know what HTTP is but for those that don't it's pretty much this request model where a client sends a request to a server the server then processes that request and then responds with like anything so right we it could be a HTML page it could be anything and so generally you type in a URL the client is that browser that sends that request and the server does the processing so that could be anything and in our cases this is generally a web application like flask or Django and then it returns a response okay so I may be talking a bit about what HTTP is and a bit of the characteristics behind it so you can sort of understand where HTTP and WebSockets it relative to each other so HTTP stands for hypertext Transfer Protocol and it's a stateless protocol which means after the initial request is done that's it the the channel between the client and server is completely lost after this initial request right and so I like to think of it as flipping a coin you flip a coin once and you get your result heads or tails and if you flip it again that result doesn't have anything to do with the previous result right so the connection is completely lost after that initial request in the HTTP model clients who are requesting data have to specify an action so that's either a get post put or delete this just tells the server what they want to do so we get request is retrieving some data a post request is making something new on the server a put request is replacing something if it does exist we're like it a resource on the server or creating a new one and the delete one is deleting a resource on the server and these things these methods and characteristics are all put into a thing called a header which is sent over the web to tell the server what it wants it to do right and the server also responds with this header so I'm just going to quickly give you a quick example of a basic HTTP requests and I'm just going to show you how you could find these headers and look at what these things look like so if you open up any web page like my Olsen web page here if you open up the inspect tools and go into the network's tab and then you can have a look down here at all the requests you've made to the server and if you can click on that you can look in greater detail at what the request is actually doing so in a general sense you have all of the chrome that does this nicely and it sort of gets the data that you will probably want to look at at the top and then it gives you more detail beneath right and so you can view the URL which you might want a request so I've requested that URL and I specified that I wanted to get that webpage and then the server responded with a 200 ok response which means that my response was valid and it returned this data so you can also look in greater detail into the request and the response headers down here by this clicking view source and you can look at exactly what headers were sent up in the web now a lot of you who have done web work probably you have interface with this but I thought I might start off and show you exactly where you should be looking for this sort of data now there's a couple other tabs which might be useful when inspecting this sort of stuff like the response tab so you can see that the server responded with a HTML page and all my browser's done has loaded that page and then you have a timing page which also shows you how long it took to download all the data for this page cool so in 2005 we realized that that model wasn't really what we wanted we wanted the web to be able to do more than just respond with some data ok and so we invented this technology known as Ajax which stands for asynchronous JavaScript and XML ok and so what this allowed us to do is a synchronously send data to the server after the initial request so as you can see with the model the client sends the initial request for the HTML page it gets responded with that and then it can continue to make requests after that and that did a lot of cool things for us so we could do a whole range of new things with that and I'm going to show you another quick example just to sort of stick that explained an example type thing so you guys can get used to looking at headers in the inspect tab so this is just a basic page which where you click the button and it makes an ajax request to an api and the API responds with some data so if I click this you'll see I'll make a request I'll just sort of make the headers tab a little bit smaller and so what I've done here you just print it out what the request is so I did a get method to this URL and I went to that API okay and then that responded with a greeting just as arbitrary greeting which I created and this is great right so we could do these things now where we do the initial request and then we can get more data from the server so if we want to grab information we can just ask the server for it and it's a really cool thing we don't have to refresh the page and we can do a whole lot of things but what happens when you want to do something which is a bit more real-time okay what if I've got something like a feed or a Facebook feed or a chat application right where I want users to be able to instantly in real time be able to talk to each other okay imagine if say a Facebook wall or a chat required you to press get all posts to see the posts of your friends you you wouldn't want to be able to sit there the whole time pressing get all posts the whole day looking for new things I'm sorry but you get the point right you don't want to sit there the whole day pressing get all posts but if you did it I guess it would work and this this model could work for certain applications but for stuff that we want to be real-time it's not the ideal thing and so ideally what we want is a model where you can make a request and get a response for the initial page and then if the server happens to have some new data in its events it can send that data back down to me as you can see you sort of buy the model it shows when there's any new data it can send it down right so in the case of a chat application if I'm sending a message to a group chat with all my friends the server will get that message and then about okay I've got this message let me send it to all the people that want to know about that message and if they're connected to this the server can then send that message to everyone is connected so that's what WebSockets does right WebSockets is built to solve that inherit problem with HTTP and to develop that dual channel right so what WebSockets does is it defines this fully duplex bi-directional communication channel between the client and the server I know that's a whole lot of jargon but pretty much it means that the client and a server can talk in real time without having to continuously make requests so what it does is it if you want to do WebSockets over the web you send a header to the server saying okay I want to upgrade from HTTP to WebSockets and it uses that same TCP connection that it originally established loading the page and the server is like okay let's upgrade once this upgrade is done those two the client server can continuously send data back and forth with each other without the overhead which HTTP might bring with it and we never talked a bit a bit about the performance benefits you can have by removing headers from requests so essentially one of the great things about it is it's really easy to implement and we'll get into some of the details with that soon and it's standardized and it's been standardized for quite a while so I don't understand why people aren't really implementing more than they are and as the last point says there the headers are only sent once so in that initial handshake where the server and the client agree to upgrade to WebSockets that's the only headers that are sent the rest is just sending data back and forth and so you don't have that overhead when sending data so I'm going to sort of go over a brief example again with that same feed idea feel free to go on this if you want and we can try crash my server and so I'm gonna sort of show you how you can inspect them and sort of see what a WebSocket looks like so if you want you can again feel free to go on there and post a message so I could be like okay my name is Dion and I could be like hey I can make that post okay and so what's happening here is I've made the initial request to the server we've upgraded our connection I've sent my message and the service saved that message to some database on the side and then it sent that message out to anyone that's connected okay and if we look at the WebSocket headers which you can find by going in this little tab you can see this initial header so I'll just quickly drag that open it's not okay so you see the initial headers which happened with that upgrade connection so you can see the status code from the server was okay we're going to switch protocols to WebSockets and so if you want to look further into please be reasonable you don't see your drop tables okay so if you want to look more into what's actually going on behind the scenes you can go into this little tab called the frames tab and you can see everything which is happening behind the scenes so you can see in my case all that's happening for me is I've been I've upgraded my connection to WebSockets and now all I'm doing is receiving that data from the server so when you guys are making those posts is going up into the server saving them to the database and then sending them back to me and you can see that by looking at these here right so you can see exactly what the event was so I made an event on my client saying add to wall which does what it says and it just adds that text to the wall and you can see the the data that came with it right so you have the username of the person that sent it and the message that came with it as well and this probably isn't the best implementation of it but it sort of just defines it defines that that connection okay so we'll go away I also do apologize in advance for in creating all the curly braces in my slides I didn't realize until afterwards I just did it sort of as an aesthetic thing and I was like well I'm actually presenting this to a bunch of Python people they probably don't know that anyway so a lot of people criticize WebSockets for its browser support and we can now go ahead and say that that is sort of a myth because WebSockets is widely supported now in all of the latest versions of the browsers including ie which is good yes if you still support that and if you want to defer the reading and find these sorts of things for other things I'd urge you to go to this website can I use com most web developers who've done any front-end stuff with probably interface with that but it's just a nice little way of seeing if you can use it okay so I mentioned earlier that we're gonna be talking about other technologies used to achieve quite similar real time goals - what WebSockets does and you may have heard of these technologies before and I'm not trying to be credit them because they still are used today but I'm saying WebSockets can do all this but it's an all standard way okay so there are these two things called polling and long polling okay so polling is sending an ajax request any X number of seconds so I could tell my my client every five seconds make a request to the server for some new data now my work actually does something like this right and it's not probably the best way to do things but it does work so you will get some sense of real-time I guess because you will be data every X amount of seconds and it's sort of a workaround to not having to use that WebSockets and it should be used sometimes when you only want to be receiving data on intervals we also came up with another way of doing this called long polling now this is still I think widely used today from what I can see and if you look at a lot of applications where you'd expect WebSockets to be used like for example a chat you'll often see that you might be able to inspect any WebSocket headers and that's because people still do use this long polling technique you might have also heard this referred to as comet programming but essentially what it does is it makes a request to the server and it tells us server not to close that request until it has any new data so it keeps that connection open until the server finds something to give it and then after it gives it back the clients just like okay I'm gonna send you another request to do the same thing and by doing that you sort of achieve that goal of having dual connection between a client and a server and there are upsides to it and downsides to it and we'll get onto that quite soon another way of doing it is called server events or server sent events sorry and it uses this thing called the event source API to send events from the server and it's not truly bi-directional because it's really based from the server send it to the client not an interaction between the two of them and that's sort of the distinction between them and web sockets where web sockets expects both the client and server to continuously communicate this expects the client to sometimes communicate with the server but mainly for it to be server data being sent back and so this generally requires an event loop or asynchronous server and one of the downsides is you don't have that binary message capability which you will receive with WebSockets which supports binary donor and one of the main upsides of this method and I guess is the arguable thing you might want to consider when choosing to use WebSockets or not is using this method allows you to coexist with existing technologies and when I say that I'm saying it works well with rest api's and with things like oh all right so existing technologies which use that that model now WebSockets can't really interface with them properly and there's a bit of an issue trying to request from an API using WebSockets so you've got to kind of consider if you want to be using those sorts of technologies and if it's worth the trade-off now the browser support for server sent events isn't as good as WebSockets which is funny it's pretty much everything except for Internet Explorer but it that can also be polyfilled so I guess it's arguable that you can pretty much use WebSockets all these in replace of each other the intended use case of WebSockets this is sort of a topic which is I guess controversial so I don't want to make my opinion change this but one of the things I want to get clear to you guys is WebSockets is not a complete replacement for HTTP right WebSockets is an upgrade we saw earlier with the headers it's it's not a replacement for the HTTP model but it's more an upgrade for that channel so you can't just replace HTTP users because HTTP provides you a number of benefits which you won't receive with just native WebSockets like for example as mentioned on the slide automatic caching and you can communicate with those other technologies like a rest and auth and one of the things that's often another critical point about WebSockets is load balancing the server's is often quite hard with the WebSocket protocol and it makes the whole implementation of deployment a little bit more complicated than just doing HTTP server and so the intended use case for WebSockets is generally things where you need that full duplex client-server interaction so it's things where you're doing things in real time like creating maybe a game on the web chatting applications where you've constantly got that interaction anything which needs that low latency real time connection over the web ok so we're gonna get into some of the clients which are used to interface with WebSockets so these are the things which you use to talk to a server and the server either upgrades or doesn't upgrade with you and I'll get into what that means in a second so these clients are built in a lot of different languages including Python which is an episode and even micro Pi and Arduino have them which is kind of cool so if you enter a IT sort of things you can use WebSockets which is awesome so if you want to control any IOT devices you can create that WebSocket connection with a web browser so for example if I wanted to interface with one of my IT devices let's say some kind of robot or something I could control it using my web browser with a WebSocket connection and so clients which interface with WebSockets aren't necessarily web browsers like we'd expect but the most popular implementation of these clients are built in JavaScript and are used as web clients and obviously they require the server to be able to interface with WebSockets if a client tries to request an upgrade with a server that doesn't support it it'll obviously respond with an error and you won't get that upgrade and if you want to read more into sort of clients and looking for the right one that you might want to use for in your projects I'd consider looking at socket IO which I'm gonna go into a lot more depth in a second and looking into just some of the web socket libraries just by doing a simple google search for a web socket client and if obviously you guys are Python people so I'd assume it'd be looking for things in Python and those do exist so for this example I'm going to be showing you the native web socket support which is in JavaScript I'm going to show you a bit of code sort of work through how quite easy it is to create a client which can interface with the WebSocket server so in this example we start off with the initial socket connection which is just a new web socket object and again I'm sorry for pushing JavaScript on you guys I know you're not here for this but you give this this new WebSocket connection and you define AWS route so for this example I'm connecting to my localhost and then you have to define a couple of functions so the this socket object has a non open event which means once the protocol has been swapped and upgraded to WebSockets this function it's called right and so in the example that I've got up on the screen it's just a lambda way of dealing with that so for example in my example I open that connection and then I send using socket sent I send the string PyCon au up to the server and then when the server sent something back to me the socket on message is called I just realize I don't have too much time so socket IO is another thing I want to also mention and it's sort of the main thing I want to push it you guys to use this is a nicer way of interfacing with WebSocket service because it's sort of like a jQuery to JavaScript right so it's sort of like this this library that takes the native WebSockets and makes a lot nicer and easier to use right and so it includes things like Auto reconnection and fall backs so say a server and a client don't agree on the handshake what it does is it uses a fallback mechanism to use long polling instead which is quite good if you want to support older browsers and it gives you that ability to do that it also as native WebSockets does handles disconnection in connection and it gives you this ability to create these things called main spaces which is sort of like a group of clients which so they can talk to and so a socket IO client side code is quite similar to what you saw before you create that initial connection and you give it the name space as one of the parameters to the URL you have a connect event and you can also send events to custom-made events right so the native web sockets can only send events to and a message event on the server whereas this I can define custom events to do that and I'm gonna skip over this slide because I'm slowly running out of time but there's a lot of support for Python servers which do do this and I'm going to be using flask for a lot of my examples which we should hopefully fly through but there's a lot of different limitations for using socket IO it's really really quite easy so you create a flask application most of you might have used flasks if you'd ever done on your web stuff with Python but it's as easy as wrapping the app with the provided socket object and then creating a route which takes in a custom event name like for example my sockets sorry this isn't actually an example for native web sockets so this is where you're using that native connection and then socket IO is where you can do custom events and so this namespace allows me to address a group of people let's say the namespace PyCon so I get all the people to connect to this namespace Python and then I can send messages back and forth to anyone that's connected to that PyCon namespace which is really quite cool for managing groups of people so let's say you've got chat groups you can use this namespace to achieve that and also I came before the performance comparison between WebSockets and just HTTP it's quite big so HTTP requests they always send that header whereas WebSockets don't so after that initial connections done every other request is only framed by a two byte header frame I think it's called a frame and it it's not as heavy as the multiple kilobytes which a HTTP request might have but then there's also that socket IO idea from before also does contribute to adding latency to that WebSockets but it's not as much as HTTP might I'm gonna quickly go to a speed test if I can this is just my localhost and what it does is I enter a number of requests I want it to perform and it just does that so I just send in a basic string to either my WebSocket endpoint or my API endpoint and you'll see how much quicker WebSockets is in comparison to Ajax this isn't actually hosted on the on the web because I wanted to take latency out of it out of the equation but you can see it looks a bit small here because it's in milliseconds but you can imagine having a thousand hundreds of thousands of users can to a connection all doing these requests you can see how that can quickly get quite expensive now I was going to do a live demo but I'm quickly out of time I'm just gonna quickly touch on deploying these WebSockets and so synchronous servers don't generally have support for this so we have to use libraries such as event late' and G event to monkey patch some of the standard libraries in Python so synchronous just means that it it uses blocking functions and using these libraries unblocked suit I'll go into more detail if anyone's to know about deploying it afterwards because I think I've literally got a minute but deploying WebSockets does add that little extra layer of complexity and doing them with asynchronous servers makes your life a lot easier and it's sort of the point I want to get across so unfortunately I'm gonna have to wrap up early but if you do want to talk more about this feel free to find me outside I'll probably be loitering around for a little bit and thank you for coming [Music] thank you dear as you can see is very enthusiastic about it so I'm sure he'll answer your questions later on

Channel: PyCon AU

Views: 40,011

Rating: 4.8242774 out of 5

Keywords: pyconau, pyconau_2018, Python, PyCon, PyConAU, DionMisic

Id: PjiXkJ6P9pQ

Length: 29min 52sec (1792 seconds)

Published: Sat Aug 25 2018

Reddit Comments

[u/hellocs1]

great presentation, /u/thekingdion

[u/joshuabelden]

This kid is brilliant to listen to. It's a shame he had so little time.

[u/sfx]

What's with all these people using the RemindMe bot, as if the save button doesn't work.

[u/GazWay]

Wait, 18 years old????

[u/SuperSensonic]

Woah I need to get my Pi robot up and running with WS. Great presentation!

[u/RobBuddha]

I really enjoyed this. He is a great presenter and I learned a lot!

[u/tansim]

bro the transport layer comes before the internet layer...