Gnubesoft: Async Python Websockets - Part 1

hey folks this video is a continuation of my intermediate Python series but the next several are going to be kind of standalone so the idea behind the whole series is that you don't learn how to program just from watching me program right you learn to program from programming on your own and this is designed to give you a project to work on because you learn so many different things doing all the things that happen in this big project of making a Dropbox Club but the next several videos are going to be specifically about web sockets I want people to be able to come in on these videos if they're just interested in web sockets and don't care about the rest of this so the like this is a great module web sockets it's a on pip it's great for asynchronous networking stuff but it isn't particularly well documented like there are a lot of people showing you the best practices for this so I have spent the last month and a half like really testing this thing out I figure out good ways of structuring stuff and I I've got it like I really feel like I've nailed it so I'm hoping that this is gonna be really helpful to anybody who just happens to come along this video and and sees hey this is how your web sockets like I don't ask people don't like and subscribe and all this stuff but I'm gonna ask you to share I'm working so goddamn hard on these videos and it's not finding an audience like if you think that this is a value please let other people who might be interested in this know about this cuz man it's it's a little heartbreaking to see some of these videos getting less than 100 views after all the effort I'm putting any of these so let's get on with this this is the the best way to use WebSockets to start this off let's make a networking file that we're gonna put all our networking stuff into and we're gonna import the socket module which is built into Python so we don't need to pip install it or anything like that but we do need to figure out what is the IP of the machine that's running our code like before we can do any kind of networking stuff we got to figure this out if you would ask Google Google's going to give you the wrong answer so Google will tell you that you go suck it get hostname to get the hostname smooth this will be whatever you've named your computer and then your IP is supposed to be a socket get post by hostname and then we would plug in whatever our host name is and this should be our IP and that does look like a reasonable IP but that is not my IP so if I go to my terminal here and do IP config we do find that IP but this isn't the actual IP I use this is only used if I'm using a virtual box virtual machine so this will work in like 90% of the time but if you've got a complex like multiple these net connections on a single machine it's not always gonna give you the right one now fortunately the the socket module can still get us the right IP but to get the right IP we need to make the computer establish a connection to the Internet and we'll figure out which of the many possible Ethernet connections it's using to connect specifically to the Internet so to do that we're gonna use a with context thing here we're gonna create a socket from the socket module now it is expecting us to tell us what type of connection we're trying to make here so we want to make a ipv4 connection because we're interested that this is an ipv4 address this is ipv6 we're specifically interests an ipv4 so we do that by giving it this socket AF eye net and then you need to tell what type of protocol it should be using so you've probably heard of TCP / IP there's also UDP that's what we're gonna be used and that is a sock data gram so it's sock underscore G gram here yeah so we're making this socket and we're calling that suck at s and I would just need to make that connect anywhere on the internet and this needs to be given as a tuple because we're gonna give it the IP and the port and the the easiest one to remember is Google's DNS which is 8.8.8.8 and then 53 and note here that 53 is being given as an integer and the the IP is being given as a string so once it makes that connection we can just do a get sock name and that's gonna be returning a tuple so we'll overwrite what we have for my IP as the first part of what this returns I I'll just show you what this returns so we want the first part of that to be my IP so now we've got our actual IP pretty hair no Stan show you so this is the lie this is the truth so you can do all the server or client connection stuff with the socket module but that's not gonna be useful for us because we need our thing to be asynchronous and socket is not asynchronous at all I get it's gonna break our UI it's gonna bring all our background processes every time it's trying to do anything so that that's not good we need to be using an entirely asynchronous module for all of our networking socket handling so instead of using socket we're gonna import either through pip install or I'm going to use the package manager here and PyCharm web sockets make sure that you've got the S at the end of this because there's also a web socket module here on the pip so you want specifically the sockets alright so we install that and let's import that we're also going to need a sink IO and while I'm thinking about it I'm gonna import trace back because we're gonna be using this for some error reporting stuff later so what we've got right now is good we want to always be finding the hostname and the IP he for the machine that were let's get rid of this part because we know that that's the wrong part so we always want to find the host name and the IP of the machine that's running our code so whenever this module is imported into our main project that's fine it should just do these things outside of any kind of function or anything like that but if we're starting a server we don't necessarily want that to always start whenever we import the networking document because that could that could be a mess so let's do it a main clause here so only if the name of the thing that is wrong is this which is the way that we describe it if networking is the thing that we're running then we want to be starting a server so let's make a start server object from the WebSockets serve and here it wants a whole bunch of stuff we don't have to give it all this the first thing it wants is a WebSocket Handler and we're going to build that in a second here we'll just say that we're gonna make a thing that's called registered client and then we need to give it the host the well it actually the IP so I'll give it my IP and then we need to give it a port and there's a lot of tricky nuance to picking a port for your program because there's a lot of different programs out there that are already using up a lot of different ports so you need to find one that's not gonna be used up by anything else for the purposes of this I'm gonna use 1 1 1 1 because it's easy to remember and I know for a fact that there is no major program that uses that anywhere so well we'll use that so now we've created a start server object a we can just stuff this into our event loop so we do async i/o get event loop run until complete we give it this start server thing and that's that's it that does it see it also get the event loop and now run forever because we're gonna be throwing other things in the event loop but just doing this starts the server we need to now make a funky called register client so that when somebody connects to us it's gonna know what to do with it so let's define this register client and when a client does connect it's gonna this serve server thing is going to pass to our handler the web socket itself and a path which like if you go to a website you got slash index or whatever that's the path that's not gonna be relevant to us at all because we're not using paths so we do have to have a second argument in our function is if we're only looking for the sub web socket it's gonna break when it tries to throw two at us so we can just put an underscore here and that's gonna basically know about but it does you definitely have to have something right there or else it's going to break because it's gonna pass the path even if there's no path so this function is where we're gonna stuff all of the things that we want our server to do once it gets a new client connected to it so a good example that it would be to wait for it to say something so async for message in WebSocket print message now you'll see that it's still got a red underline here and it's usually that means that I don't have a function that I'm calling here but in this case it's actually a type error or it thinks it's a type error where it's looking for an away table object and instead it's getting a KO routine and if you come across this in anything it's it's something you can completely ignore as long as you're running like Python 3.7 ish or later you could probably ignore this because that away table and co-routine are just different versions of the same thing that are being they're confusing the linter inside of pycharm like that that's all if that is right now is PyCharm being confused about this so it's looking for the same thing we've got it what we got a server here all right like if I run this now we should get any errors it's not doing anything cuz we haven't made a client to connect to it but this is creating a server so let's make a client so we can actually see that mmm now too to make the client part here though we're gonna have to really think about what we're trying to accomplish here so I've got a helpful little animation to to walk us through that whole process so it's easy to make a connection to a known fixed server because you just use whatever IP or hostname that is like when you're connecting to a website or whatever but we're gonna try to connect to unknown IPs and to do that we're going to need a port scanner where we're gonna just run around through our local network trying to find anything that's going to connect to whatever port we've specified as the port of our app so it's going keep trying to connect to every single possible IP in our local IP range and if it connects great if not it moves on to the next one and tries next so let's make a sync port scanner and the first thing we need to do like really like this is for real we need to make sure that we're only on a local IP when we're doing this cuz in a lot of jurisdictions port scanning is illegal so you don't want to be out on the internet port scanning all over the place like that that would be bad your your ISP would really not like that so let's make sure like damn sure that we are on a private network here so if not we'll take the my IP here and get the last three of whatever that is so if this does not equal one nine two and not my IP yeah I'm doing this right okay I'll explain it in a second I got to remember all the possible local private networks and make sure I'm nodding at all these okay so we're just getting the last three of our IP or excuse me the first three this is this : three I'll show you my IP equals and my IP the first three okay that's what we're doing here we want to make sure that's either 192 or 10 dot or 172 and if it's not one of these then it's gonna fail this and and and condition so we want to print that let let the user know this is not a private network shutting down and we'll just exit and in fact I'm gonna strongly suggest that you double check that you got this part right so let's let's just stop what we're doing we're gonna make a little client tester thing to make sure that we're doing this right so just like seriously you don't want to screw around with it so yeah we're gonna import drip box networking and async i/o and we're gonna create a client things function that's just gonna do ever is that we're testing here so we're testing the networking port scanner and we need to await that and to run this now because this is a a sync thing we can't just like call this anywhere else have that sustained work we need to actually stuff this into an event loop so let's do that the main clause here just to be neat async i/o get event loop run until complete client things all right and this should work all right it didn't crash out so it that that was correct but let's just make sure that it does close itself if it's not if we're not on a one of these known private networks so I've just changed this to something that isn't okay so this is not a private network and it did shut down okay yeah everything is working great let me put that back to 192 so that works quickly so we need to figure out what is the range of IPs that we are going to be trying to connect to so we know what our IP is and we can do that I haven't shown you guys any kind of neat tricks with lists and we're gonna be doing a really neat trick with lists here so let's say there are IP range equals and we take our IP and we because our IP is a string we can split it on the periods I'll show you what we get here so now we get a list of these four strings you notice the even though they're numbers there are strings here because they're inside of these quotes so we can now do our pop trick you might remember we did that previously or we can just pop out this last one because that's not gonna be relevant we're looking for probably print that we did that is he we're popping up this last one because that's what we want to get rid of we want to be able to put in our own numbers here and now we're gonna clean this up so you can follow it a little bit easier now we're gonna do a joint we're gonna join this list so to do this we make the character or whatever we want to be the connecting tissue between our list elements so in this case we want it to be a period because we want it to look like an IP as once we make this this little string that's just the character we want everything to join around we can do a join and we stuff in the IP range and now the IP range is this by era liquid that worked so now we've got a new string that we've created from our list that we created from our old string and it's exactly what we need here we got the the basic I was taught that this was a C class that's not correct but like the this this we're up to this part where we can stuff in things after that so now that we got our range we can just say okay let's ping every single possible IP in that range so I equals 0 while I is less than 255 here we're going to increment inside of the loop so I is now going to be 1 to 255 we're going to look for a target IP which I'll just use an F string here to put everything together we reuse our IP range then a period and then whatever I is and it's print that's to make sure we're doing this correctly sorry yeah we got all the possible IPS in this range now we want to actually attempt a connection to all these so I'll say our connection is equal to a weight WebSockets Connect I should make this a URI cuz it'll look neater your I equals and we're going another string here ws Ford slash forward slash target IP : and this is going to be whatever our port is so we're using 1 1 1 1 1 so this is the the place we want to connect to we just plug that into here and then we want to test the connection by sending a message now that needs to be awaited too now we need to get into all of the possible error handling because we're gonna immediately start hitting errors so let's stuff this into a try and start accepting the exceptions that we're expecting so big one does you speak connection fused will just pass on that or we should just pass that but I want it to be visual for you guys so we'll just say no server server connection refused and there's also gonna be well I'll let the other errors as we also closed that we just Bobby yeah let's just get this general one passed here everything else though we want to see what that is so we're gonna use trace backs print exec which gives us a trace back exactly like if we had hit an exception that we weren't handling for and that'll be very informative it'll give us the details of what type of exception we're hitting and we'll be able to handle for that so this is going to immediately kind of hit that connection refused and connection refused but here it's timing out because it's it's trying to wait a really long time for this connection and there is a timeout argument inside of WebSockets for this connect method but it completely ignores it and it's badinage so yeah you see here we did get this timeout error and that wasn't handled for but we we've got that in our traceback exception here soap and you'll see that it did move on to the next one which is also going to timeout but there is a way to make this work that's just a little odd we can use a sink iOS wait for and just stuff what we want it to wait for in there and we give this a timeout of since we're doing a video here I'll give it two seconds and this is going to create a different type of error but we can handle for this except timeout and now this timeout error is not a built-in sort of thing like these connection errors work we need to actually import to this exception in order to handle for this exception so let's do that from concurrent futures import timeout error I'm gonna put this as connection timeout error just so it looks nice and neat with the rest of our exceptions here so except connection timeout error actually make this a F string here a timeout on and we'll say the target IP oh I guess my timeout length was too short because it's not getting the refuses is there are actual machines there where it's refusing I want to be connecting yeah catching those refuses okay so yeah let's giving it enough time to connect but still moving on rapidly so there we got a timeout and another timeout and so yeah okay that's working let's now actually make sure that this will work let's cheat real quick here and skip this up to 140 as remember in our our server handler for clients it should print out the message if we now run the server and the client together so here's the server going and now let me get the client going so it's starting at 141 which is gonna timeout timeout okay so no timeout that worked so let's check our server and it did it got our message so here is where we could try to start writing some logic for like a sink for message in connection print message so now let's let's make this just reply so a wait message send or websocket send hellooo yourself and this should work three start the server let's restart the client and we got our message back so we're getting a message on the server and we're getting the message to the client and this is like I said this is where you would put the actual like program logic of what you're want to do with this client and what you want to do with this server connection but you'll notice that our port scanner has stopped it's just stuck right here on this async for message in connection it is not gonna move from that at all ever again until the connection closes like actually if I I do this it's going to create a connection closed error which I wasn't handling for oh that's in WebSockets WebSockets exceptions sanction closed there's a bunch of different connection closed types there's error and okay the closed is the superclass we can just use that that's that's what happens when the server closes things and then it moved on from there and it started trying to connect to other IPs in its range but that's a problem for us because we're trying to create a peer-to-peer network and we want all of our connections to be connecting like all of our programs to be connecting all over the place so we can't have it just give up after it connects to one so that's what we're gonna do in the next video we're gonna structure this in a clean useful manner for peer-to-peer connections instead of this simple server client relationship I hope that this was something you could follow like this is just take a deep breath at any point if you're feeling overwhelmed take deep breath and release and say python is easy because i'm telling you like it's it seems like it's a lot but it really isn't and it it's just me talking real fast trying to explain it all this is super easy compared to any other program language like we're doing all kinds of fancy ports cannon and connecting and servin like that this was all we had to do to create a server it's easy python is easy so hope this is helpful I think you were much my Petrie on patrons David folks you you

Channel: Tim Webb

Views: 3,800

Rating: 4.9340658 out of 5

Keywords: #learntocode, learntocode, learn, code, programming, tutorial, asyncio, threading, pycharm, websock, websockets

Id: Vt4DJV45sKY

Length: 28min 3sec (1683 seconds)

Published: Sat Mar 21 2020

Reddit Comments

[u/Discchord]

My latest video, released today, concludes a mini-series I've done for the powerful Websockets package in Python. This asynchronous package lets you create anything, and in this series I create a P2P network. The last 2 videos in this series cover security and encryption. and (I hope) provide a fun demonstration of how to do it right!

This is part of a larger series that covers Intermediate Python programming concepts. I try to present everything in practical real world examples. And I'm super ADHD so I keep things moving! I'm the one guy doing programming videos that you don't need to speed up in the YouTube player!

https://www.youtube.com/playlist?list=PLMD4SGMb7qkVt4drPdDxImhhGw3ZNuzbB