C# Sockets Programming - EP01

oh yeah all right so we want to talk about client-server socket programming oh yeah welcome to the first part in our multi-part series on sockets programming over the next several episodes of this I want to build out a fully functional viable real-world product using sockets and along the way we'll learn client server socket development and so I thought the best way to do this would to build something you can actually see in the real world and that's Walmart pay oh yeah so if you're not familiar with Walmart pay if you go to a Walmart then you go to one of the self-serve checkouts and you scan all your items a QR code shows up on the point-of-sale you can then pull out your Walmart pay app scan that QR code and that pays for it so that system is made up of a couple of different parts one obviously the point-of-sale talking to a back office system that back office system talking to a cloud system cloud system exposing a web api that then then the mobile app can use that web api to go make the payment so along the way the communication from the pause and the cloud system if you will is a persistent socket and the reason for it being a persistent socket as opposed to the HTTP call is the need for the server side of that component to be able to send a message to the point of sale to let the point-of-sale know it's been paid for that particular transaction so by having a persistent communications channel this allows for something known as full duplex communication which just a fancy word for saying messages can originate at both sides and can overlap each other whereas if I'm using the mobile app and I'm making a Web API that's your traditional client-server request/response disconnect and so without further ado let's learn the right sockets code come on okay so here we are I've got visual studio open there's no solution it just completely blank and we'll we're gonna end up creating three projects with inside of a solution one for a client one for server and one shared and in this first episode really all we want to focus on is solving the chicken-and-egg problem as it is and the idea is that if I'm building a client-server app well it's hard to build the client without the server it's hard to build a server without the client and so you kind of have to build them both at the same time and all we want to do in this episode is get what's known as an echo server up and running and a client that we can then test out some ideas to that echo server and so what's an echo server it's exactly what its name sounds like it's a server that when you connect to it whatever data you send to it it just immediately sends that same data back to the client all right so it's just an effect echoing back everything and on the client side will learn to send and receive a basic message some basic data and then we'll learn some basic encoding for a message and we'll wrap it up there so let's get started all right so in Visual Studio the first thing I'm going to want to do is just create a new blank solution so I'm gonna say a new project from here I have blank solution because I use that quite a bit new blank solution I'm gonna create this in my little drive down here a little shuttle drive and sockets programming I'll select that and we'll just say episode 0 1 sockets programming will be the name of it all right so blank solution again free projects now client server and shared and we're not really going to do anything was shared right now in later episodes that's where we'll start adding common code between the client and server things like a channel and encodings but for right now I just want to get the structure laid out and then we can start filling it in so let's add a few projects a new project to this solution dotnet core and we will just call this one client and we want that to be an actual console app so console app and we will name this guy client create flow and visual studio you can do it alright let's add a few more add new project this will be another console app next and this will be server create and then we will now add our shared library again we're not gonna use this shared library and this up and in this initial one I just want to have it created so it's ready to go class library next and we will just call this shared which is a really horrible name you should never name anything shared but gonna do it anyway alright and let's set these up client dependency at reference will add shared as a reference all visually there we go server at reference reference to shared like that okay these guys are all set up and now let's start building so in our server this is where we're gonna focus on first and the idea is to build what's known as an echo server it's gonna open up a socket it's gonna listen on that socket for an incoming connection when a connection is established we're just gonna begin looping and reading the code off of that socket and immediately just writing it writing it back out in our server program we now want to go ahead and add our echo server so I'm gonna add a new class and we're gonna call this echo that goes over all right so a couple of things on the way sockets work in terms of a binding or communication so your should be familiar probably with the notion of ports so when I call out to HTTP as typically on port 80 or port 443 for secure 80s non secure port 8080 is an alternate HTTP so every server listens on a range of IP addresses that are assigned or associated with that machine as well as a particular port and so when you're building a server you want to make sure that you're using a port that is not currently used by some other system or program running on that machine for my case here we're gonna use port 9000 I know nothing is using it it's well above sort of the standard port ranges and it's a free port and so I'm gonna say public void start in port and we'll just default it to 9,000 and next what we're gonna need to be able to do is create the socket that we're going to listen on and this is where the server side is a little different than the client side on the server side I actually create a socket that allows me to accept connections and when I accept a connection from that socket the result is a new socket right and so it's the server side is a little different in that respect from a client but once you understand what's kind of happening with it it'll start to make sense so the easiest thing is let's just go ahead and get the the echo server up and running I'm actually gonna cheat and I'm gonna copy some code in here and just walk through it all right troll dot bring all these in dot I'll bring all that in bring this task alright so here's a couple of things that are going on so one is the endpoint and this is where our server is going to listen all right and so for this endpoint we're going to use just the loopback address and this way that our server is not actually actively listening on the network it's only listening on the one 2700 that one loopback interface of this particular machine we're gonna create our socket which uses the address family from our endpoint or you're gonna do a stream and it's all TCP right and more about this will come later we bind that socket to the endpoint saying this is what I'm going to be associated with and then listen takes a parameter called backlog now dotnet for whatever reason and this drives me up the wall and anybody else who's coming from different operating systems and writing sockets for different sockets libraries typically the backlog has a defined constant called s Omax con but essentially what it is is is how many connections pending connections can be sort of queued up in the system that you'll allow for and really what that does is is it's a matter of how fast you can respond to those connections and process them before they begin to timeout so it's a number that you'll have to fiddle with and we'll discuss more later but really they should have provided a default for this they didn't so I'm going to use the default which is 128 it's a typical backlog backlog number and we're not doing high-volume processing where we would need to tweak this up or down or reject systems out right and then the last thing we're gonna do is just launch a task and don't care about they the result of the task and that task is gonna be called do echo and so let me come over here and copy my echo code all right so now this has a lot going on essentially this is probably the only gnarly piece of code in here and this is really just a rapper rapper extension method that's created to turn async call back style code into a task and just makes wrapping it up a little easier this is the only gnarly piece of field we will actually see from here on out this kind of got this table cute to it for the most part things are pretty pretty easy so in this case I'm just creating a task Factory and I'm wrapping the beginning except and end except of a socket so this initial socket that we create again is what we're listening on for new connections the result of that begin accept and accept is a brand new socket which is the client socket which is this socket right here and so I'm going to do a couple of things and if I go look at the socket definition so 12 if I look at a socket itself you can see it's disposable so we can go ahead and we probably should just make sure we clean that up you always want to clean up style same thing with this this network stream true we'll make this guy the owner of it and using all that up alright so essentially what will happen is our client will connect the beginning connect method gets called this role rapper takes care of this the end except gets called when we've completed it and the result we get back is a socket which represents which actually represents the socket that we can use to speak to the client and now from here we come down network stream is gonna be our friend this is how we read and write to a socket it's very efficient it's used underneath the TCP client classes in dotnet as well network stream just makes makes life much easier and we'll get into optimizations later but the idea now is I create a stream with that socket that I can use we're gonna create a default buffer of about 1k and the whole point of this buffer is just so we can read data into it as we read data in we write that exact data straight back out and that's the whole point of the echo if you notice here this veidt's red equals zero anytime you read from a socket if it comes back a zero the likelihood is and typically what that means is the client connection is gone but at the lower level of TCP we did not get the control codes to shut down the socket gracefully and this is one of the challenges in raw socket programming is the connection from the client to the server is kind of this virtual path it's not really this physical thing and unless the client correctly shuts down on its end to send the control signal all the way across the server won't even know that the client is not there the only time you kind of figure it out is when I'm doing a read sometimes it'll come back at zero sometimes I won't come back out of this read at all it's not until I try to write to the socket until I try to do something with the socket itself before I know that it's actually not there so this is just one of the many little guards you have to look at to say if I read off of this I didn't get any bites more likely that that sockets gone so from here we'll just break out of this loop and start again for the next connection all right so that is our echo server and again all we want it to do is just read data in and immediately write it straight back out as always the code is in the links are in the description below for all the code for this so you can walk through it I'm not going to spend a whole lot more time on the echo server let's go jump into the fun stuff all right so now our client we're gonna take this in phases and the first goal is to just get it up and running get a connection sends send some data receive that data back and echo it to the screen so VAR endpoint equals new IP endpoint and the address is IP address local or the loopback and then the port is 9000 oh I forgot so our echo server we need to start this so far es echo equals new echo server echo start console.writeline echo server running console refund there we go so now at least our Erica server will start up alright alright back to our client so just like the the server we're gonna have an endpoint which represents where we're going to connect to where is on the server this is where it was going to listen from we'll create a socket it was new so new socket and the endpoint address family this is where dotnet gets a little annoying with its attempted it helping but so we're gonna have the address family we're gonna get the socket type which is going to be a stream which just means where the the data is just streaming back and forth it's not packed in any particular manner and then the protocol type is TCP so our socket with the socket the next thing we want to do is actually just connect to our endpoint and so if I say socket connect and I give it the endpoint and now we can simply read and write alright so one thing we want to do here is console.writeline press ENTER to connect we want to make sure our echo servers running first console.readline I'll just love to get the inner so we connect with a socket bar network screen equals new network network stream we'll give it the socket we'll make it the owner of that socket the network stream just makes it easier to read and write data to it for message equals the canonical hello world all right and this is important network stream right and if we look at right and we'll just do right you can see that it doesn't take a string so I can create a string you know string writer and right to the stream there's a number of ways I can do this I want to start out by doing it and a somewhat good fashion and I'll say bar buffer equals system text encoding utf-8 get bytes message so the first thing is how do you represent data going across in your particular system and a lot of that is going to come down to the type of message format that you're going to use whether it's a binary protocol whether you're sending JSON or XML how do you move that data back and forth and depending on the application that you're building really is going to dictate the protocol for the example system we're gonna build we're actually gonna end up doing XML and there's a lot of reasons for XML and the flexibility behind it but even with XML or JSON when you serialize that down you get essentially a utf-8 string we're going to take that utf-8 string we get the bytes for it and the reason for doing utf-8 is to make sure that we can properly encode all characters right so obviously English fits in the in the actual ASCII table chart but if you need accent marks and foreign language you're gonna need utf-8 so that you can potentially get to byte Unicode characters to make sure you fully represent everything so just start with utf-8 it covers pretty much every case you need when you're doing sort of string based encoding for messages if you're doing a binary serialization format that'll be something a little bit different but it'll be alright for right now we're going to do utf-8 we're gonna get the bytes and now I'm going to write my buffer 0 is my offset and then my buffer dot length that just sends it across the stream now I just want to read it back essentially what data did I send out and how can I read it back so now I can save our response equals new bytes and technically we know how big the buffer is that we sent but for the sake of argument say we don't know we're just going to read back a single buffer and then just display that on the screen so now I can say network stream read and I'll put my response 0 and now the read side I'm going to say response dot length bar fight spread and this is the next challenges well how much data my supposed to read back and this is where encoding of your message is gonna come into play and we typically do that with a byte header whereas if you look at something like HTTP the content length HTTP header is you know down inside there a little ways it's a kind of an oddball protocol most of the network protocols that I implement start with a four byte header and that 4 byte header is the length of the message that's about to follow and that allows you to quickly validate the message size you make sure it's not a negative number you make sure it doesn't exceed the size of messages that you allow in your system gives you a quick way to go either create a brand new buffer or reach cycle a buffer that's sufficient size to hold the inbound message and as well as you know how much data you actually have to read in order to get the entire message for our simple one especially on this little loopback sending the message reading the message a single read right will suffice or single read will actually pull back everything we need so I can do var response stir equals system text encoding utf-8 hitch string and response console.writeline received and let's interpolate that response stir all right if we've done all of this correctly this should connect to our our echo server send the hello world string and then respond back to us so first thing we want to do is set multiple startup programs so we'll come on here set multiple start ups and start and start fly ok launch them out let's make sure everybody gets running all right so the echo server is running clients ready to connect so move this here so I connect received hello world all right and you can see our echo server client connected hello world sent and received so that's the easy part right we say ok you know I've got an echo server I can read data and the echo server is extremely important because it just allows us to to really test out our concepts and ideas and for our client we've just sent hello world and received hello world now we need to do something just a bit more and what we really want to be able to do is say how can I send a message and what does that message look like how can i encode it how can I decode it all right so let's take what we've learned in terms of basic reading and writing and we're gonna change this around a bit and we're gonna start sending a message and look at how we can properly encode a message so we know how to send it and more importantly how to receive it in its entirety so let's come to the top and for sake of generics we'll just say public class my message public string will say string property get set public and in property get set so this is the message we're gonna want to send back and forth and again to encode this message we need to pick a format binary JSON XML or some proprietary one try to stay away from fright Airy ones what I see a lot in the industry I actually see a lot of XML in the industry XML is extremely flexible encoding there's a ton of tools for it being able to translate and transform it with XSLT as part of a processing pipeline and mapping it's one of the most heavily used message integrations formats that that's out there so that's what I'm gonna do for this example here alright so what I want to do is a couple of couple of things we want to be able to do we want to be able to send the message so I'm going to say public we're gonna do this one async task I'm gonna say send message and for this we'll just do T key message so I want to send will do send async in order to send this message this will go to a network network stream all right so in order for me to send a particular message in this case my message we need to be able to encode the message so static we're gonna call this byte byte array which will be the header fight array which will be the body will say info TT message static t we want to decode by its array body so I want to be able to decode a message encode it decode it and the opposite of Sunday sync is to receive so static async task receive T network stream and stream there we go alright so we're gonna send a message we're first going to encode it so serialize the message itself and create the header which in our case is going to be a four byte for byte length header expressed a network byte order and so this is another gotcha that many programmers miss is when you're sending a numeric representation and which is different than encoding our message and serializing it but if I'm sending byte values that specifically represent numeric values and I'm going to directly interpret those bytes those bytes need to be a network byte order and what that allows us to do is be agnostic between different types of systems that have big-endian or little-endian byte ordering for the way they represent numeric values let's just let's tie it together so if I look at Sunday sync and we say okay well what what really needs to happen here so I need bar I need header body equals encode and so if i encode the message then I can do a weight network stream write async head zero header dot length duplicate that up and then this just becomes body and body dot length so that one's pretty easy alright so if I'm gonna send something I'm going to encode it I'll get back a header and a body for the encoding z-- I can go ahead and write the header then I can go ahead and write the body receive is gonna be sort of the opposite now in our case we know that the header is four bytes so I can easily just say bar header equals new byte four and now what I need to be able to do is actually read in four bytes so I'm gonna make a nice little message we're gonna swap this out here just a bit I'm gonna say static byte array decode T so static byte array read acing and network stream network stream and bytes to read our buffer gets new fighter a bytes to read this little helpful method will just continue to read off of the socket until we have all of the data or the number of bytes we're requesting for it to read like say bar fights read equals to zero and now I just a while fights read less then bytes to read bytes read plus equal a wait network stream read a sink will read from our buffer will go the offset is the number of bytes I've already read the count is bytes to read - fights read I actually want to do that up here as well figure away false just close this off the side alright here we go let's just go task fighter red and pacing there we go so what we're in effect doing is we are reading until we've read all of the bytes that are being requested and we just sort of keep track of how many bikes we've read that will become the offset into the buffer and then how many bytes that we want to try to read we try to read as many as we can this can be problematic because if you remember before if we get a 0 that means the client is gone so what we want to do instead of just this simple bytes to read want to save our I'll just save our new bytes if new bikes equals zero then basically the socket is closed so throw new exception I'm just going to say socket close dad old you know later on we'll clean that up in a way but again if you read zero bytes off the network stream the sockets closed and if you just need to go handle it and throw away that client so now what I can say is bytes read plus equal new bytes and new bytes is a terrible saw just say bytes received I before E except after all right G ived bytes received there we go and now we were turn buffer perfect so now what I want to be able to do is receive my header so I can save our header equals a weight read async network stream and for our header sizes force that's number bytes I want to read and now I have to turn this header byte package into an integer value that we can use to determine how big the body is I need to read so let's do header bytes of our header length equals suppose a bit converter to in 32 header bytes and then IP address host our network to host order so here's this part where we're taking the bytes would turn those into an integer but because they're in network order I now have to go from network order to host order why that's a function on IP address I have not a clue that just seems like the most bizarre place for that particular function to be but it is so with the header length I can now save our body bytes equals a weight read async and now I can pass in the network stream and the header length which really this is body length so the header bytes convert that that will give me the body length and now with the body I can return decode T and body bytes now it's just a matter of filling this stuff in Oh tasks tasks feet there there we go all right so we can encode so let's go implement our encoder and again we're just gonna use XML write it out pretty simple so bar access equals new XML serialize type of type of T we want to end up writing this to a string so that we can take the string and we get the the bytes and so if I look at XS and I serialize I can go to a stream I can go to a text write or an XML writer so our easiest way is for s begets new string builder or SW equals new string writer to SB so excess serialize SW and message now that's completely serialized just writes the XML into a string writer for our body body bytes equal system dot encoding system dot text encoding utf-8 get bytes sb to string and now here's where we need to create the header bytes our header bytes will fit converter get bytes but we need an integer value IP address and we're gonna say host to network byte order and now I need an integer which is the body byte stop point and return header body so pretty simple encoding function take the take the message whatever that object is serialize it X amount get the bytes from that utf-8 encoded get the length and then pull that back out now from the body we just want to decode these bytes so far STR equals system text coding you TFA get string from the body and now we need to load up this string into the serializer because we want to deserialize from this guy like a far string reader gets new string reader I think I can do it this way on bar access gets new XML serializer type of T XS deserialized string a reader and that gives me an object return as T doesn't like T because it's not that one so we'll do it this way is he all right shall we should have it so we should have encoding we should have decoding sending and C receiving a message just clean some of this stuff up a little bit so sunday sing receive a sink looking good and we've got our message so let's go test this out back in our main we'll leave our socket and our stream and now what we want to do our my message gets my message and the int property will be 404 and spring property will be hello world and let's make a little print function so static boy print my message and this will just go to console.writeline and will say my message dot int property equals and in property my message string property equals m dot string property there we go I can say console.writeline sending and print my message Sunday sync and since this is a sink we now need to change our main static function to be an async so we can do that so wait send the async network stream my message so now we've sent our message off now I want to receive 4 response message equals a wait receive async my message the type of message we want to receive back from the network stream again console.writeline received print response message all right let's see what we've got so now if we run this eko servers running clients ready to go we hit this sending 404 received 404 so we've made pretty good progress we know how to create an echo server which is one of the most useful things in client-server development is to be able to just bang out an echo server real quick to get up and running we've got the basic client fundamentals we know how to encode a message send a message receive a message it's a pretty good foundation point so with this gonna wrap this episode up in the next episode we'll begin building out the shared infrastructure and introduce the concept of channels and encoders and the channel gives you the abstraction around the communications and the encoder helps to do the messaging coding so just like we did in line with XML we'll go ahead and implement an XML and a JSON encoder to look at how those differentiate and how you can start plugging those into a channel until next time happy coding [Music]

Channel: Richard Weeks

Views: 19,802

Rating: 4.9008265 out of 5

Keywords: C#, .NET, Sockets, Network Programming, Development, Visual Studio, Payments, Messaging Systems

Id: rrlRydqJbv0

Length: 41min 32sec (2492 seconds)

Published: Thu May 21 2020