SF21VEU - 10 Introduction to WAN Optimization (John Pittle)

this is the introduction to win optimization traffic we're going to take a look at using wireshark to assess the effectiveness of your whenup features and deployment so moving right along here we go um so this session is actually an update to the one i did uh the virtual us session that i did last year and due to time constraints for today's session i'm i'm going to skip some of the background and intro material i'll also move through some rather quickly so that we can get straight into wireshark but you can find the additional background and the key concepts in the uh us virtual session and the links to that session are here on this slide sharkfest virtualwireshark.org and the link to the youtube uh session so a little bit about me i've been a shark fest instructor since 2017 been heavily involved in performance engineering since the early 80s and protocol analysis in particular since the early 90s i've been with professional services at opennet and riverbed since 2005 and i think for those of you who have seen my session before you know that i love the mystery of a complicated performance issue and you also know that i shaved off my beard in 2003 so no more beard so why why did i think this session might be interesting um well it's because when up technologies in general will modify or enhance what you might consider to be normal protocol behavior and you're going to be seeing some protocol behavior in wireshark that might look confusing and questionable i know it did for me the first time i had a look which is why i was really thinking about this session and you know the more background that you have the more effective you will be interpreting wireshark to evaluate your wenop deployment you know whether you want to make sure it's configured properly or you're doing a proof of concept and you want to make sure it's providing the benefits that you were expecting so what's also interesting is some of the behavior that we're going to look at is very similar to other tunneling and proxy technologies so you will as we go through this content you will gain knowledge that will help you in a variety of special technology situations so that's why i put the session together here's our agenda the first two bullets will go pretty quickly why went up and an overview of features it's it really will just be a subset i would ask that you take a look at the um uh the virtual us session from last year to see the full landscape uh of features but so what after that we're going to jump into wireshark samples and look at some analysis examples and then we'll wrap up with q and a so here it is it's time to time to get started so why when optimization well it really comes down to we're trying to provide some benefits to the organization improving user productivity through faster response times and also reducing wan bandwidth usage that was one of the early drivers for wind optimization but there's a new one around reducing cloud egress costs so um for those clouds that are charging you know that your rate your your cost is based on how much data leaves that cloud there's some opportunity to reduce that amount of traffic and therefore reduce the cost so those are the high level benefits let's look at some concepts to level set some terminology and do some baselining so um as anyone who's done any performance analysis knows you know application performance is is uh heavily reliant on network performance particularly around latency on end-to-end network health bandwidth congestion all of those things can lead to poor application performance and typically as i suspect everyone here would say the network is the first to get blamed um while you know it can also be other factors certainly like you know server utilization or application coding or other data related factors that are messing up messing up the application um for our discussion today we're going to be focusing on the network and and in particular we're going to focus on latency um and i want to distinguish between latency and a term i'm going to introduce in a second and i'd like for you to think of latency as that measure of um propagation delay across the network so it's really the physics of of getting the signal across a particular distance in time and generally the higher the distance higher the latency the worse the performance will be and we'll dive into that in just a minute as well so the closer you are to the server uh you might say wow the network is fast the farther away you're you're probably not going to say that and it doesn't matter how much bandwidth you provide it's the distance that's going to provide the challenge and we're going to spend a lot of you know a lot of today's session is around um that that distance effect so separate from latency and again i'd like you to think of latency as the speed of light propagation delay across a particular distance you know you can't you can't do anything about that propagation delay so the round trip time then is think of that as the time required to send packets between two hosts a to b and then back to b b to a [Music] and uh think of round trip time as including latency speed of light delay but also congestion and protocol delay as well and again more or faster bandwidth is not going to improve um latency so we we have to find different ways of of addressing latency and that's what we're going to be covering in today's session so let's get back to there we go so there's a couple of wireshark metrics that are very relevant to latency and round-trip time so the first one is um tpc and tcp analysis dot initial underscore rtt um usually represented as small i capital rtt and it's a static value for the life of a connection and it's a measure of from sin to synack plus there's an x factor that i haven't really found a good explanation for yet doesn't mean it doesn't exist i just haven't found a good one um so that's the first one we're going to be looking at the next one is ack round trip time so how long does it take to act a particular tcp segment and then the third one is axe underscore frame what is the particular frame that's being acknowledged so we'll get some sense of how long did it take to set up the connection initial rtt and then um throughout the life of the connection what can we learn about changes in round-trip time using um time to act and uh act for frame so here's an example decode um i'm going to try to zoom in a little bit to that make it a little hopefully a little easier to see whoops come back there we go so here we can see a three-way handshake sansenac ack and you can see that there's an rtt value that shows up on the synack and again that's being calculated by uh by wireshark and it's notice it's also a little bit higher than um the actual sense in act delta time and that's the x factor that i at the moment i don't have a great explanation for um so i was thinking it might be you know if we if we factor in the um uh the the third part of the handshake with the act coming back and that math kind of works a little bit just looking at these numbers so it might actually be the total uh observed of cincinnati but the key is to to be aware of it and it's going to provide some value uh in in our analysis um the next one is uh what i usually label as round trip time to ack and ack four and in this case i'm going to zoom in here as well you can see now the the uh initial round trip time column as well as round trip time to ack and act four so act four is really helpful it tells you the packet that's being act and then the uh round trip time to to to to get that ack back from the distant end so very useful numbers and i have a question on the slide which is what is the round trip time to act the client hello message so here's our client hello message being highlighted on the screen and you can see if i zoom in a little bit i'm going to assume that zooming in is helpful so we'll we'll continue to do that so the round trip time to act the particular client hello was 83 milliseconds and some change so there we go 83 milliseconds and some change so initial rtt rtt to act act 4 those are some very helpful measures of of round trip time which does include latency and um i want to introduce a key concept which um there's not currently a metric in in wireshark specifically for it but that concept is an application turn and an application turn is a the pair of a request message followed by the response message so the two together the request going out the response going back is one turn the request message could be multiple packets if you think of a post http post with a lot of payload that could indeed be multiple packets and the response that comes back might only be one packet or it could be multiple but it's one turn and the key is is that for every turn you have the duration is at least one round trip time there's no way around it um and 10 you know so if we just take an example 10 turns at 100 milliseconds of round trip time is going to cost you one second of response time or duration so a couple of really quick uh examples in terms of http the request would be a get the response would be some http code response a 200 a 301 304 whatever the response is hopefully not a 500. so that's an example of a turn in http in smb an example would be maybe a read command in this case reading the 16 kb block at file offset 0 that's the request heading to the server then there's a turn and the response comes back with the requested data block in this case potentially 16k if there's that much data and of course that would be multiple packets but at the application layer it's one turn last example um sql pretty straightforward there's a select command from a database client to the database and there's a application term the response the status of the query comes back along with the rows from the database assuming there are rows so application turns are a key concept and there's one more useful metric that i want to put up on the screen and and it reflects that some of the pro some of the d codes not all of them but some of the decodes uh are able to measure and report on the delta between the request and the actual response in terms of a time delta so i'm just going to zoom into this a little bit as well so here we have a an http get and a okay that came back and in this column time since request the field is actually http.time so with smb it'll have a different field name but it's the same concept um and the and the idea is it's giving you uh a measure of how long how how much time was there between the request and the response so another useful metric we've got four under our belt now so let's move on to talk about transport layer and tcp is it is it sensitive to latency does it have a concept of turns as well and if you think about congestion window mechanisms like slow start delayed act re-transmits fast re-transmits things like that um there are there is a a re a res uh round trip time penalty uh for some of these mechanisms so um so the answer is yes it is sensitive to latency and the higher the latency could impact your tcp transport performance as well and that kind of leads us into a key point from this entire session and you know my hope is is that if you if you leave here with only one key point from our time together today i hope it's going to be this key point and that is latency times turn rate so we've introduced what a turn is a turn rate is just let's say the number of turns per second or some interval so latency times turn rate equals user pain the higher the latency or the higher the turn rate the higher the user pain and therefore complaints from users and it's always the network so we get dragged in to help figure it out if you can reduce the turn rate you can reduce user pain and if you can reduce latency somehow you can reduce user pain so anything you can do to improve user experience is generally a good thing and when we looked at our our key benefits at the beginning of the session around why optimization it's reduced it's improving user performance reducing bandwidth these things would would hit that bullet of improving user performance so quick review again we're going to move quickly so we can get to some wireshark samples challenges to user performance not enough bandwidth a transport protocol chattiness application protocol inefficiencies and latency is what drives those top two crazy and and ultimately you have to solve all three uh to see performance benefits if you solve for bandwidth but you don't solve for latency it's not going to create it's not going to necessarily create an improvement for for the users all right so that's our that's our super high level uh background um around some of the protocol concepts and let's move now into a few slides on a steelhead features overview so first off some of the key features are around transport optimization tcp proxying and spoofing some super intelligent caching compression dedupe when connection pooling so that connections are already open between two endpoints you don't have to suffer through connection setup time let's say from singapore to london it's the connections already open overriding some tcp options at at the three-way handshake we'll see some examples of that there's also some enhanced packet loss recovery mechanism so regardless of what the two endpoints are able to do in terms of packet loss recovery and re-transmissions um the wan-up devices have their own enhanced capabilities to make packet loss less of an issue and then finally being able to detect super high latency like satellite links and switching to some optimal protocols specifically for super high latency links there's also some features around application protocols so we talked about tcp and transport but there's also some very specific optimation optimizations for specific protocols like being able to pre-fetch or change some behaviors at the at the negotiated protocol settings like for example with http if if one end says close the connection at the end of this request we're able to override that to say actually keep the connection open we don't want the overhead of of opening new connections for every single request so prefetch read ahead uh caching and then some other um quality of service policy based routing some other capabilities in a nutshell because we don't have time to go into all of these in this session in a nutshell um there are some common themes to all of those optimizations and those themes are guess what reducing turns reducing payload remember if we can reduce turns we can improve performance so ultimately if we do those things we can reduce user pain which is really the mission reduce user pain so that's it for our background uh stuff we're we're about uh 20 minutes into the session and we're going to now move into drilling into some wire shark analysis and timing samples speaking of timing samples ignore the little uh stopwatch in the top left hand corner of this slide and it may show up in some others i meant to take those out before the session so they're not really relevant to today's today session so we're going to look at two different scenarios the first scenario is from a virtual lab environment where we have a couple of virtual steelheads simulating a branch and a data center we're going to look at enhanced auto discovery and we're going to look at some transport optimization features in action and then we're going to jump to scenario number two where we're going to look at a a version of steelhead that's running on my laptop it has the the name client accelerator and it's going to be connecting to a steelhead in the in office 365. known as sas accelerator and i'll be looking at some uh controlled tests where we're going to look at transport optimization and improved response times so a little so let's get started on scenario one so a little background it's it's from our training lab environment uh all the components are from our training lab environment it's virtual everything so steelheads are virtual the client hosts the server the wan is virtual the latency injection is all virtual um and and you know i just want to make a comment virtualized components are great when you're doing functional testing when you're doing performance testing they can throw you off a little bit and um you know i'm grateful that we had an environment where we could spin things up and do some some controlled tests but at the end of the day we have to re just remind ourselves that uh virtual everything so when the steelhead is doing its processing it's subject to hey is esx going to give me the cycles i need to do my work um or not so virtual everything and we're going to explore something called enhanced auto discovery and then we're going to look at some transport level optimization um and then the reason i picked these two is you know the one of the goals for this session is to share with you some things that you might see in in wireshark that maybe look a little strange and and that's what i want you to be able to walk away with um in addition to the key point from a few slides ago of latency times turn rate equals pain so but i do want you to be able to recognize things in wireshark that are relevant to again wind optimization in general steelhead in particular and other kind of proxying type and caching type technologies so here's a little bit about our topology we've got a client that pretends to be in a branch we've got a stale head that pretends to be in a branch we call that the client steelhead we've got a simulated wide area network of a one and a half meg and 96 milliseconds of latency between the two sites then we have a server-side steelhead in the data center on the um on the right-hand side and then our web server and as part of pulling these captures we're opening the web server opening a folder pulling down some high-res images to generate some traffic and i've got four capture files which i'll upload with the presentation after the session in case you want to look at these uh after uh you know post post seminar so one of the things we're going to look at is auto discovery it it does look a little bit funny when you're looking at the capture so i wanted to make sure we talked about this it's initiated by the client side steelhead the goal is to discover are there any other steelheads in the path that are close to my endpoint server and and if those are if there are other steelheads in the path close to my endpoint server um let's set up a peering relationship and optimize the traffic and the endpoint client the endpoint server are totally oblivious to this happening in the middle of the path so it's totally transparent in terms of a bounce chart of what does that look like um we can see the starting from the left the client sends in the send the client steelhead sees that send forwards it on up the path we don't that the first steelhead doesn't really know what's in the path on the way to the server so this begins our discovery and we're going to look at the details of the tcp options in just a second of what's in those syn packets and what's in the synag packets coming back so the the sin ultimately gets passed up up the stream towards the server we do in this case we're going to hit a second steel head it's going to pass on us in to to whatever is upstream steelhead number two doesn't assume that there's not a steelhead number three because there could be um so uh onward towards the server the server gets it says hey look it's in a syn packet let me do a synack and we'll complete the three-way handshake with the server-side steelhead in this case steelhead number two the two steelheads start to negotiate some control um uh mechanisms and settings and then eventually steelhead one will complete the three-way handshake with the client and we're gonna we're gonna look at that in some detail coming up here so let's be on the lookout for some of the unique behavior we're gonna see some new decode lev uh labels that you may not have seen before we'll talk about those you're going to see a synack retransmission we'll talk about that you're going to see that the initial round trip time is higher than what you might expect we'll explain that and we're gonna see some tcp options being modified along the path we'll take a look at that so i've put together um on on this slide the four the three-way handshake as seen from each of the four capture points the land side of the client steel had the wand side of the client steelhead and then the server sorry the wand side of the server steelhead and the land side of the server steelhead so that we can see um all of the interaction that's taking place along the path and in a minute we're going to take a look at the journey of the syn packet across all the interfaces and then we're going to look talk about that a little bit and then we'll look at the journey of the sin act so that here here begins our journey of the sin packet so we're going to start at the far left hand side where the client is and and this um these tcp options are coming from the land zero interface of the client steelhead so this represents really what we're seeing from the client and as you can see if i drill in a little bit to that you can see the client says hey maximum segment size 1460 that looks right window scaling of eight that looks pretty interesting and yes i'm happy to do sac with you so that's what the client asks for and next in our journey if we um take a look at what comes out of the wan interface so now in the wan interface uh let's see if i can do this there we go and the wan interface now we've still got the 1460 we've got everything that the client provided plus we've added something called a riverbed probe query and we're saying hey my name is um client steelhead 101 120 21. is there any are there just in case there are any other steelheads on the way to the server i want you to know hey i'm on the client side this is my ip address and maybe we can set up some optimization if that makes sense for the journey so let's look at what happens next um now this particular capture from this wan zero interface turns out to be the exact same i should have left this off apologies for that and there wasn't time for me to do it uh to un undo this so you're going to see all four captures showing up in these slides just recognize that what we see on this wand interface is the same as what we saw leave the other side so let's see now what does the server steelhead present to the web server so that's going to be here and we'll zoom into that a little bit so now we've got a couple of changes um notice we've got timestamp option now so the server side still had decided to turn that feature on so that it could have better understanding of what might be upstream and we also changed the window scaling so that was a decision made by the steelhead to perhaps reduce the belief being that hey i'm closer to the server we don't need window scaling so high i don't want to get overrun but i want to be able to respond at land speed so i don't really need such a high window scaling and we can see that just in case there's yet another steelhead upstream we've preserved the original client steelhead ip uh with with the notion that this is a probe query um request so let's take a look now at um some of the decode so this was all tcp options let's take a look now at some of the decodes so here we are with the four captures overlaid on top of one another and i want to zoom in on that if it'll let me zoom it as being a little tricky so here we are at the client-side steelhead at the top everything looks pretty normal and we see some http traffic as we would expect given that this is an http test um and then if we go down a little bit we see the client-side steelhead now has an s-plus and then we see a ack with a plus plus and then we see a re-transmission of a synack with a plus so or sorry s a plus so we have a an s plus s a plus s a plus plus and an sa plus as a re-transmission and we want to understand what the heck that is all about and and i have a slide next slide we'll explain it um the server side when steelhead wan interface is the same capture again it didn't really need four captures for this example i could have gotten away with three and then on the lan side we see the um s plus star which is basically saying hey i've been through uh the client initiated i've been through a second steelhead just in case there's a third steelhead in the path want you to know that we've gone through two steelheads already trying to figure out who's going to appear with who and in this case there's only the two of them and we'll see them appearing in just a minute but notice that the synack from the server so the server just ignores all these tcp options that it doesn't care about the synack from the server doesn't have any special flags so let's take a look at what these um there we go at what these special flags mean so um they're basically it's a the the s plus is a probe hey any steelhead's in the path s a plus is a probe response um s plus asterisk i'd like to do auto peering if there's another steelhead upstream more than two in the path and then there's some other special purpose uh interesting decodes that you'll see and this is in the standard wireshark this is not any special version of wireshark it's the standard version of wireshark so some questions you might have about all of this so why does the server side sim server steelhead send have the the plus star we just talked about that in the last slide why don't we see any http traffic on the wan interface captures and we'll talk a little bit more about that in a second but the net of it is we have a connection pool between the two steelheads and the traffic that's coming from the two endpoints is going across that connection pool not as http traffic but as optimized traffic in special formats kind of you could think of it as a tunneling protocol um between the two between these two steelheads it's encrypted um it's tunneled it doesn't show up as http so why did they another question why did the server steelhead change the scaling factor well i can't tell you exactly why i just have an opinion and the opinion being hey if the thing i'm talking to is locally land attached i don't really need a scaling factor of 8. so let's let's ratchet it down so that we we don't kind of get overrun with too much traffic at once why did we introduce tcp timestamps again i think it's because hey if i have the timestamps i can make some good decisions about throttling not throttling changing receive window sizes etc it helps me understand exactly how far away my my partner is and link speeds and some other details and then finally why is the um initial rtt greater than the expected latency so for that question we're going to go back to the bounce chart again and we're just going to note that because we have this extra set of round trips between the two steelheads notice that it delays the return of the syntax back to the client so we've just seen some kind of strange decodes and wireshark that it's like okay what is this stuff why why are we seeing a duplicate act why are we seeing these funny flags why are we seeing the high round trip time and the high round trip time is because of this extra interaction between the two steelheads before we return the synack back to the client and you can kind of see that in the picture so let's now take a look at our journey the journey of the sin ack so we're going to switch direction now we saw what happened going from client to server now let's see what happens going from the server back to the client and we're going to start with the first of two synax that came from wanzero interface of the server server-side steelhead so the um let's see if i can get that to work again there we go so if we if we zoom in to the tcp options basically what when xero is sending back to um the steelhead that the original client steelhead is saying hey i retreat i have received your query and i'm on my way to see if there's other steelheads in the path so this is just an informational to the client-side steelhead hey i'm here i'm probing the path i'll get back to you in a minute so that's that's what this first synack is about and then if we look at okay well what did the web server send back on the lan interface of server steelhead so all the things you might expect hey uh 1460 uh segment size sac permitted here's here's my timestamp since the sim had a timestamp option the web server is obligated to if it supports timestamps to put its timestamp value in the synack and oh by the way oh by the way i'd like to do window scaling at level with a factor of seven now then the uh server-side steelhead on the wan interface notice does something a bit different so now at this point he's sending back in the synack going back towards the client-side steelhead hey i'm your server steelhead i'm closest to the server here's my ip address here's the port that we should set up our control connections on 7800 and notice there's no other options that are being presented back to the client steelhead and that's because these two steelheads are going to talk to each other with optimized algorithms and techniques so continuing along the path again [Music] from the wanzero interface of client steelhead and then the interesting thing about let's see what the synack looks like going back to the client and notice the timestamps are not there sac permitted is there segment size is what you'd expect and window scaling again has been dropped down because the client steelhead knows that i'm locally attached to the client so i don't need that bigger window scaling size so that's the synack journey and there's probably a couple of more questions so i mean because i had these questions why are there two syntax on the wayne interface capture we've talked about that the first one is a probe response saying hey i got your i got your probe i'm continuing to probe the path i'll be back to you in a minute and then why did the server steelhead change the server scaling factor and that's because at the client that questions just say at the client and that's because the client is land attached to the um the client-side steelhead so we've just finished discovery discovery is complete now let's focus in on creating the connection pool for the inner channel so that there's some so that there are some pre-built connections that are available between the two steelheads just in case the client says i want to open more connections we don't really have to do that now across the wand if we have a connection pool created between the two steelheads which is a form of optimizing uh latency because now we don't pay the penalty once these are set up we'd no longer pay a penalty for opening new connections from this client to that to this server which for a web browser uh transaction there would be a lot of connections open so let's look at the connection pool now so here we're looking at the wan capture on and it wouldn't matter which side we're looking at the client-side steelhead and notice that there's a bunch of 7 800 connections that have been opened there's actually 28 according to my count and so these have been set up there in place and now we know what they're for they're part of the connection pool that was created after discovery so uh we've talked about overriding tcp options we've seen what that looks like we've talked about the connection pooling concept between the two steelheads we've seen what that looks like so now let's look a little bit closer at ax spoofing tcp axe moving so in this particular connection and i'm going to zoom in a little bit again in this particular connection notice that the um [Music] the initial round trip time is recorded as 402 milliseconds so that's our kind of stake in the ground that says hey between this client and this server when we did the connection setup there was 402 milliseconds of latency of round-trip time now some of that could be congestion some of that could be other issues like in this particular case it's lan uh sorry lab virtualization unintended consequences um because it really we we know there's 95 milliseconds configured on the wan generator or simulator rather but the total time to actually complete the setup was 402 milliseconds so let's take a look at our round trip times uh sorry round trip time to act for example um we're we're looking at the get uh gets forward slash which is packet number 11 and then notice that the ack for that is in packet number 12 and we have sub millisecond response time it's like 21 microseconds so you can see the round trip time to ack we're acting packet 11 and we did that super quick now notice that who is sending the ack the server side uh the server is sending the ack now it's actually being sent by the client steelhead on behalf of the server so this act comes back quicker which stimulates the client if i can say that stimulates the client to say okay well let's keep let let's let's send some more you know your you your uh request was received if you have more data now is the time to send it like in the case of a post where um or sorry an http continue where now the client may want to send more of the request payload and then we have another example um in this case it's the get icons blank request which is in packet number 14 and then the ack for that is in packet number 15. again it's being sent by the server the server in the data center but in reality it's being sent by the client steelhead and we can infer that because there's no way that the server in the data center could possibly get an act back that quickly so that's our look at act spoofing and now we're going to move into that's it for scenario one and now we're going to move into scenario two which is using steelhead technology on my laptop with a steelhead in the cloud called sas accelerator and that that steelhead in the cloud is specifically set up to accelerate traffic to to put a steelhead close to certain sas applications like office 365 salesforce box etc servicenow and then with steelhead running on our endpoint laptops with a version called client accelerator we can now set up optimized connections between our endpoints and sas applications so in scenario two i've got client accelerator on my laptop in orlando sas accelerator is provisioned specifically for the riverbed tenant in office 365 azure which i believe is someplace on the west coast judging by the the latency and there are cloud steelheads running as part of something called a service cluster behind an azure load balancer and all of that gets set up as part of provisioning you don't have to manage it you don't have to maintain it it's all part of the service for sas accelerator um but at the end of the day there's there there are versions of steelheads in the cloud so we have our test plan uh that there's a test script that was planned out in advance we're basically going to take a powerpoint deck create multiple instances of that deck with different names and we're going to move that from my desktop on my laptop to my onedrive on my laptop which would trigger synchronization up to the cloud we did before i did before and after test runs i grabbed packet captures and i also did some screen video captures of the actual test so i could use the timing of what does it look like from the user perspective in terms of the actions during the test would copy from desktop into onedrive using file explorer and then watching how long does it take to synchronize my local onedrive with my cloud onedrive so that represents an upload to the cloud then another way of doing the upload upload is drag and drop from my uh it says onedrive here what it should say is from my desktop to a browser view of my onedrive in the cloud and what's interesting about that copy is first we upload it and that takes a certain amount of time and then because the cloud onedrive is synchronized with my laptop top we now need to download it to my laptop so that it's so that that file is synchronized with my one my local onedrive on my laptop then some various other activities like editing the powerpoint on my onedrive like duplicating a slide and then seeing how long does it take to then uh synchronize back up to the cloud likewise using sharepoint online or powerpoint online to do that same type of edit on a different instance of the file and then watch how long does it take to synchronize to come back down capturing packets for all the steps so the goal of all of this activity was to demonstrate time savings and the reason there's different file names is because i wanted to show that caching is based on byte patterns not on file name so these are not file objects caching this is byte pattern caching which is more efficient and more flexible because what you're going to see with one version of the cache i can edit with powerpoint on the cloud i can edit with powerpoint locally i can do drag and drop from file explorer i can do drag and drop through a browser and it's all using the same cache content which which is actually pretty interesting and then taking a measure of the time savings so i built my little test script in advance i've learned over the years that you have to at least i do i have to i have to plan out my test otherwise i get started on the test and it goes sideways so this is my checklist of what to do doing a ping in between certain um certain steps so i've got some ping traffic as markers in the uh in the capture files as well so let's take a look at the first test which is when op disabled so i basically turned off client accelerator on my laptop and our topology looks like this i've got my home office with my internet provider i've got 11 meg upload 100 make download at the time of the test i was using a 2.4 gig gigahertz wi-fi ssid and i was capturing in this case i'm capturing from my wi-fi interface i can't do tcp dump from client accelerator because it's turned off so i'm doing the capture from the wi-fi interface so in looking at and and i want to zoom into this we're going to look at the evidence of what is the real round trip time when there's not optimization so let me zoom into this a bit okay so we're going to take a look at um we're going to focus on uh well i didn't i didn't make the frame the frames complete sorry about that we're going to focus on this first segment um ack where the the the the rightmost arrow is i wonder if i can highlight that can i highlight that with this we're going to start with this arrow right here and and notice that it's an ack from the laptop so the laptop my 192 168 address the laptop is saying hey i got your data and i'm ready now for the segment that starts at stream byte 7537 and now the server is going to once it gets that ack it's going to then send down that payload uh the stream bytes starting at 75.37 and if you notice the delta time here between whoops between the ack if i can get this back here come on yeah if you notice the delta time between the ack and when the data arrives is 498 milliseconds so almost a half a second for the act to get back up to to the cloud on the west coast and for the payload to arrive back at my laptop so here's an example and there's tons of examples in this capture i just picked one out you know one because we're we're going to run out of time here shortly and it it illustrates what happens with the high latency between me and orlando and my cloud onedrive which is somewhere on the west coast probably seattle or somewhere in the state of washington so 498 milliseconds round trip time to ack it's a key measure and again there's lots of evidence like that so the next thing i want to look at again this is a not optimized uh capture or test rather so up uploading that 56 meg file from my uh local onedrive up to the cloud so there's another cost though and this kind of gets back to my point about does tcp pay a price for round trip time and here we're looking at a bunch of retransmissions that are all clustered together a lot of duplicate acts and re-transmissions and the there's a cost to these retransmissions in terms of the you know the throttling effect and i don't have pointers in here about which packet refers to which but i went through here and what i found in just this one screenshot is about 350 milliseconds of delay just dealing with this one loss packet and that's because it takes time for one and to tell the other and hey i'm missing something because we have to have that round trip time to inform the other end that hey i'm missing something so there's a round-trip time delay not only with real payload but also with dealing with re-transmission control so now what i want to do is let's turn on optimization and we'll run the script again with optimization turned on and see how things look we've got about uh roughly 10 10 minutes or so to wrap this up so that we can get to q a so here we are i'm in my home office um going through the cloud but now i've got client accelerator turned on and i'm peering with steelheads that are in um in azure and now we've got an optimized connection between my laptop and and some steelhead technology in azure so that we can optimize those connections so here's what we're going to see we want to be on the lookout as we get into these captures we're going to see some phantom tcp connections we'll talk about that we're going to look at some rtt timing and we're going to look at um we're going to look at data coming from a nearby cache and the fact that there's no re-transmissions and faster faster times so the first thing that threw me off um and it was quite a throw-off is when i looked at the land capture there's these connections in there that didn't really make sense i didn't know what they were um it's like these 6300 uh ports it's like what the heck is that where did that come from what is that doing so the first thing that i found is these ports this traffic doesn't it never leaves the laptop it's internal part of when up processing that leaked into the uh into the capture and if you look closer like i sorted by bytes if you look closer you'll notice that these are kind of paired it's like oh hey these two moved 150 mag these two moved 116 meg these two it's like hey wow these look like they must be related and it turns out they are so in order to really analyze this capture the first thing we have to do is get rid of these extraneous 6300 ports which is really easy to do so basically a display filter with tcp port 443 and then creating a new capture file using export selected packets and now i've got a new pcap with 443 only you can see see the name of it there so now that we're looking uh at a cleaned up capture let's take a look at the client hello handshake as an example of how long does it take you know round trip time have we have we improved round trip time so here we have a client hello um that's happening let me zoom in on that we have a client hello a big big packet number 58 2624 immediately followed by a server hello and you can see that in this act right here we're acting 20 26 24 and we're doing it in 15 milliseconds so we know that the the initial round trip time for setup was in this capture here 89 milliseconds and yet we're able to get the uh server to act the client hello within 15 milliseconds so that's telling us that we're actually serving that from a local cache we're not serving that having to go all the way to the server all the way back we're actually serving that that server hello um from the loc from actually from my laptop from client accelerator on my laptop and for ssl traffic what's happening is my laptop is ssl terminating for the client and then we're going optimized across to the the cloud steelhead and then we're getting back into ssl to hand off to office 365. so if we look at another example real quick uh okay if we look at another example um in this case we're looking at um payload coming at going out and then the next uh segment arriving so we it's similar to what we looked at in in one of the earlier examples we've got an act saying hey i'm ready for stream bytes at at this at this point uh 4065 and then what comes next from the server oh yeah it's 4065. how long did that take that took um less than a millisecond so we know that that's being served from a local cache and one last feature slide and then we're almost done and we can get into q a when we talk about cache from a steelhead perspective there's some um patented technology called scalable data reference it's it's actually you know it i don't know all of the technical details about it but it's very very effective and it enables the two steelheads to synchronize their cache and in the um the server side where the data is being served out it's able to signal to the client hey um start to serve out content from your cache at particular addresses and hash points and it's able to do that very very effectively and we saw an example of that where it came out um like within zero milliseconds the next segment showed up uh to the client so very important piece of when optimization technology is caching and and this one is a special one so last item there's no re-transmissions in this optimized connection so if you think about i'm at home i'm doing this test i've got wi-fi on 2.4 gigahertz i honestly i don't know what the other family members were doing at the time but um we i know that i was not the only user of wi-fi at the time so regardless of that we still were able to do this test with no re-transmissions so not only did we get the benefit of ax spoofing and the benefit of local caching and one of the side benefits is no re-transmissions no re-transmit timeouts no fast re-transmissions no duplicate acts the the sender and the receiver don't ever believe there's anything wrong with the network that that we need to throttle back so this is actually a good thing no re-transmissions and then the last the last slide um in terms of this testing that i did i did testing on two different days and what was interesting on the second day like 30 hours later i i ran some of the initial uh scripted tests again following the script and the and apparently the cache was still warm not only on my laptop but also in the cloud and when i look back at the video captures of the speed what we went we went from us we went from 64 seconds to upload this file and then a that that's with a cold cash no cash um to 16 seconds the next day it was still in the cash pretty pretty awesome and then downloading was nine seconds versus 24 seconds so there's some real user productivity savings with this technology but to recap you do need to expect that your captures are going to look unusual if when optimization is in the path we looked at some examples of that um that the testing that i did was pretty very conclusive that with um not only with branch steelheads but also running in my my laptop talking to a cloud version that for me working at home the productivities are are really significant and it gets rid of latency gets rid of re-transmissions and all my typical home wi-fi issues of not enough bandwidth or my ring ring camera's going off and dropping my connection none of that none of that's important or none of that impacts me when i when i've got this turned on so quick summary slide of what the different when up features are we talked about many of these and before we get into q a i just want to please ask you to take a moment to complete the session feedback form i i not only for today but if you're viewing this on demand i believe i believe this form will still be available so please give it a try even if you're if you're doing on demand so that's it thank you very much um for your time and attention i know this is a little bit of an esoteric topic but again it's it's an it's a review of some very strange conditions that when you see them for the first time in wireshark they could be very confusing and i wanted to just help explain what those conditions are and

Channel: SharkFest Wireshark Developer and User Conference

Views: 390

Keywords:

Id: jWUaVbwcSLQ

Length: 64min 55sec (3895 seconds)

Published: Fri Jul 30 2021