Becoming an Expert Troubleshooter and Packet Analyst with Hansang Bae

[Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] all right hello welcome double check my audio it's good welcome to the was that what do I call the packet bomb livestream experience I'm your host Kerry Rogers we have a very exciting show for you today we have for the first time ever a live special guest you'll never guess who it is this is not in the title of the live stream hopefully all the technical there will be no technical issues and it will go great but we're gonna have a chat about you know packets yeah we'll talk about packets will talk about Wireshark we'll talk about networking career whatever else well it'll be fun and we'll certainly have some Q&A at the end if anyone has questions would like to talk I believe I met our guest I want to say it was 2010 at shark fest for the first time which I went into thinking I knew what I was talking about well me what am I gonna learn I'm here to teach these people well I was wrong and I basically kind of had my mind blown at shark fest and Han sang bay was one of the people who did that so we see we've got some familiar faces in the chat we are you know starting a bit earlier in the day at least here on the west coast so I'm glad to see some of our friends across the pond joining the live stream so I think without further ado we're gonna get into it have a chat please everyone put your hands together or hunt song Bay thank you everybody here we are yes please please everyone stop hold your applause please everyone sit down sit down please please sit down you know you know yeah we sure to do yes all right how are you today good beautiful Friday Nick carrion and and it's not general mattis Mad Dog Madison but all my life I wanted my title to be Supreme Commander Allied forces but nobody would give it to me so I named stick Tony fair enough Phaneuf of yes the Supreme Commander of Allied packet forces back of course is not like that there we go so as I was saying you know I think I think I'm wearing the 11 2011 t-shirt I I couldn't find the Tim but I'm almost positive 2010 at Stanford that's round does that sound right yeah he was there a couple of years I think at Stanford beautiful campus coming from Mississippi boy but living in New York City never seen a university campus quite like that you know being intimidated you know a bit not shirring knowing what to expect but I did kind of think because I was like the packet guy at work you know I I was the one that you know coming from a sysadmin background and seeing how we were in tech support manually counting the bytes and you know do packet the connection forwarding protocol some riverbed you know proprietary protocols didn't have a dissected and so I spent a weekend and and wrote one and I was like you know what I were they gonna teach me not quite not quite that but I remember being in your session and just like so many things that maybe I not so many but some things maybe I had kind of learned but I was I could not articulate it in the way that you were doing you know in front of everyone and then just pointing out things in the packets and as you went through your case studies just like one you a constant mind being blown now I may not have I think I didn't speak to you probably until the year which may be in the second one to 11 I went to your made sure to go to your session again and at the end you announced you were joining riverbed and I think I ran up and introduced myself yeah and I think you had somebody from QA there that's right happiness right yeah once I started letting people know that we had done this little dissector where and started maybe looking at other things engineering started popping their head up saying oh well we have one oh this team has one so I kind of hand it off to the QA guy and he pulled everyone's work together into like one unified version so I gotcha yeah so how did you end up because I believe you've been to all of the shark fests is that right I have except for one because Jerry our old boss pulled a last-minute All Hands meeting so and that was right right right right he had one first Europe when I think yeah I remember ant that so I missed that one and then I missed another one because my my I had a voice box issue yeah I miss - I think of all of them so yeah it's very enjoyable and here's breaking news folks Kerry mentioned you know when I go up there and explain things and articulate I just make stuff up so no I'm kidding okay I I think it's scary you're one of these people Jasper - he's on the chat session but there are people who are very technical and they're very good at what they do but they can't teach to say their loss or they fumble they they don't know the words or they don't know how to use analogies to make very complex topics easily understandable bite-size so you can absorb it and I think I think that's kind of unique in a technology field where there's a lot of good people I used to turn propellerhead in in a turn as a term of endearment right but you know challenge ourselves because if you have to teach if you're on stage and you have to teach you really have to know your stuff embarrass yourself so I think it's a great motivator to go very very deep not be superficial don't be that iPad generation go deep to learn it and then practice your public speaking skills by teaching yeah so it's it's win-win-win no I there's been plenty of times where you know I'm pretty sure I understand something until I need to explain it I go oh maybe I don't understand it as well as I thought I did which kind of pushes me to go figure it out for real yeah so that I can so that I can explain it so you so were you how did you come to know about it or get out in the community yeah so I started long before there was Wireshark actually and even before no general sniffer and there was a European or UK company that made a thing called spider analyzer and I was working for campus computing so this is actually kind of you know those seminal moments in your life I had a choice to go work for campus computing for 325 an hour I think it was or they were looking for an engineering student to run a CNC machine for $8 an hour if I remember eight or nine hours which was phenomenal money for students back then little did I know 30 years later I'd be in to CNC but I said no no networking campus computing that's what I want to do I went there and my boss Steve who ran the campus computing said hey you know anything about Novell like no well here's a bunch of disco set it up you know anything about UNIX as it yeah I know UNIX he's like okay go maintain the Usenet servers and the DNS and the bootp yeah and it was just trial by fire you know University student jobs and I don't know if that's still the case but incredible learning experience that was true for me and so huh right and then and then one day I walked into Steve's office and I say hey what's that and I said it's a spider analyzer I said what does that do he said where I'm watching the packets and I said Wow so I started sure you know the things that you always study about right that the the block diagram 16 bytes of source destination port number header and it was it was not literally but almost literally a matrix moment of watching packets fly-by-wire and it just completely sucked me in and and I jumped on it and I learned how to use it and we had an incident about a stolen laptop laptop we were able to track it down using mac address filters and and that was I guess almost 30 years ago and it just sucked me in yeah and that was um and I have Kristin to thank for this Christian who used to work for me she was awesome I II was a network general sniffer not even the Windows version the DOS version I had those keystrokes memorized I could fly through those at 5f1 a pair of met and then I just saw Kristin next to me like how'd you do that and I was like what is that it's like Wireshark I left I got sniffer man I don't need that I just kept like how'd you do that how'd you do that how did you do that never you know I'm obsessed with optimizing everything that I do right I don't want to take like even when I'm in the kitchen making my coffee I don't want to make multiple no I don't say roundtrip save your coffee right I want to make it as fast in the most optimal way and and wireshark did that that sniffer could never do and then and then that was my second rabbit hole right and then from that on it was again it was all rooted in packet analysis protocol analysis but the UI I always said this when two vendors make troubleshooting tools that get out of the Troubleshooters away and I don't impose a workflow on me let me identify the most optimal workflow and adapt to me and that's what exactly what Wireshark did so I fell in love I found ya so you university obviously that's yeah I had a similar experience being a student sysadmin and then working in the IT department was a great experience builder which then you know led to the career in tech support that I that I've had so when you went from there to what was after school what what build your your I know yeah obviously I met you and you were City running on what a performance team where you're in the weeds on packets what was your path like to get there okay so let's talk about that because I get this question fair about right I'm starting out how do I get started and back then the answer was for me the answer was get on Usenet because because while there was a lot of alt die die die Wesley right newsgroups yeah there was an old binary let's not go there yeah yeah but there was comp decom Cisco there was comp decom lands Ethernet and these were where rich Seaford one of the original authors of dicks standard Ethernet standard hung out and answered questions the guy from HP that wrote IP / he hung out there and I ended up spending probably hour and a half two hours on Usenet for about a year just sucking up and what that did was it it was a force multiplier gave me about eight years of experience in troubleshooting just by helping people write troubleshoot because you're getting it you know to the whole world the people who are asking technical cogent questions and but now I don't know we stack exchanged maybe right there's wife's our QA against little repetitive though and so I don't know if there's that experience but you know cut your teeth on the foundation Kerry and I were talking about this earlier don't don't expect that iPad experience of it's easy answers right here you have to understand the underlying technology that means you have to know Ethernet that means you have to read RFC's you have to read rich Seaford's Gigabit Ethernet book for example or you know there's a seminal book on Ethernet itself by Charles Spurgeon that you should read and understand then you have to read about tcp/ip which is a next level up and those are the spike comer by you know the tcp/ip Illustrated and including routing you know that the Doyle that Kennedy's of the world on the routing side radhiya permament of the world they go hand in hand this packet analysis seems to be mostly network domain so get deep in whatever you choose to study and don't be superficial about it and it's muscle memory so you have to do a lot of it so wherever you are just open up a packet trace and see what it looks like because over time you'll know what it's like when you see it it's hard to explain but you know it when you see it right so right yeah I hang around a few places you know there's a the networking subreddit is got a lot of talented people in it okay but I hang out on the the Wireshark one as well just to see if you know occasionally there's something good but mostly it's kids wanting to know how to get IPs of people homework questions answer no not even that good it's for like Call of Duty how do I get the IP of the people in my chat or whatever alright I mean I guess so they can attack I don't know what they want to do to be honest but that's that carry believe it or not back when Duke Nukem was the rage which by the way I have my little see if I can bring it to my little Duke Nukem oh it's that you there and that was that was a chewing bubblegum taken at kicking ass he actually wanted to know how to do networking back then it was with IP XO di drivers and packet drivers because duke nukem ran on IP x came up and then they went with packet drivers so i had to teach him how to set up i PX stack and he got so interested in doing that he became a network administrator hey look at us so hey how about that huh yeah so at its one thing I think about you know you know at a large place like City where you maybe have specialized teams that are responsible for one particular thing you know that there's you know people's work environments run the gamut so some people you know they they don't want to take the time they want to push it off somewhere else or like this is someone else's job you know so in terms of especially when people say is if there a network problem and so many times you know we we look at this and it's not technically a network problem so what are your thoughts on people you know the who owns this these problems TCP problems yes I think so the obviously the people who suffer are the consumers or the developers who get yelled at along with networking but I think as Network folks we own the scene you we own the the connectivity fabric of everything that we do everything that we do in modern life so we have a kind of a responsibility as shepherds to help them troubleshoot and I can't tell you how many times I would get a thing saying hey network is slow please help and then I would reply back and change the subject line and say application is slow I'll help you and a couple of couple of developers caught on to that and called me and said hey I see what you did there and but it's also because you don't know you know we think of networks is this magical thing and it is I mean Carrie you're in San Francisco no I'm in New York and people down espers in Germany and God knows where everybody floods Vlad's here and here we are having a live conversation and it's all because of network yeah of course applications like even Skype but I said that with a little disdain but so Microsoft if you're watching I didn't mean that it's just the UI's little home but so it's it's a fabric that holds everything together and we're the only ones that can see it yeah now it's getting more and more challenging because TLS and encryption and cloud technology will you know there was a question on that I'll get into in a second but it's still at the fundamental level the TCP is how the language of how the fabric works and and so we have a duty of responsibility to help folks where there's either a TCP stack problem that's a system administrators function right that's not application guy but within the application you can set buffer love buffers and whatnot so that it's it's the union of all of that so we have nobody to help yeah no absolutely absolutely agree so you know at your time I've I've certainly heard tons of stories and case studies from shark fest that you've presented right real real live stuff what are when you think back your time in the trenches what are some of the ones that just kind of pop to mind some of those their sons there's so many too I'll see if I can in real time anonymize some of this okay first one is actually don't jump to conclusions because if it's too easy you'll probably get burnt um you know you know and so there's two factors to that one is just took technology wise it may be leading you down the wrong path again experience is what saves you from making that mistake the other one is you know not not being humble but being too defensive with your team obviously it's a funny story I'll tell you we did a study on a vendor I'm going to change the name they're a credit card module used by the consumer bank side and it's a dispute module meaning when customers dispute the charge they use this application to make note of you know back out the charges or what not so the application is called dispute so the company I'll just make up a name Pegasus let's say Pegasus there was a slowness issue so my team and I got engaged I think it was either mallesh or Tom maybe it was sandy who worked on that wrote a report I looked it over makes perfect sense send it off to the developers and I get this from the PM I get this email saying Pegasus no what did I say dispute and Pegasus yeah so the subject says Pegasus disputes finding yeah so I will see seed on it cuz I don't think I was leading on it and I just saw that I would what what are they disputing it's black and white right that's our okay I'm gonna get on this call I'm gonna tell him what's what it's black and white packets don't lie so I get on the call with an attitude because you know it was my team I need to defend them like another call and everybody's are all hot Sun iced it you know thanks for joining us and no problem so what's the dispute I got right down to business yeah kind of chilling to New Yorker and they said yes the dispute I said yeah what what is it is on first thing goes yeah we're here to talk about Pegasus disputes finding and I said I know what is the dispute my team and everybody was quiet and then the PM goes Hansa he goes we're here to talk about the module application Pegasus dispute the finding that your team wrote and I was like what anyway so don't jump to conclusions yeah but I guess the the the good one I was kind of happy was Citibank they had this app you know click image check deposit he was brings with it Simon and six days before go live the team came to me and said nothing works this is too slow something's wrong with a network and I was like ooh so we jumped on it and it was actually a third-party service a b2b connection where their application was not optimized and it was a combination of ODBC ODBC buffer size and also TCP window size so both needed to be adjusted and once we did that the check deposit times were fine and we cut at about six days before go live we found one with the equities global fixed income before they went to algorithmic trading we found a TCP delay they were getting murdered and so the the be use were very very happy about things like that so there's there's a lot of things like that and and others that I can't talk about because if I kill everyone on here so like when you were presented with a problem and you're you've got the data walk us through you know the your basic process I assume there's some commonalities right across whatever the issue is you're going to do some of the same things every time and then that leads you down different paths based on what you found so walk us through what that kind of looks like yeah so I never trust the timestamps of what they tell you so in the military we use we use GMT for a reason because you don't want to start to attack and find out Oh wrong time zone Dan right so that's bad that's bad right time on target it's it's very important that everything is synchronized so I had one instance where someone sent me this trace is that this right here drink this 10 minute was a problem I made a cursory look through my checklist of the usual stuff right delays application size buffer tearing all the things that I usually talk about on shark fest I didn't see anything there's nothing there and I thought what am I missing so it kind of intrigued me because it wasn't a parent right so by now I've seen a lot of problems I can triage it very quickly easy problem you know medium interesting or really really hard I should take some time to look at this and there was none of those I couldn't figure it out and then I finally called Hobbs and said dude I don't see anything here what am I missing describe the problem to me and I realized the the generic word sniffer that we were using at the time was it wasn't said to GMT like it was supposed to be so start with start with the actual time that it was captured so you know that you're troubleshooting the right thing start with that yes right number two is go through your TCP checklist and develop that workflow so if you don't know what that workflow checklist looks like attend the shark fest sessions because a lot of people here you included Carrie and Vlad and the Esper and others they helped you develop that checklist don't eat resolvers use the one that makes sense to you about use it based on all the work that carry myself you know folks like Jasper's Christians of the world have developed and started there so you have a pre-flight checklist airline pilots and fighter pilots use it for a reason and go down it because the one time you skip over something yeah yeah get you and though you're probably it's a problem yeah okay I think some of my actually one of the ones I just did recently I think I did a live stream I think it was a live stream it was um you know someone it was actually Call of Duty he kept getting kicked off the disconnected and and so I went through I went through the whole analysis and I found the problem was a duplicate IP you know some some other device was with sending gratuitous Arps and if you click the easy button in the bottom left corner Wireshark there it is right there in the you know the expert analysis whatever it tells you duplicate IP so I went a long way around but because I just kind of skipped over the basic step I think you like to put was it TCP analysis flags as one of your yeah thank you make a button click it show you everything that's right that's right and there scroll down and absorb it and let your you know there's a lot of benefit to machine learning AI but the one thing that humans are very good at is to looking at slight changes in variation of what you're seeing so so that's what you need to develop it's it's it's comes through practice so let's talk about that you know I I get people you know I'm a public person about pack a packet analysis I encourage people to reach out happy to help maybe it turns into a case study or whatever and I run into people who are truly I imagine most of the people on this stream have an interest in this or interested in learning the art and science of this but he certainly come across people who really are only interested in getting from A to B as fast as possible give me give me the answer I'm not that interested in the the details in the process whereas that's why my videos may be a lot longer than other people's sometimes because to me the process is where all the good stuff is so what do you think helps to develop that or what is what sort of makes someone who's has that skill or who can develop that skill of of learning the process better I think it's it takes a aha moment so if you're starting out you need that one aha moment to get you hooked mm because it's fun and you found something it's it's actually like my other passion woodworking when you make something people go oh where'd you get that I made it and people go you made that because it's not a common skill right yeah packet analysis troubleshooting is the same thing so when you find things people go oh how'd you find that and then when you start talking that I get whatever does you keep right answers so I think that as a troubleshooter you need to have that aha you need to be positive you need to be optimized OCD focused because why troubleshoot if you don't you know you're not gonna troubleshoot if you don't want to optimization right because if it's not fast and you're okay with it they're okay with you would never sell shoot but if it's just a slightly slow you have to start asking why is that slow why did it do that so that aha moment and you have to be addicted to the word why and just keep asking why is that why is that and never accept an answer unless you truly understand it because if you do you're gonna just parrot that one scenario and the minute you that focus is off you'll never know right because you've only did this and you don't really know so if you keep asking why and I'm gonna go back to my and maybe a lot of people here you're when you were taking calculus differential for EA's of the world if you just parrot the teacher and when they're doing it right you're sitting back like that's easy I got it and then when you get your first test extent you're like ooh what is this and you apply it and then don't be that guy calculating your GPA based on getting an F on this exam right yeah there's a lot of people doing they if well I'm gonna funk this one so what's my GPA gonna nosedive to so understand the underlying foundation so that you can build up on that tool set that we talked about earlier yeah yeah I think my general opinion I guess philosophy is that there are just certain types of people that this appeals to and there are certain types of people who it doesn't and that's fine we're all we're all different things interests us you know obviously there's a pretty decent community around who enjoy this type of stuff but when I do come across the folks that you know they're in a tight spot they've got a pain point you're trying to solve but they they just don't care really other than to solve it you know if I can help fine but I much I will spend hours and hours of my free time helping someone who's interested in learning that in the process along the way because if you know if you help well you know the teacher person the fish kind of mentality I guess ya know yep and I call that tech ESP and so I have this thing about good or bad I snap judge people instantly mmm-hmm never wrong I've never had to you know constantly wrong but when it comes to snap judgment especially the tech ESP I've never been wrong I've actually hired people who were on a troubleshooting call didn't know what questions to ask but they immediately went down the right path because I ring sit if so technology you can teach that ESP and ethics you cannot yeah so I actually when I exchange review people for my team or network engineering in general we start with a written question you know 120 questions of none of this stupid bogus tech questions of you know what's the 18th bit of an IP header like okay yeah but and I and I've had those questions where I go why do I need to know that I can just look it up now in an RFC that's why it was written but you know these kind of like logic my troubleshooting questions and you can always find the troubleshooter because they even if they don't know how to solve it they ask the right questions but I would ask for this I would look for this I would look for this I would look for that yeah so that's a techie SP and someone mentioned you know what do you guys study everything related to packets and IP there's it changes every year there's no TLS is right around the corner you should know about TLS and how it works you should know how certificates are work and exchange our certificates that are exchanged and what certificate or key pinning means to Troubleshooters so you should add fiddler to your mmm-hmm-mmm know set to your arsenal of tool set because fiddler gives you the visual represented and the jason's and the graphical part of what you're looking at from the packet so right it used to fiddler because that's they go hand in hand especially in a TLS world the other thing is go back to articles go back to papers go back to RFC's so that you can really really understand it and and every time you read it it's like watching a really really funny comedy movie you all catch a joke that you didn't catch yeah so eating constantly reading things googling things is important and the one other thing I'll say is someone asks you know Microsoft says in this cloud world you don't really need to be technical which is somewhat true so the automation of cloud makes things easier because you don't need the brunt work force to set things up but troubleshooting is not about when things work right your things go wrong it's when they go wrong all hell breaks loose when something goes wrong and then those automation technologies cannot help you it can help you migrate workloads it can help you kill containers and bring them up and you know anthos azure the world kubernetes are the world service measures that the world do a very fine job with that but ultimately to troubleshoot you need somebody who understands it right so that's my when we I was interviewing you know for new tech support engineers Troubleshooters Network type folks yeah you obviously you need to have some base knowledge about answer it the the trivia quit you know the Google answers right as long as you know a little bit that's fine but if you you miss some that's okay but if you do well enough to get a second round interview the key to me was I'm gonna ask some of the same questions and you better get them right this time because now you know what I was looking for the first time this you know you get you're doing a phone screen with someone you don't exactly know what you're gonna ask you you read the Job Description that's what else do you know now you know what they're looking for so you go and you study and if you do better you you nail the ones you missed the first time you get a check for me but if you miss other ones I'm not concerned but going back to the troubleshooting is a huge mistake to only quiz people on their trivia and not do troubleshooting scenarios so if you folks are out there if you're hiring or you're interviewing I'm much more interested in how you handle yourself and how your process we will we want you to think out loud if you don't know the answer that's fine but are you asking the right questions we've had people get so frustrated they storm out interview or people who nail all the details but they couldn't troubleshoot their self you know out of a wet paper about it's right that's right yeah yeah yeah okay so I know people have asked about it why don't we start talking you've posted about talked about TCP course I believe yeah in February it's now wait what time is it so I know there's been things going on in the world as opposed as you you've been building bookshelves or sorry so let's let's hear an update on the plan and where we're at sure so it's actually a question that came up how is the video quality really good what's cuz Kari and I are so good looking well how that helps that's that goes without saying I think no it's it's a camera that we have so I have a Sony camera I think Harry and I we watched the same deal as our video guy who said hey just use this rig if you want to be live-streaming so so it's a the lens or the class as the people in the know the cool kids say the glass is important high bandwidth obviously and and software that carries using to live screen but it took a long time to get this so I'll give you an example so I'm going to switch my video source right now live Gugu fantasy and and you can see me writing right 2020 I got 2020 that's a new word you guys can use it just you make sure you put Erik quote trademark on Sonya right so I in order to so why did I do that video switching so I have this little ATM unit that allows me to switch from four different computers because when I'm going to go into training I have to draw things right things show you a trace here show you a diagram here and maybe have a technical reference here and to do that on one screen is and you record it it's a lot of post work yeah by post I mean editing the video so you don't have the luxury of it being live where whatever happens happens post so I had to learn audition and DaVinci Resolve to get my sound level right right so it turns a thing called Lu FS loudness normalisation 16:23 being the standards and this is that OCD nature of a deal shooter yes I'm gonna do something I might as well do it right so I need the right camera I need the right microphone and then I went down the the rat hole of why their microphone is different and why do they sign sound different and how do i optimize that that doesn't time took me some time to order this 8m unit because apparently the whole world went to whom yes yes broadcasting yes and I couldn't get ahold of this unit and knowing me being optimization freak if it took me five six hours to edit a post knowing there's a much better way to do it I'd be inclined to like say actus sucks right so I had to wait for all that to show up once it showed up I had to learn how to do the the noise canceling in post work for audio that's important how to boost it when it's how to do gating so that took a long time and so I've mastered audition I've mastered the Vinci resolve I got my whole studio set up here I had to build my sound panels to deaden the sound that's very important so they're my woodworking skills came in a little ahead because I was able to make my own panel right so yes it's kind of brilliant the answer you you han seong did you yeah you you went down the rabbit hole and every topic there is about AV production but now you're an expert and you're ready right so what already all right so we're gonna start this Tuesday hey so ECP Tuesday or TCP Thursday so I'll be putting out regular cadence and and and it'll be remember you know yeah spur always makes fun of me because it's like oh this is a beginner course and then sometimes I go to not so beginner topics and I got carried away so I'm gonna pace myself and I'm going to start with the very basics the foundational stuff because what I found Carrie is that especially when you said you know people who just want the answers they miss the forest for the trees it really is that use case where because they don't have the underlying foundation they know something that you taught them oh if I do ABC I get the answer but the minute it's abd right they're lost so I want to build up that foundation so for some of you guys it might be very boring and you're dismissed you don't have to watch it but hopefully I can make it entertaining enough I can help tell a story because remember this is how our brain works I went up a battle there two people remember books and movies that they watched and saw from years ago they don't remember a table or something they had to memorize from a week ago because yeah story it's it's a multi core operation and you remembered into your long-term memory better so hopefully I can tell stories analogies make it funny and I build that foundation so we'll start slow and they will start taking I may have I have thousands of traces that I've kept over the years I may go through every one of them who knows right so depending on how this this training course goes will hit every one of those okay well you heard it here folks next week this thing kicks off and if you don't hear from them then you know you have to just badger him I guess I don't know ready I see uh several comments about the sink yeah so this is the first time we've done this I certainly noticed in my view a little bit of audio sink we'll figure it out for next time okay me that was on me because I used an external microphone not connected to my camera and then the camera to HDMI to Skype there's a little delay in HDMI processing so that's hundred-percent on me next time I you know thought to the camera so HDMI syncs up if I look if I look at this their skype itself I think it's okay I think it's some something about the import into OBS well well I have to do another video for you guys on how we accomplished the technical part of this or that's watching like an old kung fu movie where they're so in the description of this livestream you see you should see Han songs channel just click on that and subscribe to his channel that's where the course is gonna be right on your you try and the only thing I would say is I may show some Dewalt table saw operation you want an optimization there but it's troubleshooting so things aren't working right so it's a Troubleshooters channel you might see and and vr flight simulator stuff but mostly tcp Tuesday and Thursdays will be packet related great absolutely perfect okay so we're about 45 in how about we any if anyone has questions they are topics of discussion that we could tackle in the last little bit here I'll scroll through the chat that we've had so far but please go ahead and drop something in in the chat I think I mentioned on a previous one I've got I've got my stack of books under my desk here and I'm gonna figure out a way to do giveaways on the on a future livestream I did start researching it and like you have to adhere to you know state local and federal contest losses all right yeah so my cowboy I can't I just like make this easy and spin a wheel but we'll see so the the some of the books that han seong has talked about tcp/ip Illustrated etc I'll be giving those away okay and and Kari there's one question actually I think we touched on here with everything being encrypted you're right TLS 1.3 has PFS and other technologies that make you in positive is it you know ethereal if ephemeral keys and whatnot so that's where the Fiddler's part comes in but here's the thing even without looking at the application when you're talking about performance issues there's a lot from TCP that can be gathered all right so TCP will probably get you sixty seventy percent of the way to say I think this system or this conversation pair is causing the overall slowness and and you can actually use size of the TCP payload and TCP headers to help developers say hey I notice a pattern here where you're not sending out more than 64 K so do you have a 64 K buffer in your application right and so those are some of the techniques that I don't know if they're advanced techniques may be somewhere between middle and events but so there's a lot of TCP can help you so think of TCP as a buzz saw to help you get into the right area and then get your scalpel out and that's the Fiddler's of the world yeah to help you get to the final huh I see Simon has joined I met Simon through a case study that he reached out to me and we it was this issue it was a tomcat server download that was poor performance and it was buffer size and buffer size on the application delayed ACK and then some weird Microsoft know if it's a bug or whatever but if you go look at my case studies you'll see one about Tomcat and that's Simon and I went back and forth on that for quite a bit I think I was able to reproduce it in AWS that was a fun one all right what else we got I just wanted a meal a discount from Simon but that never happens it's too late now chole now what else so we talked about yeah encrypted in it for the book solely your I have water and coffee I'm afraid that's all I've got it's still a little early I suppose for anything else so what's this question about cloud networking form of IP or IP am i oh yeah I mean I paint yeah yeah so I think this is one thing that you should study up on is network services header it's going to a lot of orchestration a lot of you know vendors used network services header now there was two different back in not too long ago there was two different standards for network services that are now they combined and there is one unified standard learn how type length values in network services header can be used and it's very much like IP and IP or VX LAN technologies of the world and so again that's another opera stage what do you guys read well nsh our RFP RFC's for example so you know if you don't like reading if you don't like learning you're the wrong or the wrong profession you have a stack of RFC's next to your bed for some bedtime reading you know what I'll say this so this is my little guilty pleasure I'm never more than our arm's length away from this but it's mostly just fiction yeah well so might get away I hear that I hear that GDP performance is fine TCP performance is really poor so you talk about you know people say well I do UDP or I do whatever tests and then the throughput is fine but when we when we do this test with TCP there's a problem what are the common things you think people should look out for this yeah so immediately top of mine is you don't have a bandwidth issue perhaps because UDP is blasting it but look for any kind of retransmission events retransmission time out from multiple losses so TCP really suffers when there are multiple passes and at the same TV window size because it collapses even with the newer stacks so I would look for a retransmission timeouts the other one that I would look at is you know Carrie I think you wrote this there's a new field on their TCP analysis sequence analysis field if you open that up you can see a number of bytes between pushes yes and so and and so if you chart that you might see that that number never gets above 64 128 256 these boundary numbers right 512 and what so chart that out and see if there is a plateauing because that's controlled by the application for the most part and obviously you know you have to worry about TCP window size exhaustion but those are pretty simple to identify yeah and and and probably too deep to go into an app but those are the things that come top of mind all right and you know tune in next week when the TCP course kicks off right yeah I wrote that bit when I was looking at the the case study with Simon I was like okay there's got to be a pattern but it's it's hard to find so I yeah HEC visualisations yeah yeah yeah and I submitted two patches that one got approved you know cuz I'm not a coder I know enough to be dangerous and the other one again trying to find a pattern was how many how many segments is each act acting because it wasn't always consistent it wasn't like two it was it was it was varying and I was trying to find a pattern but that was really difficult to figure out so I wrote a patch for that but it had too many bugs in it that's how they they kicked it back I see and by the way some you know KK you asked yes there are newer stacks that are coming out not all of them take hold because you know the old joke about God created the heavens in seven days because there's no installed base take how many desktops are out there right now it's so it's a it's a it's a hard problem to say oh I have a you know brand new however having said that Google has some of luxury of doing this with quick and whatnot because they owned the Chrome browser so they can modify it from there so so it is important to keep up and you know Vlad actually he's on the chat here did a session a couple years back so go back to retrospective European shark fest sessions he did a session on congestion algorithm multiple stacks and it's you know it it'll be eye-opening about how it reaches class and time and whatnot so look into that but yes you must keep up with the stacks yeah not all of them because not all stacks a lot of them are academic Riley's right they are changing pretty consistently any thoughts on quick okay so I think um there's a part of me that's against it there's part of it because it's not fair but then maybe it's okay I don't know but I don't wanna get cancelled so I can't really say what I want to say okay they kind of know so I but again it these are so number we had era where long fed network was a big problem you can kind of past that now most operating systems have gone past long fan networks now but some of the fundamental TCP problems of applications inability to use it is still there mmm so again build up your portfolio and don't bother with quick if you don't understand what window collapsing means what TCP versus UDP behavior means etc so again build up the foundation and the newer technologies are incremental changes because the fundamentals are still there so that's how I would tangler okay all right how about last one here is any time spent using transom apdu or opinion on its effectiveness no so I don't know which PDU that is for I haven't used transom I might have from a packet perspective but I I'm not familiar with that one so fair enough Andrew SMB please reach out to me I'm currently looking for SMB case studies so reach out we'll talk okay don't forget SMB has two different windows there's a different Reed versus right window yeah and if you do it within office it's different behavior than when you do it from file expose Explorer so be consistent in how you test that yeah that's fun protocol there all right so I hope everyone enjoyed this Hasan thank you so much for giving us your time I'm sure everyone is looking forward to next week so if other than your YouTube channel anywhere people should find you social medias or any linkedin's or yeah no one's litter so just link it LinkedIn would be best email Han sang at gmail you'd be surprised how many people don't contact me asking for help it's okay I mean I can't help everybody obviously but don't feel like yeah he won't I do my best to respond to everything so and and one other thing if anybody's watching this later if you comment on my video if you don't use at the time to ask that question I'm not gonna answer it because some of these traces are years old and you go oh package seven why did that happen and you go and I'm like I don't know I can troubleshoot that so I always give me context about what it is that you saw help me to help you not to be cliche about it and then so just help me to help you yeah exactly exactly so you do not man Thunder okay pal and lived it there we go you heard it here alright everyone thank you for joining I hope you enjoyed this we will I see some people in the chat that I would love to bring on here for a future conversation and yeah this video will I mean it's it's basically recording right now and it'll be on the channel so people can rewatch Hansons link is in the description and we will see you guys next time thank you thank you [Music] [Music] [Music] [Music] [Music]

Channel: PacketBomb

Views: 1,304

Keywords: wireshark, packet analysis, tcp, networking, hansang bae

Id: gRCj7IDJypM

Length: 60min 19sec (3619 seconds)

Published: Fri Jul 10 2020

Reddit Comments

[u/fbm1003]

These are so helpful for those of us looking to get our feet wet. Can’t thank you both enough

[u/karyhead]

Interview I did with Hansang Bae about packet analysis, troubleshooting, networking, career, etc. His youtube TCP/IP course starts next week. He's one of the best, enjoy!