- It's just like when you call up into a call center and you say, "Hey, how you doing? Can
I check that flight?" And they're like, "Please hold." And then you say, "Okay,
I'm just gonna hold. I'm just gonna sit here on
listening to the hold music." And they're like, "You know what? I don't think it's the network. I just don't exactly
know how to prove it." But, we bring up Wireshark and we see these black lines with these red letters and we're like, "Oh
no, the sky is falling. The networks being ripped
apart at the seams." So, let's actually see
what's really going on. (lively upbeat music) - Hi everyone, it's David
Bombal back with Chris. Chris, you've got a
really interesting topic to talk about today. - I do, at least I think it is and that's how to filter
traffic with Wireshark. - Yeah, it's a problem isn't it? Because you've got like... We spoke about it last time. You've got all this traffic and how do you find stuff? So, I think if you... If I remember correctly, you're gonna show us
about Wireshark filters, is that that right?
- Absolutely. So that's really a big
topic that we have to... Especially being Wireshark
users, troubleshooters. It's super easy to end
up with this mountain, this huge, huge haystack full of packets and we gotta dig in and find the needle. So, the faster we can get to that needle, the faster we're gonna be
able to resolve problems, find cyber security issues, all the above, and filters are a big part of that. - Yeah so, take it away Chris. Show us what you've got. - Okay so what I'd like to do David, I'm gonna show you a few filters that not only do I use in
my consulting practice, these are just common
filters that I like to use. But also ones I like to make
sure people are aware of. Sometimes you might go, "Oh, I didn't know I
could filter that way, that's awesome." Right? So, these are things that I've picked up from other analysts, or just found out by tinkering on my own. Really something that I've found too in my training classes, also on my YouTube channel is just teaching people how to build those
filters and remember them. 'Cause let's face it, sometimes the syntax can be
difficult to remember, right? Like what was that? Was it ip.addr. or whatever? Like, how do I build that filter? That's what I wanna talk about today, just some common ones that people use. So, I think-- - Yeah, I've had a lot
of feedback on our videos where people have said, you know, they didn't know how to. They didn't that they could drag like filtering as an example. They didn't know some of the filters that you were showing last time. So, they're looking forward to this. - Yeah awesome, okay. Well good, you can see
my screen all right? - Yep.
- Okay, fantastic. So, the first thing I'm gonna do is just set a regular IP filter, but we're gonna talk about it too. So how do we do that everybody? How do we set just a standard IP filter? If I just wanna filter
in on a single machine. Let's just pick this address right here. 192, 168. I want everything coming
and go to that guy. All right, so let's go up. We're just gonna do ip.addr. If I wanted that station
to be the source, okay? SRC source. If I want it to be the destination, dst. But in this case I want
it to be either one. I don't care if it's going to
or coming from that station, go ahead and show it, ip.addr== And now we can just do 192.168, okay? So boom, I got my filter and down here on the lower left, I take a look at my number
of displayed packets and I can see that my filter worked. I have some things that matched it and now I can start to do a
little bit more tinkering. So that's just a common filter, that's standard, you're gonna use that all the time. Let's go ahead--
- What do they do? Why do they have two equal signs? Is there a reason for that? - So it's the standard
syntax of the way that Wiresharks' gonna be doing its equals. You can actually have three equals there and that means something
slightly different. I don't think I've ever used that one. You can also have things like, okay, ip address does not equal. This is where we could come in and we can put an
exclamation point in there. We could also, with some of the other
filters I'm gonna show you, we can also use a greater than symbol, a less than symbol. So, really just depending on the type of filter that we're looking at, we always need to first give Wireshark. We need to tell it what
field are we focusing in on? And then do I want that field to equal something, to not equal something or to give it a range? This first one, what I like to do? So I really do start off with that. Just, let's just take an
IP address and just see where we can go from there. So I do 192.168.0.10. All right, so from there, now I go, "Okay, I've got the address, but now maybe I wanna fine
tune that a little bit and start thinking about like a protocol or a port number." Well, once I have the address, then I can come here and I can just say, ampersand, ampersand. Say that, or I can use
the English word and. Huh, right there. Either one will allow me to then make an addition to this filter. Now I can start to go,
"Okay, how about..." I can see it right here, TFTP. What if I do, and tftp? Hit enter and I can see my packets on the bottom they disappeared. So now I'm being real specific. I'm saying everything
to and from that device on this specific protocol, go ahead and apply it. That's a basic plain Jane one. You're gonna do that all the time if you're working around in Wireshark. - Can you do or instead
of and, is that right? - Absolutely you can. In fact, why don't we take
just a quick little... I know neither one of us
David are slide people, right? We really aren't. We like to--
- Yep. - We're not mentioning
the PowerPoint slides. But, I am gonna show you
one that is just gonna help us to see a couple things here. Okay, so here's some common filters that you can use with Wireshark. So you saw the syntax for
just a single address, also where we can make it
a source or a destination. Another one that I use a lot is going to be the address range. CC ip.addr== and then you give it the address or you give it the network number. And you see I got my /24 there. So that's how I can
denote an entire subnet. And that /24, going to the CCNP people, CCNA people, learning how those subnet
masks work in the bit level, filtering what we can do. We can give it any range. I could say 10.0.0.0/8 So I just want to see
all the 10 addresses. So that's a nice one to use when we don't. We're not really specific on
a specific address on its own. Maybe I want these 10 addresses, any of those will do. So I can back up that subnet mask and I can adjust it accordingly. A couple of other ones
that I use quite a bit are port numbers. So we have... Here have tcp.port==80 or within the TCP protocol
I do a lot of flag filters. So tcp.flags.syn. These are some common ones, but as we're learning
how filters go together, a nice thing about Wireshark is that it will help you. So if you mess up a filter
it's going to show red there. So, I'm back to your question about equals or one equal sign. That's not the proper
syntax for our filter. So if I miss... If I just put in one equals, it's gonna show red. It's gonna say, "Hey Chris, you missed something here. Go review this, I didn't like it." And then when I get that syntax proper, then it'll show for me. Now to the operators. Here's some of the operators
we need to use, okay? So, you can use either the
top row or the middle row. So either the two equal symbols or the English eq, or we can use... Okay, if I wanna say not, I can see the exclamation
point or the word not. The or, so it's a double
pipe or the word or. Two ampersands or the word and. Greater than symbol, gt or less than lt. Notice below that I have
a filter here in that set. So ip.addr and notice I say eq,
so that's the English, and then the IP, but then I use the symbols && tcp. I can use either one, it's all up to you what's your style? Some people like to use all the English on the bottom, some people like to use just symbols. Wireshark doesn't care. I kind of use a combo, I get my own style going and then I just figure
out and see what happens. All right, so those are
some common filters. So, let's go ahead and
set some filters and use the Wireshark interface
to help us to build it because there might be some
people out there thinking, "Okay, this is great
but I still don't know exactly how to build these filters." Let's go back into Wireshark. - So Chris, is there a way that people can get hold of that like PowerPoint? 'Cause it's, it's a nice summary. - Oh for sure, absolutely. It's actually a part of a
course that I have coming up-- - Oh great.
- And that's gonna be. It's gonna be all about
getting started with Wireshark. And those slides came from the section about Wireshark filtering and how we can build them and even save them into profiles and make more use of display filters. So coming to a Udemy near you. - So depending on when
you're watching this, I'll have the link below and you can get them as part
of Chris's Udemy course. Chris that's great.
- Awesome. Okay, good. Now let's go ahead and
go back into our PCAP. Let's make this practical. Like down to earth. How I really set filters. Now I'm gonna be honest,
I'm not much of a typer. I don't like hammering
them out, typing 'em out. It just too much keystrokes and too much opportunity
for me to mess it up. So, what I like to do is I often will use some of the other features
of Wireshark first. So let me show what I mean. I'll come up to statistics for example, and come down to conversations and then I'll take a
look at TCP for example. Now a lot of times people
they'll send me a PCAP and they say, "Hey, look at
this weird file transfer," or, "Check out this strange whatever." So what I can do from a high level, is I can take a look
here first and I can say, "All right, what ip's do I have? What is talking to what? What is it likely that they're
wanting me to focus in on?" I can sort the bytes column. I can jump back to the top here. Hey, this looks like a
juicy little conversation. 65 kilobytes, maybe
that's the one they want. Let's right click there, apply as filter, selected, A to B, B to A, click. Boom, close, there's my filter. I just saved myself
from needing to type out ip.addr== && tcp.port,
you get the picture. It took the four two pull
out of the statistics and it put it up here for me. There's my filter. Now this is a pop conversation, I can start to comb through and now use some other
aspects of Wireshark to do some further digging, right? So right click filtering. Just right click, it's easier. I can even do that from here. I can just show you a little bit further. If I right click a packet, I can come down to conversation filter and now I give Wireshark the
basis for the filter, okay? So, do I want the basis
to be the Mac addresses? So is this gonna be
between these two Macs, I wanna see everything going on? Is it between the IP's? So if I have several
different TCP conversations on top of one IP conversation, I could say, "Hey, just
show me the IP addresses." Or if I want to be specific
to the TCP FourTuple? Which is the two IPS and the two ports, that's where I come down to TCP and I could have set
the same thing that way. Also a very common filter. Those are things I think I do. I would almost say this is the one that
I use the most, right? TCP conversation filters like that. Now, there's a few other ones that I also would
suggest people know about and even save as a part of their profile. Okay and I'm gonna show you one of mine. If I come up here to... Over here on the top right, no doubt you noticed that there's quite a bit of buttons
that I have up here. So what I can do is I can
start to save filters. When I create a filter that I like, I can go, "You know what? I never wanna have to type that again." I wanna come up here to
plus and I can name it and then I can add it
to my filter buttons. Now I'm gonna cancel this one, 'cause that's an unlikely
one that I need again. But if I come here to TCP connection, this is actually a menu that I created, I'm gonna show you how to do that. And then I have all the buttons that are TCP connection filters under it. So you can see some of the ones
that I have in this profile. Bad TCP, that's gonna show me
retransmissions, Dup ACK's. TCP ACKed unseen segment. You notice the filter
that I use to get that? It's called tcp.analysis.flags && !tcp.analysis.window_update. So Wireshark natively, when it sees TCP thing happening, like something bad, like a retransmission or a Dup ACK or anything like you see here, it's gonna flag it as a TCP analysis flag. It's gonna say, "Hey David,
there's something wrong here. You should go check this out." That files under TCP analysis flags. Now for me, window updates aren't necessarily a
problem in of of themselves in all cases. All that means a TCP
window update just means, "Hey David we're connected, so you're sending me
data and I have room for 10,000 more bytes. Oh wait, I just cleared out some data. I've got room for 12,000
bytes now or 13." Right? If I update my window, then that means that I'm just telling you
about more window size. That is a TCP window update. Now, for me that's not gonna be as, like, like a critical problem. In fact in Wireshark, we don't even paint it black and red. It's just a... And in fact, let me see it real quick. - You're looking at all flags, analysis flags where
something's gone wrong, except that one, is that right? - Correct.
- Yeah. - Like here I have one. You see this CCP window update? Let's actually investigate it real quick. Let's go on a little side
tangent chat, shall we? Now this station it ACKed data above. So server was sending, looks like an email down to this station. Station said, "Thank you. Thanks for all that stuff, Mr. Server." I just clicked this packet and you can see on packet
239 there's my check mark. That means packet 240 is acknowledging everything up to 239, I'm good to there. Well right after that, the client sends a TCP window update. So let's peek real quick at what that, what that actually means, what it was doing. If I go to the packet previous, I'm gonna come here to the TCP. Gonna come down here to window and you can see this is 58280. So that means it's still
has room for 58K more bytes. So if I take a look at
the next packet down, the window size just went up to 64. So his buffer grew by a little bit or rather his buffer was
probably cleared out. So the application had
data processed out of it and he's now got room to accept more data. So that's why a window update, I don't usually focus too much in on that. If I come back to bad TCP, that's why I wanna see all TCP events that are retransmissions, Dup ACK's, things like that. So, that's what that
filter will catch for me. Okay, cool. So let's do this, you know we can probably
even cut and paste this one down into the notes down below, but really all we do is we
just take pop that guy in and then all you gotta do
is come over here to plus. And then, to add it as a button, we could say TCP flags, all right? That alone as a button will
be added to the top right. In fact, I'm gonna go ahead and do that. I'm just gonna say, okay. And you notice that we
have TCP flags over here? - Yep.
- Great. That just means, if I remove this filter and I come back over here TCP flags. Wonderful, I never have
to type that in again. Now I know about any TCP
events, retransmissions. By the way, in the real world David, if I see retransmissions
and a bunch of Dup ACKs' and things like that, I start to hunt after packet loss. That means that TCP is having a hard time getting data across. If I come up to TCP flags and there's nothing here and it's clean and there's no retransmissions, Dup ACK's, anything like that, then guess what? I say, "Hey network guys, we're
not dropping packets, cool." It might not be the network. - Yeah.
- We can start take... Looking at latency and
other things like that. But anyway, so I start
to get real busy up here. So you know what I wanna do, I'm just gonna right click that TCP flags and I'm just gonna say edit it. And what I'm gonna do is
in front of this label, I'm gonna come over here and I'm just gonna say TCP con, okay? 'Cause that's the name
of my menu up there. Slash slash. Now, Wireshark will nest TCP flags under the TCP con menu. And then I say, okay. So now I've doubled, I had one that said bad
TCP and I have TCP flags. I'll remove that later. But anyway, this shows you how people can start to nest these filters. And if you notice some of
the ones that I have up here, another common one that
I use is TCP reset. I wanna know when one side or
the other aborts a connection. That's important to me. In this case, I know 'cause I captured
part of this PCAP, I know that I was doing
a port scan with Nmap. So, Nmap was just hitting
on a lot of different ports that were closed, right? So those ports are
coming back saying reset. But if I'm in an application environment and something that I'll listen for from my customers if they say, "Chris, it was really weird. We tried to use this application and it just suddenly shuts down." Or, "I try to connect to that TLS server and it just won't connect." A lot of times things
I'll look for are resets. That's a nice filter to have
and not have to type it out. I just go to reset and there we go. All you gotta do everybody, tcp.flags.reset==1. Let's go ahead and save that and then you can nest it if you choose to. Now where do I get that at reset from? Let me go ahead and talk
about that for just a moment. In the TCP header, if I come down to flags. If I expand that out, the reset bit, if you look down in the
lower left down here now, I got tcp.flags.reset. So here's another feature of Wireshark. Look, I'm not a filtering genius. I'm not, I'm not a filtering genius. - (chuckling) That's what you say. - I don't need to be, Wireshark tells me all of
the syntax right down there. So if I ever have anything
in the packet at all that I wanna filter for, if you see it here, you can filter for it. Any of this. Calculated window size. Look at that, tcp.window_size. Even if--
- It just shows again Chris, you have to, you can drag it to the top or can you just right click or something? How do you get it to that, into the top part of it? - Oh, how do I apply the filter up-- - Yeah.
- as a display filter? - Okay, great question.
- Yeah. - So, let's just say I wanted to filter on all packets that have a
calculated window size of zero. If that's what I saw and I'm like, "Yep, that's what I want." I can right click it and pop it up to the display bar and I can just say... Now it's gonna ask me, it's gonna say, "Hey, do you
want to replace the filter that you already have? Do you want to end the
filter that you already have? So do you want the filter and this, Or do you want the filter or this? Or do you want the filter and not this?" Right, so it gives me some options. Let's go ahead and say selected, just to replace it. And it automatically will
apply that filter for me. So these are some of the features. Honestly, sometimes in some of my courses, people come in and they say, "You know, it's like, the
filtering, the filtering, it's just so hard to remember and know what filters are useful." We'll keep it of mind that Wireshark gives you these little hints. You always can find the
syntax for any field in the lower left. And then like you just did David, you can take an example of that field that you wanna filter for and drag it up, and automatically that'll apply. So you can see all the
packets that meet that filter. - I think the problem is like, how do you know which filter
to use for which problem? And that's where your
experience really comes in. It's like you... So what was your example? When people can't connect to something, then you look for resets is that right? - Yeah, that might be. Especially if it was like an abrupt thing. If they tell me like, either disconnected or could not connect. Something's blocking connection. My mind starts to go to down, "Okay, reset or..." If you tell me it was
slow in connecting, okay? So let's take a look at the
handshake and let's see, do we get retransmissions? You know, are we starting
to see that kind of thing? I have a couple other traces
queued up for you David, but those are the general ideas. - The great thing here is
you taking your experience. And I was just discussing this earlier with someone else today. You taking what you don't get in a book and telling us when to use it. And that's a hard thing to do, you know. Often books just show you the interface and they show you the switches, but they don't show you when it's valuable to use the switches. - Yeah and for me too, like even in my trainings, if I don't use something, then I just tell you, "Forget it act like it
doesn't exist, move on." Because I can be honest in that I've never practically used that aspect. Or I just don't use that type of thing. - Things you do this day in a day out, so if you don't use it. I mean, there might be some edge cases, but for most people it's
not gonna be that useful. - Or in some cases too, something else I run into, a lot of people are in a certain place they're in one network
all the time, right? So, they might have that one application that's always running on port 13,470 and you know and there
might be some filters we can build for them there. But you know, for me I'm
jumping into different networks all the time, different applications, different environments,
cloud, virtual, wireless, all kind of stuff. I just use... That's what I wanted the people to get from this session that we're having today. Are here's some common filters themselves that I practically use. And also where I get it from. I just use Wireshark to
help me build a cheat sheet. There's a few more I'd like to show you, what do you think? - Yeah, it'd be great. I mean it's... Yeah, if you can give us the rationale of why the filter is used
and then the filter itself that's gold. - Okay, let's do this. I got one other type of filter that I wanna make sure
everybody's aware of. And so I showed you some basic ones. We looked at conversation, we looked at how to do a single IP filter, you know between two endpoints. We looked at a port number. But there's some special
filters too that are used, I use personally quite
a bit with Wireshark and that's gonna be these three. So we have the contains, matches, which would basically be the regex or the pro compatible expressions. And then we have the in, which is a membership operator, okay? So, (mumbling) what's all that mean? First of all, a lot of times when we're
talking about a string, if we're looking for a
string of actual text, okay? So let me flip back to my Wireshark here. All right, so if we're looking for a word like an actual, if you see here in my Wireshark here. NTLMS, let's just say
that was something that I was interested in looking for or here we go, desktop, right? So there's sometimes there's clear text in packets these days and DNS it's still there, there's a handful of other protocols, even with TLS. In the client hello, we still have access to the website that we're trying to get to. So sometimes I can use that as a filter to try to trigger off of. But first I gotta be able to
find out where that syntax is or where that string is in the PCAP. So to do that what I can do is I can come up here to my display filter and this is where I can
start to apply this. I'm just gonna do frame contains. And let's use this as
an example, NTLMS okay? Now contains is very specific. It's gonna say, "Okay,
anywhere in the frame. So starting from ethernet, any packet that contains
this string specifically, go ahead and show it." Okay, I only have three appear. Now, the reason why I like frame contains, and I've seen some people
come in they say IP contains or TCP contains or DNS contains. Well, I don't wanna be overly specific. What if I want that string and okay, it's likely IP that I'm using, but what if it's IPV6? Or what if it's another type of protocol? Well regardless, I don't care what's contained within ethernet. If you see these characters, show the frame. And that's how we can
look for a specific word. If I'm just looking for-- - You can capture it on another old link, it would still, it could still capture. It's not just ethernet, is it? It's just like any Layer 2, is that right? - Whatever's in that frame, yeah. And I'm gonna be honest with you, it's been a while since I've
tinkered with not ethernet too. So--
- Exactly. - I'd have to test that. I'd have to test that. But yeah, if it can be decoded by-- - I'm showing my age here. Sorry, go on. - (laughing) I'm sorry David, we can't talk about token ring today. I'm sorry.
(both laughing) But if I wanna be specific to like, let's just say DNS or HTTP or... This is where I could start to say, Okay, I don't care about all the DNS. I wanna know, do I have HTTP contains? So now looked at that
protocol contains this string. That's one that I use as well. Now, let's flip this on
its head a little bit. What if I wanted that
string but I don't know if. Man I don't know if it's capital N, if it's lowercase TLMS, I don't know if it's uppercase, lowercase? This is where I can come in
and I can do frame matches. See now, the... Now it doesn't matter as far as the, it's not case sensitive, right? So it matches-- - So, contains is case sensitive yeah? Sorry.
- Correct yeah. Contains is case sensitive matches is not, because matches gets us into more of a free flowing Regex filter
expression afterward. Okay, so this is honestly
one that I use quite a bit practically when I'm looking
for a specific type of string. And a lot of times, just to give you a practical example Dave. So a couple of... I can't remember now. There was, I think it was
the last one that we did. The Nmap one? We were talking about how to trigger on specific areas within a PCAP. If you have like a lot of
different PCAP's, right? And I wanna find where the
person hit this website, or they were going out and
talking to this server, this is where what I can do is I can go in and I can set a frame matches and then I can find where
they hit that website. Now I have a bit of a context
I can widen around that and I narrow down to a
narrower part of traffic. That's the whole idea. I just wanna narrow down
the amount of packets that I'm dealing with to try to find that, needle in the haystack a little quicker. Now, let me show you while we're still trucking
along with our filters. A lot of times I might be
tinkering with an application that has more than one port. So say for example, how about a web application
that supports both HTTP and HTTPS? So here I can come in and I can say, okay, tcp.port==80 but I also want or tcp.port==443, Okay, so that would give
me both port 80, port 443. But it's also possible that
they're using some other. Some other nonstandard port for web. So what if they're using port 8,000? let me just do or tcp.port==8,000. You see how I'm using a lot
of these same statements? I'm like TCP port this. TCP port this, TCP port this, this is where instead what I can do is save myself some typing. Okay so, here on this... For this filter, so instead of having this,
this, or this or this or, what I can do is I can come
in here and I can say in, tcp.port in {80,443,8000]. Now I can give it a bunch
of different parameters. I want this port or this port
or this port or this port. I can even give it a range, ..8004. So now, this will show
all applications or ports that match any of these or are in the range
between 8,000 and 8,004. Honestly, this is one that
I use quite a bit too. - It's curly braces, yeah? - Yes, exactly on each side. - I notice once something when you, on the previous one
where you did the equals. You didn't... On some you had space and
some you didn't have space. So when it... Like when it's a operator like equals or something like that you don't, the spacing isn't important, but it's important
between the words, yeah? - That is correct, yes. So you were saying okay-- - I gotcha.
- like 80. I could have that or
I can have the spaces. That's totally a style thing. Wireshark does not matter
or it doesn't care. Okay, let's back up a second. We've learned how to do an IP address, we've learned how to do a conversation. We've learned a port number. We've learned how to look
for a specific string, like a text value that's
anywhere in a packet. Now let's use some of
the features of Wireshark to start to find delays, lag. Like, can I build a filter
that would show me slow? I mean, does that sound useful to you? - Oh yeah, that doesn't sound easy though. - Yeah okay. Well, thankfully we got Wireshark. So, I'm gonna show you there's several different time filters that Wireshark has built into it. You just gotta know that they exist and then learn how to filter for them. So here on this PCAP, first thing I'm gonna do just... Let's just go ahead and
set our port 80 filter. In fact, you know what I'll do instead of port 80, I'm actually gonna come
back here and do HTTP. Now what's the difference? Well you notice that a lot of my green lines and red resets
and all that went away when I did HTTP. Now I've got the pertinent payload part. Ooh, three P's in a row. (David chuckling) This is is the actual
application communicating. When I say TCP port 80, I'm asking for the handshakes, the ACK's, the resets, the FYNs all
the underlying TCP chatter, as well as what's being carried
within that conversation. When I say HTTP I'm like, "Okay, get rid of the transport layer. I don't need that handshake, I don't need all that stuff. Just gimme the actual payload. The gets, the responses,
the posts and so on." And that's what we can see over here. If I say HTTP, this is
where I can come in. And you notice here that
Wireshark is saying, "Okay Chris, this packet here 1122, goes with this packet 1166." You see those arrows? Now I have a way to measure response time, but Wireshark helps me out. If I take a look at this second packet, let me look at the response. And then I can come down into
my HTTP part of my packet. If I expand HTTP, you see time since request right there. Now in those braces... In those brackets right there, anytime Wireshark presents
information that's in those that is a Wireshark derived value. This isn't actually a
part of the HTTP protocol. This is Wireshark doing you a solid, Wireshark's like, "Hey, just so you know, that was 59 milliseconds so... All right, now I'm out." Right, but the cool thing, check out this thing on the lower left, http.time, I can now filter on
that so check this out. HTTP dot time. Let's filter on that. - And this is, this is your local machine time isn't it? So it's like when it
captured the first packet and then when it saw the
response, is that right? - This is from whatever captured it, correct actually-- - The capturing device, yeah. - Yeah, whatever capture device. When this packet came in
it got a timestamp, boom. And now my machine is representing it, according to that timestamp and also considering my... If I have time of day like I have here, it's also considering my time zone, unless I have it set up with UTC, but that's a good question. - Yeah, so in other words it's like you did the IOTA thing last time in our other video. You used like a little device to capture a whole bunch of traffic at Y speed. And then you take that off and then you... That you could have
captured that say in the UK or someone sent you a
capture file from the UK. And then when you open up in Wireshark, it just changes the time zone. But the timing is based on that device that captured it in the UK, correct? - Correct. It's well... Actually technically it's based on seconds since 1970, January 1st. (both laughung) - But I mean the time is based on, how do I say the... Like the delays and
stuff between the packets is based on that device
that did the capture, which could be in the UK, but you were looking
at it in the US, yeah? - Good man, good man. Yeah, so the time and place... When I capture, I'm bringing traffic in
and I give it a timestamp and that timestamp is universal. If I then... So if I'm capturing this, if I'm looking at it in... If I'm in the United States, for example, if I'm capturing at 10:00 AM and I say, "Hey David, check this out." And you're in New York
and I email it to you, it's gonna look like it
was captured at 1:00 PM. - Okay, so it adjusts it--
- To you. If you have time of day here, not UTC, if you wanted to see what I saw, then you would come up to view, time display format UTC. - Yeah, so that's important
because in other words, if you start sending
stuff across time zones, you gotta remember to set it to the time zone of the
place that did the capture. Otherwise, it messes with
your brain, doesn't it? Because I could send
you a capture and like, it could be the following day. Or you send me a capture. You could send it in your evening and to me it's the next day. Yeah, that kind of thing.
- Yeah. - Yeah.
- So it gets a universal timestamp
and that's a great point. You know, sometimes I'll
be teaching my classes and I have people from India, from the UK. I've got people from South Africa and from the United States, all in my Zoom. And I say, "All right everybody,
let's open up a PCAP." And if everyone has time of day, everyone's gonna see something different. - And that's default, yeah? - The default is actually seconds since beginning of capture. That's where we start. Start a stopwatch and it shows us when a packet came in. But if I wanna see for example, what time of day did something come in? That's where I can come into
view, time display format, time of day or if I would just want
everybody to be on UTC. So what I'll do is I'll tell
the students in my course, "All right everybody, go UTC so we're on the same time reference." - Yeah, otherwise it gets confusing yeah? - Oh yeah, yeah. And that was a good point, good catch. But this time, HTTP time is not telling
us the time of day, what it's telling us, If I come down here
it's time since request. So this is, I start a stopwatch on the request and I stop a stopwatch on the response. So this is an HTTP response. So let's go ahead and do this. Let me, Okay I've already
brought HTTP time up here, but what I wanna do, I'm actually gonna right click this and I'm gonna apply this guy as a column, just to take a look at it. And here I can see for this entire PCAP, how long did it take for those responses to come back from the server. So this is application response time. Wireshark's already measuring this for me. So let's just say that
I'm in a data center and typical application response
time from a web server is, let's just take a look at this. Okay, 25 milliseconds or less, okay? I expect to see 25 milliseconds or less. And I want Wireshark to tell me, are any of them longer than
25 milliseconds or slower. So I can take HTTP time and I could say, show me HTTP time that
is greater than .025. That's 25 milliseconds, enter. I've got two that were
slower than 25 milliseconds. I can now save that as a button and I can call that slow HTTP. Now you might be thinking
25 milliseconds, big deal. 25 milliseconds, that's nothing. Well, that's why I've got
another trace file for you. - I just wanted, so
before you go there Chris, can you like real world, which time do you typically use? Like in your class you
said, "Everyone to UTC." That makes sense because
you've got obviuosly students in different time zones and it would be like really confusing to try and get people to
look through the PCAP's, but like, when you're doing this, do you, I mean the person
just sends you the PCAP and then what time do
you use to keep sanity? - Depending on the type of application, one that I use quite a bit
is gonna be TCP Delta time. And I'm building to show you that one. - Okay, yeah.
- This is the easiest way to show you with HTTP, but that's a great question. If I start to see long delays coming back from servers
that are yeah more than, I start to think in terms of what would. What would a human being
actually observe, right? You and I don't really know what 0.025 is, but we do know what one second is. We start to know a lot if
we experience one second, a lot of times. So I think, you know, about half a second. If I start to see
anything longer than that, but if everything around
it is 50 milliseconds, half a second is 10 times longer. So that's where for me, I'll take a filter like this, and then I'll just be up here and I'll start to play
with these numbers, right? I'll start to adjust it and I'll go, "Hmm, You know, let's take a look at, Okay, is anything a hundred milliseconds? No, okay all right." So, that's where I can start to move these filter numbers around to fit into that troubleshooting
exercises that I'm up to. But are you ready to see one? Okay, what's the standard
complaint as network engineers? You and I are network engineers. What do we hear all the time? - Yeah, network's too slow. - Yeah, exactly okay. - So let me bring this up, this next one-- - It's always our fault, Chris. - (chuckles) I know, right? And then who do... Yeah, exactly. We are often-- - Guilty until proven innocent, yeah? - So are you ready to see one
where the network was slow? All right everybody, join me
on a consulting opportunity. Is everybody ready?
(David chuckling) - Okay.
- How much do we get paid? - You're gonna be the... Oh man, you get paid--
- Nothing. - Right, nothing. But, you're gonna have a good
time with me and David Bombal. I mean, that's... Is that not reason enough to stick around? - Definitely.
- Okay. You're the consultant and your client or your customer said, "Hey, we're getting just beat
up by the application people." They're like, "Boom, boom, boom, boom." Saying, "Your network's
terrible. Your network's awful. Blah, blah, blah." And they're like, "You know what? I don't think it's the network. I just don't exactly know how to prove it. But we bring up Wireshark and we see these black lines with these
red letters and we're like, 'Oh no, the sky is falling, the networks' being ripped
apart at the seams." So, let's actually see
what's really going on. So this was a PCAP and the complaint was
that the network was slow. And of course, they put out
a bid to upgrade everything. To go from one gig to 10 gig and go from this ISP to the other ISP and upgrade their bandwidth, and upgrade and upgrade and upgrade. 'Cause what do you do
when you have a problem? You just throw a bunch of
money at it with no facts. - 'Cause this is from a real
client yeah that you have? - Yeah absolutely, yeah. And this is used with
permission and as well, if you notice over here
packet size limited, what I do is I can cut the payload out. So all you see is the headers. Actually, it's a good time
to mention this to people. If you see down here on the bottom, like look at this packet, you see it says 1514 bytes on wire, but 90 bites is captured? So what I did is I used a tool from a buddy of mine named Jasper, he's an amazing analyst. He wrote an application
called Trace Wrangler. Definitely gotta link that one down below. But what it does is it allows you to change addresses and cut out payloads. It just anonymizes a PCAP. Yeah, it's a great tool. And thanks Jasper, you're awesome. He's a fantastic analyst too, okay. So, SYN, SYN ACK, ACK. What have we learned
in those three packets? What's our network round trip time? There's our 97 milliseconds, right? All right, so SYN, SYN ACK, ACK, our network round trip
time is 97 milliseconds. All right, I'm gonna store that away. What have I learned so far? I got to the server, he responded, he's got room for a new connection. I'm saying great, thanks. So my three-way handshake completed. By the way, I'm gonna come up here to view time display format, seconds since beginning a capture. Just to, just kind of start the stopwatch. All right, handshake is successful. Client sends get to server, "Hey, give me this." And there was so much data. There was a cookie and stuff in that, in that request. It was so big that it actually
spanned more than one packet. So we filled one that was 1460 and then we had some residual traffic that went on the next packet, all right? So two packet request, these just get fired off. So far right now, let me ask you David, if you started an application and you were 104 milliseconds in, would you be punching your screen yet? Is it slow?
- Mm-hmm. (chuckles) Maybe. - Probably.
- Yeah right. Yeah, maybe. - Depends. It depends if I'm at... It depends where I am in the world, yeah. - Yeah right, okay. That's that's a good, well said. But usually, as humans 140 milliseconds, we're not, we're not too punchy just yet. We're pretty expectant, but that's still pretty quick. Packect six. I know my packets got
there because packet six, if we look down into our sequences and acknowledgement numbers. By the way, tune in for
TCP deep dive part two, where we're gonna be going deeper into the sequence number
and acknowledgement number, ladies and gentlemen. Coming down to acknowledgement number, my get got there received. It's just like when you
call up into a call center and you say, "Hey, how you doing? Can I check that flight?" And they're like, "Please hold." And then you say, "Okay,
I'm just gonna hold. I'm just gonna sit here
listening to the hold music." So 45 seconds later-- - Oh wow, I didn't see 45 seconds yeah. Okay, that's not too. That's not too long. (laughing) - Right? 45 Seconds later. Is it the server or the client? If I had check up here, clients start connections. They initiate the SYNs. The client is the one. 45 seconds later is like, "Um cool, server we still good? Did you disappear?" Server comes back 99 milliseconds later, which is the network round trip time. You see 99 milliseconds, 106
milliseconds, 97 milliseconds. Those are things that my
eyes look for everybody. When I'm troubleshooting I wanna see, do I see any drift? Do I see a big drift
in round trip latency? Does this go up to 500
milliseconds or 300 milliseconds? No, it's right around
a hundred milliseconds. The network latency doesn't change. So, my stuff's getting there. ACK's are getting back. Servers like, "Please hold." And I'm like, "Urgh." Okay great, I wait another
whopping 45 seconds. See what's happening here, the TCP stack on the
client wants to shut down. If I perceive that
you're gone for too long, like, Hey look, I got
other stuff to get to. I gotta free this port up. So if we're not connected anymore, I gotta move on. That's why it's gonna
try to keep it alive. Like it's gonna say, "Hey, keep alive." And if you don't respond
to this keep alive, then I know, okay I can do a reset. You disappeared, you left the building. But your stack comes back and
says, "We're still going." That means the applications haven't let go of that TCP connection yet. That state is still being maintained. Packet nine. We do it again we say, "Keep alive?" Server comes back, "Yep, I'm still here." And check this out. Just when I saw this, 18 seconds after the second keep alive, the server comes back saying, "Here we go. Okay, here's your stuff?" 1518 bites. - So what is that like a minute later? Over a minute later they got
their piece of data, yeah? - Well, let's go down and
see time since request. You see time since request? - There you go, nicely. Yeah. - We waited 108 seconds for this application to begin responding. TCP was just doing its job, keeping the connection open. The network could have done nothing else. So you're talking about light speed network round trip time
of a hundred milliseconds. There's zero amount of
upgrading on the network that would've improved this. - Tell me Chris, how much money were they gonna spend before you got involved? Or was it just--
- I just-- - that they got involved? - Look, I just go over and ask them. I just pull my pocket
out and I'm just like, "Could you just put the
money here instead." 'Cause that's more than I'm gonna make on this consulting thing anyway but. No, I've seen... Honestly I've seen people. I don't even like to say it, but I've seen one customer dump a million dollars into
fixing something like this. 'Cause once you start
talking big bucks upgrades... And you know and then the
sales guys get involved and it's like, "Well you know, you're at cap 5E you should go to six or you're at six and you
should go to six whatever." You know, it's like... Then you start upgrading
things that frankly won't have any impact on the performance. - So, it's just nice to have war stories. So you've got involved
in situations like this where we're talking like a million dollars or whatever crazy money. And you basically saved
the business the money because you could find something
like this is that right? - Absolutely, but here's the problem. A lot of times they don't call me until they've already spent
it and then it didn't work. (David laughing)
And then the manager's really mad and he says, "We
gotta fix this, you know?" And that's where, we don't wanna blindly just
cart blanche upgrade things without having basis that it's going to actually work, right? And that's what packets can help us to do. So here in this case, we could have upgraded the
network to kingdom come, but it wouldn't have adjusted or it wouldn't have improved our latency. So what we had to do is we had to move our point of
analysis to the server side. This is about all we can get
from the client side, right? Because the client just doesn't
see what else is going on. So now we gotta go, "Okay, on the server side
I see requests coming in. What is that server doing next? Is it talking to some backend system that doesn't exist anymore? Is it just going back to some SQL box, and it's just hammering that SQL box and that's really the
cause of the end slowness? But here's what burns me up. So in this case, this is a war story. A lot of times IT organizations are large organizations
with different cost centers and you know, payment methods and such. Well the network guys, they brought me in to do
this analysis to basically, we proved it wasn't the network. But then we tried to get authorization, to do some packet captures
on the server side and we never did. So anyway, I never got to know where that 108 seconds was going. But check this out. If I take that time since
request, going back to filters. And if I say now, okay, time sense request. Let's do this. Let's just do http.time>1 That sounds bad. Show me anytime you have a request that takes longer than
one second to come back. Here I found, and I can come down
here and I can actually add this as a column. And here time since request, check out the response
times from this server. I had 108 seconds,
which was the first one. Then one after it was like
four and a half seconds, then another was 4.7 and then 30. And I said, so am I
getting the right picture? When your employees sit down to try to load this application, they are legitimately
waiting three minutes for this thing to actually start being... You know, up to three minutes. And they said, "Yeah, we start the... We start the browser and we
sit there for three minutes and then it populates and
then things are good." - That's mad. - And it's the network. - Of course it's the network. - (chuckling) Yeah crazy, huh? So if something like HTTP time, this is a great way to get an
idea of application response. Now, this one-- - You could do that for the application. Sorry Chris to interrupt. That could be FTP, whatever it is, yeah? Whatever app is giving
problems, just do that. On a capture on a client, and that will allow you to see if it's TCP or if it's an application. Well, it helps you break
it down, doesn't it? - Yes and so, let me clarify
that statement just a bit. It's not for every application. Wireshark needs to be able to interpret that there was a request in the dissector like it just did for HTTP. So it knows what gets are
and it knows what posts are. And then when it sees a response code, it can merge those together. It knows this time or this
request went for that request. This response went for that request. So it can do this for DNS, it can do it for HTTP. It can do it for a handfull
of other applications. But just because we can do it for those doesn't mean we can do it for everything. So--
- But the big one HTTPS so-- - Oh, right. So HTTPS, we don't know
what the request was. So we can't add the response in. But that doesn't mean that
we're completely in trouble. You want me to show you what
I do in those scenarios? - Yeah, I think that. I think for a lot of people I mean, I want to, I want to get
that is like real world. A lot of apps use HTTPS today. It's great to see it in HTTP, but how on earth would
I do that with HTTPS? - Okay David, so I'm glad you asked about the HTTPS, 'cause that's just the reality
of what we're in now, right? So secure applications, we don't have a get string that we can filter on with a response. So, what I'm left to do in those scenarios and I'll go ahead and take a
look at this PCAP with you. This is an example of
just a simple TCP thread and what I'll do is, I come in here and I'm
interested in take for example, I'm just taking a little walk down here... Oh, by the way at the end I have quick, which is a whole nother animal. We'll save that for another conversation. What Wireshark is able to do is it's able to make measurements based on not just the application, but it can also do it for TCP. Now in this case, I just have a single TCP thread. But if I come down into TCP, let's check out some of the
timers that it can find. You notice below, the
options I have timestamps. I can expand that. I've got times since first
frame in this TCP stream. And this is the important one, time since previous
frame in this TCP stream, right click add as column, why? Because, when I have
multiple TCP connections going in parallel, like in a real network
this is what you'll have. You'll have a lot of different TCP streams and they're not nice,
filtered connections, like you see in a lot of my PCAPs that I use for demonstration. In a real environment
I wanna see TCP Delta, because that considers the context. So I can have nine TCP streams, all of them are going all over the place, but this counter is gonna show me what was the time from this packet to the previous one that
was in this conversation? Which Delta time will only show me between this packet and the
one immediately before it, whether it was a part of
the conversation or not. - And it groups it together does it? - TCP Delta will be in context. Yeah, is that what you meant by grouping? - Yeah, so like in other words, it's not all jumbled up. You don't see all the sessions together. Like I see a line on the left hand side, is that like it's grouped them together. So you get the one conversation? - Yes.
- Yeah? - Yeah, good spot. So when I have, in this case I have a single thread. So all of these are a
part of the same thread. But, if I had several going in parallel and maybe we'll do that on another video. Where I'll see this
line but it'll break up where I'm on a packet that
is not involved, yeah. But TCP Delta is an important thing. In fact, if I come down, I can even now filter on this. You see down below in the lower left, tcp.time_delta. Now that's something I can filter for. So I can come up to the top and I can say tcp.time_delta>.1. Now I see all of the
hiccups in that thread, even in a secure conversation. This is what my eyes do. I look at that first packet. This is from the client. The client is taking almost two seconds to change its cipher spec in continuous some application data. Note to self. The second one, this is from the application taking time to get back to us. So, to be honest, between the two of these it
would be more concerning to me. This would be the one, I don't like to see server delay. Client delay sometimes you'll see just when the client is
entering in a password or just slow to go up and click on a bar. Boom, that's called client
think time, client delay. In this specific case, I might wanna look a little closer. Why is it taking the
client so long to do that? But usually in most cases, I'm gonna look for this server delay. Let's go ahead and remove our... By the way, save this as a button. Call it TCP delay, put it under TCP connection. I'm gonna make sure everybody. Everybody you gotta know how to do that. This is a live class, you know? - (chuckles) At some point
I'm gonna have to give you. You're gonna have to like go through all those little buttons you've got then give us the ones that you've created. - Oh man, we haven't even
gotten the signatures-- - I know, there's a lot-- - and we have country codes and yeah. So all right. But your question was, how do I do this with secure web? How do I troubleshoot
in a secure environment looking for delays? Well, here's one of them. Look at the context. Around, I've got 20
milliseconds, 24 milliseconds, 24 milliseconds, 265
this is 10 times longer. And this is waiting for the server to come back with data. This is an example of something that would catch my attention if I was in a secure environment, even though I can't see
the application request, I can still see the packets and the direction they're going in. I see a big packet go to a server, I see an ACK come back that's empty. That means the server got it. Then I time how long it
takes for that server to begin responding. I don't care what the data is. I know it's not the network so-- - Pity they didn't give you like half the money they saved Chris. - (chuckles) That is a pity. (both laughing)
Come on, man. No I have--
- It's sad like you said, they spend the money and
then they get you involved rather than, you know, doing
it the other way around. But sorry, you wanted to say something? - No, I was gonna ask the audience. Hey guys comment below. Do you wanna do like a war stories thing? That would be fun.
- That'd be great, yeah. - I definitely... I've got a lot of different examples that would be fun to walk you guys through in some practical, real life scenarios. - I think the audience, you know, let us know. But I think, Chris I think if you can
give us like a war story and what would've been
a better way to do it or how you saved money
that would be great. Like practical stuff.
- Cool. - Yeah, definitely. - Saving money with packets. I can see it now.
- Yeah, yeah. - (chuckles) Yeah, so back to filters. Let me just see if there's any other ones that I haven't shown you yet. I'm just gonna breeze
through my buttons here. Slow IRTT, this is one that I use. In this PCAP it doesn't flag anything but, we can see the what's behind this filter, tcp.analysis.initial_rtt > 1 RTT means round trip time, is greater than one second. So I wanna know basically
that initial handshake, that initial TCP three way handshake. Is that taking any longer than one second? If it does, network
latency what's going on? We gotta take a look at why it's so slow. That's when I would--
- That'd be a network problem. - I would look there first, yeah. Like what's going on? Congestion. Slow stuff. Also DNS dot time. Also, everybody can use that as well. So dns.time, if I have a DNS response that's
taking longer than a second, This number is always
like a full second, right? So I could do 0.1 would
be a hundred milliseconds, 0.01 would be 10 milliseconds. So just depending on your
environment, what's normal, and how far do we wanna drift from normal before we start to hit this filter. Tinkering, oh I'd be in trouble
if I didn't show you guys how we can simplify our PCAPs. Honestly, so I get into an example, or someone sends me a PCAP and they go, "Oh, I don't know. I don't know what's going on here. Here's the IP of the client." And I can go, "Okay." So I don't know a lot about
what to filter for first. Okay? So, if you're sitting back going, "Okay, I see how to build a filter. I just don't know which one to use." Well, you're not alone. I feel that way too sometimes. So what I do is I start out by going, "You know what, I don't know what it is exactly that I'm looking for just yet, but I know what I'm not looking for." I know that there's things that I can scoop out of this PCAP to make it smaller. For example, what if the application that I was
troubleshooting here was not over TCP? What if it was over UDP? So I could say, !tcp, get outta my way. Oh, it's over quick. Okay, at least I was a
able to cut down my PCAP. All right, let me show you the one that I use actually quite a bit. As far as removing things
that I know are not involved. So I do, ! Open parenthesis arp Okay, if I can see that
ARP is working well, get it out the way. And then if I have a bunch
of stuff on the network like, stp, so spanning tree updates. Okay, spanning tree's doing its thing. All right, get outta my way. Or CDP, okay wonderful. We can see our link neighbors and so on, get outta my way, I'm
not interested in you or whatever it is. I can start to fill this out. Or lldp, or eth.addre== You can probably guess what
I'm starting to build now. ff:ff:ff:ff:ff, get out my way. Now I have a way of just
scooping that stuff out to simplify my filter. I'm still not sure exactly what it is, but at least I can get some
of this noise out of the way. - That's a problem, it's all the noise. All those other packets that you've got. - Yeah and in some environments that's a significant amount of traffic. So at least that gives me what's left over and then I can start to comb through, okay, this conversation,
this DNS, and so on, I call this just the... I believe that I have this
under no broadcast chatter. - I like that, yeah. - I just say, just, just, just-- - So click on it? Is that exactly the same
as you just bought? Yeah. Oh, there you go yeah. - Except for ICMP--
- ICMP, yeah. - And then as I learn more, I'll just add stuff to it. Like this is where I
could say or in this case, you know what, I'm not
even looking for TCP. Let's get all my quick, right? So the idea is, I really want to just
simplify these PCAPS, to get them to be as simple as I can and then I can dig through
and start using these filters to really try to find stuff. Now, as far as... While we're still on filters
maybe just to wrap up, Signatures. What I do and... On this profile I don't
have a ton of them, but I have a security
profile with a lot more. What I did is I honestly
just started studying suricata and snort signatures. And I started building Wireshark filters based on those IDs signatures. And that way what I could do is I could start just
ripping through here and, okay, weird TTLs like this one is a TTL that's less than
50 or greater than 30. The reason is because if I have
a TTL that's in that range, well it probably started at 64. That means I'm dealing
with a network that is more than 15 hops away and it's just in this weird band where it could be fabricated, right? 'Cause a lot of times I'm not
dealing with networks that I'm 25 hops away anymore a lot of times. I'm usually comfortably
within the 10 to 15 hop range or even, or less. This might be something I wanna check out. I think I got that from
a suricata signature. Country code I could have USA-- - Russia.
- UK. Yeah, Russia. That's a Russian country code. If I wanna see any traffic
coming over from there. There's some dodgy place over here. (David chuckling) So those guys, they're up to no good. That's just a kept. That's how you catch all
the David traffic everybody, just do geo ip country GB, and that's probably David talking to you. A TCP SYN with no options, that's weird. So, here I have a TCP
flag it's going out and the TCP header length is 20, super unusual. I should see TCP options. You guys, remember that
from the TCP deep dive. I should see MSS, I
should see window size. I should see timestamps. I should see that stuff. So if I see a SYN with no options, no way, nah, uh uh, that's not good. - That's Nmap or something, yeah? - That could be. I've seen that from Nmap and I've also seen it on a true scan, some scan activity and there
was just no options in there. I'm not sure if they were using that tool or a different one, but yeah. But either way, I know that's not coming
from something real. Yeah so hopefully just to
kind of recap a little bit, hopefully this doesn't overwhelm anyone. But, we started simple
with those IP filters, the conversation filters,
the TCP port filters. And then we pivoted more into some of the performance filters. So like HTTP time, DNS
time, TCP Delta time, those are great filters to use. And then we wrapped up at the end here with some security filters
that I like to use, where we're almost acting
like an IDs at this point. Just using signatures to try to see if we can
find any weird traffic. - So Chris, you said we could perhaps do a deep dive on performance, yeah? Like slow networks. I think that's gonna
be a great topic to do. - Yeah, absolutely and
honestly those are... I have a lot of examples of that because-- - That'd be great yeah.
- That's what I get called to do. In fact, maybe I can tease
everyone with this one. I have one where they were
literally getting ready to just gut replace the network, because things were so horribly slow and it came down to one
little mini configuration and I'll show you what
we found in the packets. - (chuckles) Now that's a good teaser. Chris, really wanna
thank you for your time. And I'm really looking forward to that, 'cause now you've made me very curious to know what that's about. Buy yeah, thanks so much
for sharing your knowledge. You know, this is like years
and years of experience that you're sharing here. So thanks for doing that. - Hey I have a lot of fun doing it. So thank you for having me on again David and appreciate everyone that watched and please let us know how you liked it and what you wanna see. - Brilliant Chris. Thanks, cheers. (lively upbeat music)
