VoIP Problems and How to Correct Them

that's already up there well hello everyone today I'm gonna be talking about common void problems how to detect correct and avoid them my name is David ETS or Samuel David atius I've been in the iand I've been installing VoIP systems for seven years I've been a mikrotik user for five years and I currently hold the mikrotik certifications of MT CNA MTCR II and MT cwe I've also been in the IT industry for 20 years so the purpose of today's lecture is going to be to inform mikrotik users on how to identify and resolve void problems this lecture this this lecture is going to be a lecture style presentation more like classroom type and it's really geared for people that are first getting into the voice voice over IP field the voice over IP industry or for early startup VoIP providers that are currently having some call quality issues because those are tricky so today's agenda is going to be to identify factors which will affect the VoIP call quality we're going to correct call quality issues with router os QoS we're going to be doing that by Martin marking packets with mangle and we're gonna be shaping VoIP traffic with Q's we're also going to be detecting VoIP call quality problems and we're gonna check for drop packets in queue views and we're gonna use writer release packet sniffer along with Wireshark to analyze void packets and then I'll teach you how to completely avoid a bad situation from even stepping into that situation so we'll get to that and so to start what is wipe well VoIP is several protocols used together to send and receive real-time voice calls through an IP network or networks so let's identify some factors that affect to call a VoIP call quality because that's what we're gonna be talking about so a few considerations to keep in mind about voice is that VoIP calls a real-time so RTP packets or the voice part of voice over IP can't be retransmitted it's got to get there right the first time without any problem so calls are real-time next the connection between the phones and the VoIP servers must have low delay and even lower jitter and if you're a company like me you're gonna see that if you're a company like me my server is actually a cloud-based and we are we are in three data centers across America so customers phones will connect to any one of those data centers based on where I decide to set them up another thing to consider is you must have enough symmetrical bandwidth for the users because VoIP is actually symmetrical so you the calls that you make are gonna be at 88 K on a specific codec and you're also going to be receiving 88 K so it is symmetrical so if you have a customer with a thousand Meg's of download speed but only one megabit of upload speed you can only push as much as your upload again it's symmetrical so keep that in mind and to do some of the math you'll see over here I like to use the g.711 you law codec and that's because it's the most widely accepted and it is one of the standards on the internet for doing that if you need more bandwidth or if you want something with better compression you could use GSM here is some of the what the payload is per per call it actually comes out to eighty seven point two K per channel alright and that's with 20 millisecond of voice payload per packet and sip messages sip is the actual signaling that tells the phone will tell the server make a call or the server will tell the phone you know ring or transfer a call or put a call on hold that's the signaling so the max sip message size can be no bigger than 65 point 5 K so keep that in mind when you're doing your bandwidth calculations so now let's talk about some problems that we're gonna have with voice over IP not considering hardware problems because that's a whole nother ball of wax and by the way any tiny little hardware problem will start to affect your users call experience I had a bad dim card a little bad memory chip and it made it appear that there was jitter so we checked everything and we're looking at our network and we're checking for lost packets and wasn't any it was hardware so we're not gonna be talking too much about hardware at all we're won't be talking at all about hardware we're gonna be talking about things related to networking so what can effect call quality high jitter levels and what is jitter this is what generator will sound like when it was a very very experiments visit it's not fun it really frustrates people but the meaning of jitter is packet delay variation you could also think of jitter as time lapse between each packet another thing that'll affect call quality is packet loss we don't want any packet loss so we're gonna talk about how to reduce that or prevent it from happening completely and the last thing is delay delay is actually a really tricky one delay is interesting because you can have no packet loss you can have no jitter and you can still have a really bad VoIP experience so delay is just what it just what it sounds like how longs it take for your voice to get to the users here that is delay an acceptable level we found for delay is anything higher than 200 milliseconds in either direction is going to be a problem why because 200 milliseconds it's roughly a quarter of a second in humans we can actually hear that so the voice that you're saying on the phone the user is gonna hear about a quarter of a second later so when you're done talking they don't know that yet and so you continue talking thinking that they're not going to respond they have nothing to say then you end up talking over people the reverse is also true when someone talks you don't hear it until 200 seconds later so 200 milliseconds later that would be bad yeah you don't want that but if there's no packet loss no but that's delay so delay is actually a tricky one yeah again acceptable levels for that anything above 200 milliseconds will start to be noticeable people will start talking over each other acceptable limits for pack loss is really zero you don't want any you could have a little bit really that I'm talking less than 1% and of course jitter levels that's that's an interesting one so we have something called a jitter buffer and what that does is packets that come in to your voice device in this case it'll be a phone they could come in with different levels of jitter all right there could be variations in the time between packet 1 packets who a jitter buffer takes it and it actually buffers it for a very short period of time and then it'll let out that stream nice and evenly so you're actually by using a jitter buffer you're actually increasing your delay because that's how it works it introduces delay to even it out on the output so acceptable jitter levels we found 50 milliseconds is really the limit you don't want to go higher than that once you start going higher than that you're increasing your overall delay and you've already expended what your jitter buffer is going to be capable of so let's try to keep jitter levels under 50 milliseconds in the real world packet lost jitter packet loss and delay jitter packet loss and delay can happen anywhere between the phone and server so if your company is like mine and you're a VoIP provider your customers will be using their internet connection to get to your VoIP system and of course packet loss can happen anywhere along those routes luckily we found that 90% of call quality issues happen at the customer's location all right ISPs have done their part to keep their networks nice and clean and really in order and quick with enough available bandwidth then what we have to do is we have to start taking care of the customers and because like I said that's where we found most of the problems happening and why does it happen at the customers location because networks are rarely configured properly if contribute at all for QoS and voice so keep that in mind use a mikrotik now let's talk about correcting issues with router os QoS so what is QoS there is no one QoS QoS is actually a group techniques that are used to basically categorize and prioritize traffic and I'm going to emphasize categorize and prioritize because that's exactly what we're going to be doing and I'll show you some examples of how to do so first we're gonna categorize our packets then we're gonna prioritize them we're gonna get in priorities of who has well more priority over another stream another think us does isn't its it ensures sufficient bandwidth it controls latency jitter and reduces data loss and I say reduce because part of QoS and in this case queuing is that we actually want to be selective about where we're gonna delay packets where we're gonna drop packets and where we're gonna let packets through with no delay or no interruption so we're gonna reduce data loss QoS is will also regulate data flows by classifying scheduling and market marking packets based on priority and by shaping traffic this is really just the definition of the other two because that's what we're gonna be doing we're gonna be classifying which is categorizing scheduling is putting them into queues where they're going to be in there for a temporary holding period almost like a very short buffer until your bandwidth you have enough available bandwidth and it lets it out we'll get to that later so how do we start categorizing packets well in mikrotik world it's called mangle and what is mangle mangle is a router arrest facility that marks packets for future processing so again first we're going to mark these packets we're gonna categorize and mark them and then later on we're gonna do something with them the mangle marks only exists within the router that's important to know these marks will never leave the router therefore it's a mikrotik only thing it's also used to modify some fields in the IP header like dscp and TTL we're gonna talk about the SCP a little later because that's important another thing to keep in mind about mangling is that you only get one packet mark and one connection mark per packet and that still is enough to do everything we need to do so in this in this example the technique I'm using is really just a conserve CPU processor or a processor time so first what we're gonna do is we're gonna mark the connection and every packet in that connection will then be marked with a connection mark then what we're going to do is we're going to use those connection marks as the identifier to do your packet marks and why is that efficient well because once the session is in connection tracker all packets for that session are marked this is more efficient because mangle doesn't need to scrutinize every packet based on all of your criteria it just needs to know if the packet is in that connection or not so we're gonna mark the first packet that starts a connection then once that connection is in the mikrotik connection tracker any packet that is in that connection tracker will then be marked with a connection mark with a connection mark and doesn't need to be scrutinized so it does save on processor power so here is my example my sip server 1.2.3.4 it's not a real server I'm just giving an example and the SIP port is your standard 56 D TCP the RTP port range is going to be the Esterbrook standard of ten thousand twenty thousand UDP so this is how this is actually a manga rule and to go to and to create a manga world just go to IP firewall mangle click the Add button you get this these two boxes are actually the same rule I'm just showing you different tabs so on the first tab the general tab this is where we actually classify or we're identifying traffic so the chain is going to be forward because it's for a packet that flows through the router then the destination address address of course 1.2.3.4 the protocol will be six port 5060 now notice on the bottom can well hello and notice on the bottom connection state the connection States going to be new and that's because we only need to mark one packet in a connection so I'll make it the first packet then every other packet that is in that connection will automatically be marked so we don't have to process we don't have to scrutinize every single what's inside every packet header now on the right after we've classified what are we going to do with it in the action tab that's where you do it and over here let me just see we're gonna mark the connection and I'm gonna give it a name the connection park is gonna be sip connection pretty simple that's what it looks like in the command line and this is actually what it looks like in your manga list now that we've marked the connection let's mark the packet okay and we're actually going to use again we're going to use the connection mark as the identifier to mark the packet hence saving CPU time so again the chain is going to be forward but over here now all we're looking at is we're looking we're checking all packets for the packets that have the sip connection label on it or the sub connection mark on it then again you go over to your action tab and we mark the packet and I'm gonna mark it I'm going to give it a name called sip there you are in the command line and that's what it's gonna look like over in your manga list first we mark the connection then we mark the packet pretty simple do the same thing for your RTP the same exact process the only difference is I'm going to use any port 10,000 and 20,000 because RTP packet RTP ports are selected dynamically on the fly on the source and the destination so I just chose any port between ten thousand twenty thousand those values can't be changed on your server alright so let's see now same thing at the bottom the connection states going to be new I just need to mark the first packet in that stream then every other packet in that stream is automatically marked and on the right the action tab again mark the connection I'm gonna give it a name RTP connection there's your command line and there it is in the mangle list alright and then same exact process mark the RTP packet with a packet mark change forward connection mark is going to be RTP connection that's all we have to look for now and again the action is marked the packet and with the word RTP there's your command line there it is in your manga list so we've actually just created four rules even though it's a two-step process it does save processor time now you could do this the way I think every beginner does this you just mark every packet directly then you'll only have to mingle rules but then you're asking the mikrotik the router to actually go in and inspect every single packet every single header in every packet with all the criteria that you're looking for so you might see an increase in CPU usage depending on how much traffic you pass you may want to do it that way if you pass a lot of traffic like I do you might want to start thinking about saving your processor so how do we know this is actually working well it's always good to check your work now the first thing we do is we go over to firewall we go to IP firewall connections and here's your connection track or in mikrotik so if you see over here that doesn't really work this pointer so if you see over here we have a couple of connections if you see all the way on the right under connection mark you see sip connection for two of them and one RTP connection so we know that the connection marks are now working because it shows up in your connection tracker another way to do this is you could do this in any mangled rule just open up any one of your mangled rules I like to choose a mangled rule that has packet marks where we're actually I'm sorry doing a packet mark not a connection market because the packet marks depend on connection marks working properly so we can go in and we can put a check by the log and the log prefix just give it a name so that it's easy to pick out in your log list when you're going and all your logs are flying up you can just really quickly pick out I wrote RTP packet and if you look on the bottom all the way on the bottom I've actually pulled out one rule out of the log and you'll see over there in the beginning I've highlighted RTP packet and that proves that that is working now it's talking about changing the SCP so what is dscp the SCP just stands for differentiated services code point otherwise known as diffserv it's a filled in it's a filled in an IP packet that enables different levels of service to be assigned to a network traffic this is another way of marking traffic only this is a standard and this field in the header is actually going to traverse the network this is excellent for telling other devices on your network which packets that priority in which don't now in this example I've actually chosen to mark packets that return that come back in from the internet because packets that come in from the Internet I'm going to have any DHCP values why not mark them at its first point of entry so over here again I create a new Mangal rule and the chain is going to be post routing post routing is when the packets get processed after the mikrotik has made a routing decision or you could think of it as just before a packet ooh exits an interface so I chose that my out interface is going to be VLAN 20 because that's where I put all my phones and by the way you should always VLAN your phones keep them in a separate network you don't want them to hear any other chatter or any of the stuff going on in your regular LAN and I've seen it where it actually makes a huge difference so please VLAN your traffic so over there the out interface is gonna be VLAN 20 again remember it's traffic that comes back into the network so the exiting interface into the network will be VLAN 20 the packet marks that we're going to be changing dscp on are the RTP packets because those packets can't be retransmitted and we want to give it a high value for for queuing I'm sorry for for priority and if you go over to the action tab you'll see that we're gonna change the dscp value we're gonna change it to 46 which is the highest level for dscp otherwise known as EF for expedited forwarding and this kind of forwarding this kind of I guess a priority is the best for applications that require low jitter low delay and low packet loss so that's how we change dscp next let's talk about mikrotik use well now that we have marked our traffic we have all these marks and all sorts of traffic well what are we going to do with it what we're going to queue it and queuing and mikrotik queues does all this I'm only going to be talking about queues mikrotik Hugh's one of the things that'll do is limit data for certain IP addresses subnets protocols ports and other parameters I'm also going to prioritize some packet flows over others and the first thing you need to understand about cues in mikrotik world is that we have something called parent cues and child cues the parent cues are sometimes referred to as inner cues and their only responsibility is to distribute bandwidth we also have child cues otherwise known as leaf cues and that is responsible for child cues are responsible for consuming bandwidth think of like a parent feeding a child right and keep in mind you can't feed your child more food than you have same thing applies here over here here's the cue list and we're gonna be using cue trees and if you see the parent cue is up on the top of that list child cues are indented kind of in the bottom as a sub-q now again parent cues are only responsible for distributing traffic to the child cues the parent cue will first try to satisfy the child's cues limit at traffic then try to reach the child's max limit so limit out is your Cir otherwise known as your committed information rate and that let's think of that as your guaranteed bandwidth your max limit is how much could you possibly consume for this cue what's the most bandwidth you could push so those are two values dual limits as mikrotik calls it another thing to keep in mind about paranoia about mikrotik cues is that priorities are ignored on parent cues so if anyone here is a parent you'll know that your priority is ignored and your children have priority so that's how that works same thing here also keep in mind that mikrotik has eight two priority levels eight is the lowest priority one is the highest this is a little bit backwards from how you've ever seen it done in cos or TOS and I actually like this technique a little bit better because you can never have anything higher than one one is gonna be your highest level now mikrotik use the cue with a higher priority will have a chance to satisfy its max limit value before lower priority queues so that's important to understand an actual traffic prioritization will work only if limits are specified a lot of times I've created cues and totally forgot to put limits in there well that you is gonna do absolutely nothing because it doesn't know what to do with that traffic so let me let me start cubing some traffic and let me give you the scenario this scenario is gonna be simple it's my home I have five phones and my internet bandwidth is thirty five megabit download and four megabit upload so to create a cue tree just go over in your wind box two queues and go to cue tree and then add a cue on the left you'll have the command line but on the right let me explain what that is so the name I'm gonna call this is just called upload the parent cue is actually going to be an interface and and the interface is going to be Ethernet one and that is the Ethernet one gateway which is actually my interface that connects to the internet so this is my uplink and that's important to know because queues only work with exiting for cue trees only traffic that exits is actually what we're controlling here so anything leaving Ethernet one will be upload traffic the key type is going to be default we'll get to that later priority is gonna be one which is ignored this is going to be a parent my max limit the most I can actually push is for Meg now this is where we actually create a child queue to create a child code it's real simple all you got to do is select the parent queue that we just created which in this case is going to be upload now here's where we put in packet marks so remember how we mark packets packet mark for this one is going to be RTP and the you could actually add more than one packet mark to any queue it adds several I just like adding one because later on when we're checking for drop packets if there's only one traffic type in that queue and you see drop packets well you've know you've drop packets for that traffic type you don't have to figure out is this RTP traffic zip traffic TCP acts I just do one / Q again the queue type is gonna be default and that right there is just packet first packet a packet first in FIFO first in first out that's how it processes packets for the queue and that seems to work just wonderful for what we do the priority is going to be one again that is the highest priority queue now remember how I said that I have five phones and my house has only been provisioned to support a maximum of five concurrent calls so with you La Kodak of one call equals about eighty eight K so I've set my limit at my guaranteed bandwidth to four hundred and forty K cuz eighty eight times five is four hundred and forty K and I've set my max limit to four hundred and forty K because I never need to go over that remember it's also symmetrical and you could do the same thing for your sip packets the parent again is going to be upload your packet marks are gonna be sip and a q-tip again default priority this one's going to be two I chose this one to be two because my TC my sip packets are all TCP if there actually is a drop packet it'll be retransmitted okay so that's why I did that and again the maximum sip message size is going to be sixty five point five K so multiply it by five because I can't see there being more than more than five concurrent sip messages going on at the same time in my network sip sip messages are usually a lot smaller but I just decided to go with the max so again sixty five times five is 325 K and I've done that for my guaranteed bandwidth and the maximum I could push you could change that if you need to but that's how I do it now an important thing to think about is what about packets without any marks you know they are still consuming bandwidth and if they're not accounted for and if they're not queued then your queuing and your priorities are not going to really work because when does this really start working this cueing really starts really starts to do its magic when you have maxed out your bandwidth when your parent has reached its max limit now it has to start the siding which traffic are we gonna allow through first and second and third and so on and so on so you could think of it similar to like traffic on a highway where I come from the 405 is always jammed but you know what we still need to leave some priority lanes for those people that our ambulance our police are whoever that need priority to get through will be in the event of emergency so think of that as being the priority and that's exactly what we're going to be doing here we're gonna be selective about which ones are going to go through first second third but packets without any marks still need to be accounted for so let's create a queue and again the parent will be upload the packet marks this is actually a default in mikrotik you will always have the option of a no mark packet so you would put that in there the queue the the queue type is still going to be default and the priority is going to be 8 that's the lowest so in the event of a traffic jam we don't want to drop RTP packets or sip packets we want to drop all the other stuff your email your YouTube videos it's okay to queue that traffic and then eventually drop it if bandwidth doesn't free up so this is how you do it over here now this is what your cute you're cute tree list should look like once you're done configuring it now notice I also did download interesting thing about download some people say that while it's true that you can't control what traffic comes to your internet connection but I got to say most of my customers if not all of them don't have any servers that are open to the public Internet like a web server and they're not serving you know thousands of visitors a day or anything like that so in this case downloading where the initiated traffic is on the customers land it's perfectly acceptable so this is this is where we actually I've seen really good success with this kind of setup next thing is if you want to be really nice to your customers you can also prioritize other traffic types like TCP acknowledgments DNS you do that too now let's talk about detecting problems what happens when a customer calls and says hey I have bad call quality well you better have an answer otherwise that might not look good to them so let's check for drop packets that's the first thing I usually check for before I go into other types of logs and I enable the drop packets for you to do that you just go over to your cue tree and a little drop-down carrot on the top right click that click show columns then click dropped and if you look on the right you're gonna see that we have drop packets exactly where we want them to be dropped if you see packets being dropped in other other queues well then maybe you need to resize your Q's set new limits do another bandwidth test make sure you have the right kind of bandwidth available to the router and on the Internet but over here once we see dropping packets in priority 8 that's perfectly normal again we're selective about which packets we want to drop queue and which packets we want to have go through without any issue I also want you to notice that you have colors over here for each Q we have actually green yellow and red so green means that you've consumed between 50 and 50% of your available bandwidth for that Q yellow means that you've consumed between 51 and 75% of your available bandwidth at Q and red means you've gone beyond 76 to a hundred percent of bandwidth utilization that's when your packets will start queuing that's something to start dropping over here you see we have red that's when it was dropping I was doing a bandwidth test when I took this another thing that's kind of interesting this could be fun a mikrotik packet capture so you've checked for your drop packets what's gonna happen when you actually when a customer calls and says hi you Dave this is what I sound like well we need to find out where that's happening so first thing you should do call the customer and start a packet capture on their phone and actually I think I'm gonna do that right now hang on just give me a second I gotta set this up we're gonna do a we're gonna do a live packet capture with one of my customers and my cable is unplugged hold on is this thing supposed to be plugged in or does it go let's see yeah it's plugged in I got nothing yeah yeah I got I got ya I got nothing though see anything that could cause that well I'll go through the GUI let me think and let me try to log into I could I use Wi-Fi well we're gonna use Wi-Fi let's see if that if that helps at all okay now let's say all right this looks alright let me share my screen so I could show you what I'm doing okay now let's go over to win box let's log into this customer and we'll call them up on the phone and we'll start the packet capture if we can download okay so to set up your packet capture just go to tools packet sniffer give it a file name also don't don't check only headers we actually want all the payload we want to we want to be scrutinizing and running statistics on everything not just the headers I'm giving this a file name called wipe that pcap so now now let's call this customer and sorry get into that hang on you're gonna see that in one second let me just call her name is denill and I told her to be next to her phone when I call so let me put her on speaker oh let me start the packet capture go over to filter give you no capture the IP oh that was bad packet sniffer and start okay and you see in the file we're actually starting to capture that file let's see I'm gonna leave her such a good message and I'll return your call as soon as possible thank you then now you know I love you I just needed you to do one thing can everybody say hi to Danelle please that's about a hundred people staring at me doing a packet capture that's not really gonna do much right now but still love you I'll speak to you later it's okay though well it's okay because we actually did capture the the RTP for that and let's stop the packet capture so no actually the server picked that up and never made it to our phone so we didn't capture anything but let me show you with a packet capture that I've already pre captured just in case just in case someone is at their phone isn't at their phone so if you see over here I have one called Kyle - hi poppy cat I've captured this earlier and if you open it in Wireshark come on Wireshark there you go first thing you do is filter out the RTP stream and it is case sensitive so do RTP RTP hello RTP oh okay and then just pick any packet in this stream because well it's only gonna be for the phone that we've targeted then what you do is you go over to telephony go over to RTP and then go to stream analysis and this is actually some really good stuff this is where the magic happens over here you can see your max delta you can see how many packets you've passed you see how many drop packets you passed over there where it says lost RTP packets right here sequence errors any packets that don't come in in sequence are automatically dropped so if you have a customer with an issue you're going to want to do one of these first check for drop packets and your queue views then do one of these another thing that's really nice about this and I'm gonna build on this later is that you see how you have the player button right there well the player buttons nice you can actually replay this RTP stream I'm going to do that right now just click decode pick the legs that you want to actually play we're going to play both directions and now we're gonna that I already click decode I'm gonna click play listen well that I can't get it to play on the I can't while I have ups I have a mic so I have an external speaker that I've actually bought with me just for such occasions which I can't find right now but well let me try to play it again and that's how you can actually capture a packet capture a sip stream RTP stream play it back to the customer this is important this is you could do this real time if you don't have any type of special software all you need is Wireshark in the mikrotik packet capture and it works great and again run those statistics right here see what's going on lost packets look for your jitter levels your mean jitter your max jitter so over here is going to give you that that kind of information and this is something everybody should have access to Wireshark is free let's go back to the presentation and where did I leave off oh look at that we're from the beginning let's see how fast I can click okay so we saw that over here the RTP pick it out play it back and that's good now this is how to avoid problems avoid standing it getting into a really nasty situation first thing is be selective about who you take on as a customer make sure their internet connection is adequate and I'm not just talking about bandwidth or symmetrical bandwidth available because I actually have a customer with only 1 Meg of upload speed and they are flawless but before you take on a customer do some very basics to kind of a monitor on their internet connection I use Avex and I just do a simple ping to their server but with the response from the ping I am testing those things that will affect call quality I'm gonna be testing jitter packet loss in latency ok and this is actually a graph of a customer that I have to tell sorry can't help you don't get wiped from me don't get it from anyone until you either change your internet connection or correct any problem that it has unfortunately he could only get a t1 where he is which was oversubscribed and really wasn't that great to begin with because of the location that he's at if you see over here this is actually measuring jitter and if you see his jitter goes way beyond I will not way beyond but it goes it goes above 150 milliseconds which is really the danger zone I don't want to take any customer that has more than let's say 50 milliseconds of jitter and why bump because once you do all your QoS and everything you could redo that reduce this to a certain extent but you can you can't eliminate something like this this is just too much so I told him and he's actually I know I'm very well CP our industries in Southern Cal huge company but pipe is not for him so be very careful you don't want to sign up a customer you don't want a salesman going out selling someone that has an old DSL line because they live in Timbuktu and they and just their connection is not adequate salespeople he'll do that they'll sell anything they'll get the IT people to install stuff and then when it comes to go live all of sudden customers like Oh everything doesn't sound right I'm getting jitter I'm getting Peck I'm getting call drops and getting all sorts of stuff and then you're like well go go speak to your ISP and that's when the finger-pointing starts and you know don't get in that situation because try explaining to a customer what jitter is when you tell them it's your internet and they say well I could browse bad situation stay away from that all right so I think some of us have been there but but really that's that's one way we do it it's also a good idea to monitor the latency or the delay and to measure packet loss again don't just do jitter do those three things you know show the customer hey this is what we're monitoring and you look like you're you're good to go with VoIP and by the way this will build trust and let me tell you I have not lost one customer to date since I started my void company because I take care of my customers this way you should too and you'll see they'll they'll be loyal and actually now they start calling me up asking hey Dave I can't load this web page could you take a look at the monitoring and so they start asking you for also what just happened they start asking you for all sorts of stuff which is which is a good thing so to avoid problems you could also do you could capture every calls every call that you ever that ever goes through your network you could get that in a peak app there's a free software out there it's open source it's called sip capture org I highly recommend it and what this will do is you you will send the data stream of your server for all the RTP and sip let you know kind of mirror it to your sip capture server or VPS whatever and it will capture everything it'll put it into a packet it'll run statistics on the call it'll do some of those statistics for you so if you have a customer calls up and says hey I called someone this morning and it sounded awful then you go you can actually go back to that call you could look at the statistics for that call you could replay that call if you want some HIPAA people are a little bit about that but you could do that and it's gonna show that you really do care about the customer and you care about their call experience nine out of ten times when I do this it turns out the person on the other side was on a cell phone in the middle of nowhere and I say hey by the way this looks like it might be a cell phone because I don't see any packet loss I don't see any jitter on your side but even then that's not enough everything could be perfect and you could still have problems and the other person could be on the landline and when does that happen it's rare but it does happen it actually happened a few days ago between Verizon which is now frontier in California and cogent networks so you can go to internet health report and it will show you all the major carriers the backbone carriers in America and who they appear with and if there is a problem like over here we have between Verizon and level three if there is a problem and you've checked everything and you showed the customer that you've checked everything and everything looks good but hey this call is going to a carrier from my server to the carrier and it's taken a route that's going through one of these bad connections good news about this good news about this is that carriers you usually take care of their problems almost immediately they are really fast about this so you show them everything and they build trust and they trust you and then they're calm when you don't have an answer oh that's when you're in very big trouble when you say I don't know let me open a ticket you know that I don't like hearing that when I call my carriers I want them to know right away and when they say hey there's a problem with this and that and okay cool how long can't give you an exact time but we know in the past they've solved their problems within twenty minutes thirty minutes or whatever so that really really is a huge thing use those three things you know check for lost packets do your packet captures and also check the internet if all else fails see what's going on so to summarize we've talked we've learned about how we've learned about some factors that will affect avoid call quality we've learned how to reduce or eliminate call quality issues using in queues and we've learned how to find issues and diagnose issues with Wireshark and we've learned how to avoid a bad situation all together so that concludes the presentation for today are there any questions thank you there are a lot of questions okay let's start the first thing we do with with our sub systems is to turn off the sip helper and mikrotik how do you work with the SIP helper I didn't hear you but how do you work with the SIP helper and in the mikrotik system I completely turn it off I do and it works and I'll tell you I'll tell you what happens when the way to check that is go into your asterisk box and do a sip show connections and you'll see what the way in connection it's seeing and also what the internal IP that that phone or device is reporting when you see both of them are your when you'll know right away that you have a session helper problem so turn it off immediately and all of a sudden you can see the internal the external and everything will work fine I do that by default by the way for all my customers automatically turn off sip how does mikrotik respond to that yeah the truth though is it just works it works so just turn it off it works and it's gonna vary based on what kind of network you have what you have in your data center are you running that out you're not running that in your data center I don't is the customer running that they should most of the time but it just it does its job next question okay so are more of the more than the technical approach you would take to engage the customer you say salespeople they know how to sell so basically how would you engage or try to solve the problem with a person that tells you well you're talking about either you do what's dear is not the other so so what would you tell a customer that is constantly constantly telling you I can hear my peer I can hear the voice of my peer and how do I solve it how will you take the approach to talk to let's say an end user and one soft question RTP yes it would be shouldn't it be s RTP or TLS shouldn't it be secure Thanks that's just the signalling you could make it secure if you want I don't it it makes no difference to me if anything if you want to be if you want to be HIPAA compliant heard a couple of questions in there but if you want to be HIPAA compliant the only thing you have to encrypt is the voicemails that you store HIPAA compliance only talks about stored information information you could encrypt your sip messages you could even encrypt your RTP messages it's just going to put a bigger load on the server but the question you you said also something about salespeople how to use how do you tame a salesperson what I know love though the technical jargon easy I actually take a deck of cards with me and each card represents one packet and so I start putting packets on the table alright and then I stop and I make a delay and then comes the next packet and then the next packet and then a quick stream of packets and I say the time between packet one packet to think of that as you're back at the lake variation and then I talk about jitter buffers but that's only if they have an IT department a regular customer what do I tell them I say to just sign here everything will work you know that's it I am you showed using care tree for doing voice what about using simple keys book use simple these are interesting I've always used cue trees I like to use cue trees because cue trees offers one direction whereas simple cues it could be both ways so it actually eases Mangal configuration I don't have to make Mangal mangled marks or packet marks for upload traffic and download traffic so it cuts my mangling in half and simple keys has a couple of other features like you could do it based on time of day I don't ever want to do that what else simple cues cue trees is just what's worked for me always and I'm doing it based on protocol not by IP address or anything else thank you also if you want to use simple cues and you have a list a long list of cues you know the traffic has to go through each cue one at a time with cue trees boom it happens all at once so a little bit less on the processing if you have a lot of cues next question when you set your max limits for your child cues is there a benefit to setting the max limit to your maximum connection bandwidth or should like you guarantee that - 440 kilobits whereas like if their download connection was 10 megabits could you set the SIP and RTP Maximus to 10 megabits that's actually an interesting question good good good measure for setting up queues is that your max limit for your children of a parent do not exceed the max of the of the pet the sum total of your child's cues don't exceed the Mac the max limit of your parent I was asked this by a friend of mine saying well how do I do dynamic I don't just want to allow this much bandwidth for this type of traffic let's say HTTP traffic let's say I want to allow the max but when you're not using that max we want something else to take its place I I'm experimenting with that we're just setting all of the max limit to the max limit of the parent and see how that works I haven't messed with that too much but I'll experiment and we could do it maybe after and I'll tell you what kind of results we get I'm just going by what mikrotik says to do and what suggested to do so that's the best I could tell you with that okay thank you next question we've got about 30 Asterix servers running recently about six months ago we started using VPNs we were doing all of the queuing that you were doing up there but all of our queuing all of our QoS went out the window when we started using medians and you can understand why yeah that's that's actually a good question and the first thing you should do is find out what your maximum segment size should be if you're going through a VPN you may want to lower your Maxim its maximum segment size because VPN will take some of that overhead what's the question though we also changed for internal reasons we changed the packets from UDP to TCP because we're using we're using some special software on the asterik servers to do sip signaling well it's where we're using it to run blf we're using on the old Cisco phone to 77961 we're using it like a key system instead of a straight PBX so we had to change it to TCP and your question question is the QoS once again went out the window I can't I'd have to look at your configuration there's too many variables one thing I'll tell you that I swear by is the RTP I always make it UDP because you can never really retransmit voice it would be like taking words from the beginning of my sentence adding them at the end because it's been retransmitted always use UDP and also do your logging you know the key to finding any answer to any problem is do your logging do your packet stiffs alright and analyze it in Wireshark see exactly what's happening it could be one of a number of issues it could be something with NAT could be something with the MSS you really is there's a lot you can speak to me after and we could go through some of those things okay thank you do we have any more questions yes okay can you show what you did in Wireshark one more time real quick cuz you were very fast like a ninja so they call me the ninja am I still on screen here good alright so after you've captured your packet do you want me to show you where are you hey York do you want me to show you just mikrotik packet capture you want me sure okay after you've downloaded your a packet your your pcap file like we have right here just open it in Wireshark alright there's gonna be some other stuff in that stream so you could just filter it out by RTP and again lowercase R T P and then apply it and now these are all the packets for RTP click on any one because this is only going to be for that phone we don't have to go and sift through which RTP stream is for which phone and I'll show you another nice little shortcut and how to do it this even easier but I want to show you the statistics screen that's why I showed it this way so after you've filtered out your RTP click on one packet could be the first one go to telephony go to RTP and go to stream analysis take a pic come on I'm gonna photobomb it ok all right good so then after you click stream analysis here you go it's not only gonna list every single packet it's also going to list some information about your your capture so for example the things that I look at is total RTP packets expected this many lost packets sequence errors right there lost packets and sequence errors is going to be most likely indicative of a QoS problem let's not assume that there's a problem on the internet but you're gonna look here you could also see that kilohertz that was recorded in you could see well how much voice actual voice payload was in per packet you can tell the time difference between packets so that's how you use Wireshark for that and then again you could just go over to the player decode and then put a check for both legs of the calls and the caller and then just click play I don't know where the mic is on this damn thing so that was with me and my friend just for in case this I also want to show you a quick another way to do this and this is nice if you just feel like capturing all packets from a customer's land right you don't got time we got to do this quick I'm having the problem right now you jump in do your sniffer I would say just go to let me let me clear this I would say go to telephony then go to VoIP calls this is nice and you could also see the sittin the SIP conversations if you do it this way go to VoIP calls now in this case I only have one call but if you had 10 people on func you would see 10 seconds here some of that might not be complete the status over here the state may know for this one is completed because I started the packet capture before we made the call so it's got its invite and its goodbye message but if you capture a call in the middle of a call it'll say in progress and then over here once you have this the cool thing is you can actually click on flow uh we crashed so let's try that again real quick we go to telephony we go to VoIP calls come on there you go it's gonna actually find all the VoIP calls you're gonna go to flow and we crash again well if you click flow you'll actually see the SIP messages between the phone and the server it'll give it to you in a nice what when you see it if you're a void person you will recognize it and by the way that is an excellent way to tell when a call dropped happened because the buy signal is either gonna come from your server or from the customer so when a customer says and this happens to me I mean I get every single crazy idea someone has they always throw it at me so I had a customer that said I keep getting call drops alright I pull up my stuff in sip capture and I look at the SIP conversation and then I'll go look where the buy signal come from sometimes it comes from the customers phone meaning they click buy they click something oh I forgot I didn't put them on hold I actually hung up on them so that's another great way to troubleshoot sip problems when it comes to when it comes to call drops any other questions ok cool thank you let's

Channel: MikroTik

Views: 29,616

Rating: 4.9418607 out of 5

Keywords: mikrotik, routerboard, routeros, latvia

Id: mnX6Im8GlJw

Length: 57min 31sec (3451 seconds)

Published: Thu May 05 2016