Oracle Weblogic 12c Thread Monitoring

hi everyone my name is PA Charbonneau I'm a senior IT consultant and the other on Java support pattern blog baulkham today's tutorial will be basically demonstrating new the oracle weblogic 12c trade monitoring capabilities i know some of my readers have been asking me for this type of training so that's what we're going to shoot today I'm going to show you a couple of tips and tricks how you can use the WebLogic console to monitor the JVM tread more precisely the WebLogic trail pool which is very crucial if your web logic administrator so I'm going to demonstrate you how you can monitor the hugging tread and also the stock trading the console so I really hope that you enjoyed this tutorial so in order to understand the tread monitoring what we did we created the local WebLogic 12c environment and we created a simple java program and with the zero program we're going to demonstrate how you can monitor the the treads in WebLogic 2oc and more precisely how you can troubleshoot hugging treads and structure problems which is a very common problem if your web logic administrator so as you can see here is the WebLogic - OC administration console so i'm going to log on to the console now as administrator you're probably quite familiar with this so we created a very simple domain it's called pH training in term of servers if you go into the environment servers you can see that we have only one server which is the admin server so in this case we're using the admin server also since we're pretty much in development mode we're going to deploy this application to the admin server meaning that the actual console and the application is available that same instance ok so in order to to monitor the actual threads you need to go as as I just did under server then you select your server which is it could be managed over one month over two and then you go under monitoring and against there's a treads tab right here and this is where you will see their cross sections to monitor so we'll start with this approach because it's very important you understand and because I sometime a lot of people when I see the when I review some team or some clients they are not taking advantage of this especially for trying if you're dealing with a live problem it's important you understand this so you can see here then you have first you have a summary right of the situation so active if you probably saw my other articles when I was describing this the active executed treads as you can see it's a it's a little confusing if you look at all of this right but the way to read this is pretty much like this this is the execute read total this is a total number of tread created by WebLogic right because that's a self tuning thread pool which will grow and shrink depending of your application workload okay so the execute is pretty much the total number of tread that WebLogic created and then depending of the traffic you may have some active treads and idle track right so this one is the total number of active meaning that you have 24 tread total created out of 24 23 or active one thread is in standby mode meaning that it's not ready to to process yet right so WebLogic will create thread will promote them to active state if any of your workload it may decide to put some of trajan standby if the load is too low for example and then out of 23 active trail you have 21 which is idle I don't mean you have 21 treads available to process transaction okay and the reason you have 21 idol is because if you scroll down you will see a thread for and thread 6 we try which were actually executing a request at the time we did a refresh of the page right because this this trade you can see the console app so just using the console when I was refreshing this page was using the trend number 6 and then because the console is pulling the data from the gmx server then it triggered another call another thread which is our mi connection in this case alright so you have 2 treads executing a request in this case so which is why you get 20 treat treads -21 write which is these two tread so when you look at this portion typically the contour that you care about or this one the idle thread count-- right also the cue lamp but more importantly the hugging thread can ok the hugging thread couchy hnu it's a new counter that was introduced by WebLogic about version 9 so quite quite a while now and you will see this disturbed mentoring look and feel it's quite similar since version 9 and 10 so it's not something new as per 12c the major improvements as we will see like they are keeping track let's see of the start time of the trend as we can see here this is fairly recent what overall the look and feel is is is pretty much similar so this is the summary tab and this is where it's get useful because each thread you will get the the tread name which is actually get the actual ID of the trail total request executed by this thread then you get the current request which is for example it could be an HTTP request in this case or in our michael and then you get all the different contours if it's idle right for this counter stock and hugger very important right a hugging shredder or hugger simply means a tread taking a bit of time doing always the same thing right so let's say you have a tread which is running since about 10 seconds doing the same task for 10 seconds so WebLogic will promote it - true hugger true and then depending how much elapsed time that shred is accusing the same request right again the same request then we promote it to stuck what stuck is a configurable time so in this environment what we did is this if I go back to the server there is a tuning time and you get a stuck thread max time in our case the default is 600 seconds or 10 minutes so meaning it WebLogic at some point after 10 minutes we'll identify a trade our stock right and then we'll print out in the actual log the full stack trace of that trend okay it's very useful mechanism when you're troubleshooting problems if you're dealing with stock threads I if you're using work manager WebLogic may also take some decision let's see if they take stock tried to turn off the work manager but we're not going to talk about this right now in this scenario as you can see we did reduce it to 60 seconds because we're going to demonstrate also start red as well okay so let me go back to the monitoring and then we'll get started with the simulation so in order to do this simulation what we did we created a simple program as you can see this is this is the Eclipse or the Enterprise version for for WebLogic so as we can see we create a very simple spring MVC application with two controllers one which is called hugging thread controller and one stuck thread controller each of them is associated with a JSP file for the rendering of the page okay so what the hugging thread controller does is it's very straightforward it's same to execute that method hugging trial and then it does at read sleep inside a sink and synchronize block thread sleep for 30 seconds so the goal is to replicate hugging threads so you were able to see in the console I can create an action you see it's always best to learn with a simple example before you go to more advanced problems now if you look at the stock treads very same logic but we do a tread sleep with no synchronize in this case but we do it for 90 seconds remember right we set up the stock max tread time at 60 seconds so with that controller taking 90 seconds to execute the goal is to replicate a structured problem okay so let's get started with the actual simulation okay so I'm going to execute a request as you can see that that was from a past request I'm going to retype and refresh this as you can see it's still loading because we put that thread sleep now if we go to the actual page let's see here you see I did two requests you see one side it's in seven seconds six seconds if I do a refresh you see not that same request 18 seconds 16 seconds now if you look at the open application auto-generated your file from Java reaches my URI of this sample application the GFI is auto-generated because we're using web logic was started from eclipse so in this case it's using auto-generated name that's why you see this as an application name but this is essentially our sample application this counter is set to true you see I get read true right because remember that the program is starting is sleeping for 30 seconds and this is why so this is why it's very useful to mitre because when you're starting when you're getting to some sort of situation you're able to do a live monitoring and you able to trace down that sing these trailer in this case thread 7 & 8 taking too much time I put the Refresh obviously these two are gone so let me send a few more requests and you can see now we have three threads right or for trade in this case six again six existing requests now the other function as you can use a dumb tread stack right away so this function if you're from your with it is essentially a Gerald read them then you can do directly from the console okay so that's a facility in WebLogic it's it's it's quite useful i generate the tread um gmx format so it's really nice when you're dealing with tread like this you want to do a quick analysis if you look at this thread right 22 this thread is a block state it's waiting at this line you see this is why it's interesting right hugging thread this is exactly our controller right pH at line 37 it's blocked now let me go at this line in the code you see 937 you see so that trail it's pretty much blocked here okay and then you see this red you will see it sleeping 18 this this one is in time waiting right because that red is sleeping because we're waiting for 30 seconds and if you look at the other treads you see you will see some of them also in the block state why are they in a block State well this is where it is a limitation of this trade Dundee the tread down from the WebLogic console is not and it's not necessarily native tread emits it's not giving you the native information on the tread so you're not getting a detail like which object monitor it's locking it this is why we wanted to show that to you because you need to understand the limitation the reason it's in block state is because it's we are allowing only one tread right because what we did we created a static lock or a semaphore right and then we're synchronizing on the static lock so we're pretty much allowing only one thread at the time to sleep to this method so in this case it means one thread will sleep and the other request will queue up right here right and wait for it as an object lock on the monitor which is which will be held by the note read but as you can see you be able to see this in a trade down from the console so you need to use a native trade dump as you can see right this is why you see this one is the black state so let me go back to here we're going to do we're going to show you how you can do this very quickly like to send a few more requests so ever it is will also fire up G visual VM which is different clear tool that I recommend then you can definitely generate at red dump from J visual VM and in this case is going to be a native trader so we need a quick assessment it is good to use the console if you if you detect some blocked red and you need to worry about object monitor a lot then use a tool like G visual VM or if you obviously if you're using a unique space like Linux or Solaris then just do a kill - tree of the PID right and that will give you this information you see the native ID plus the lock you see trade is block as we saw from the console but now we know what is waiting to lock right waiting to lock so we know that trade is waiting to lock on another thread right just dis synchronize so if I continue this guy also you see three five eight you see if I same object because we have a global lock here this is our guy I see so tread 17 is in sleep mode but before going to sleep mode did acquire a lock on the object three five eight this is why and this one is in time waiting because the thread is sleeping this so this is why you see the other trade are waiting to acquire lock right but we couldn't see that in the tread down from the console so again I sister shoe just the limitation just keep that in mind when you're dealing with tread lock on tension always use monitor analyze the native tread down okay so now back to the program I'm going to demonstrate you now a stock trade remember will create another controller that we call stock tread right so just like this going to execute the request a few I remember the stock credit we'll be sleeping for 90 seconds so we'll be able to trace down these trails so I generate quite a few requests as you can see right I think red is already tree and remember we did set up our stocked red max time at 60 seconds so it will take about 60 or 65 seconds before WebLogic is able to identify our shred at stuck and I'm going to clear the actual screen so I can show you what WebLogic is going to do after the it detects that trailer in stock state it gets in out about 30 seconds you see it's very useful to to track these red light like this at some point you will see you just follow up the contour right another refresh or were closing in about 45 seconds 50 seconds you see so in if you're using a real-world application you're able to monitor also your application trade using exact like this oh there is we have some trend now you see stuck true you see that now definitely have some trading stock date and notice the state here you see very important to realize that WebLogic once it detects a stock trade and I'm going to generate another trade dump while this trade or actually stuck right here okay there it is and then I'm going to explain you I quickly this principle so as you can see when you see these threads you'll see them in active state right but as soon as well the stock trade is ritu a branch it will update the state in stock this is part of a trend name okay so what it means that when you investigated a tread dump for from a WebLogic environment it's always good that you search for stock because since WebLogic is ablating this to the tread name it means that stock will show up in the so if you sow if you do have any stock tread these will show up in the tread dump and when the trend name so this is really a nice add-on by WebLogic even though the tread is healthy from a deviant perspective what logic is able to detect its stock and to sharpen a tread them we can see it here right the trader moo generated from G visual VM you can so if you have them or enacted look at this you see this is a negative tread them that we generated from zero console right this guy is sleeping for 90 seconds so beyond the stock tread max time limit that we can figure and WebLogic will automatically update the trend name right here so the idea remains the same but instead of active you see stuck so it's very useful because once you see this you will know since how long the tread is is executing right it's even doing a root cause analysis after an incident in addition of this let's say you don't have time to take a tread dump which quite often I see it could happen well WebLogic another good thing is our Blodgett is it will print out at least the tread stack trace in the log for you so this is very useful if you don't have any monitoring tool or any mechanism I co2 generated tread dump so at least you'll get a trace if you're dealing with stuck treads as you can see here right 6:04 WebLogic was able to detect that tread 22 right the one we're looking at has been working this request for sixty five seconds is you see so in this case showing us the print stack trace and as long as you have the stack trace then you're able to know which execution path this trade is doing and in this case it's exposing our program code which is doing a sleep from the stock tread controller just precisely what we wanted to demonstrate here ok so again use the console as a starting point for live monitoring for historical data use a monitoring tool of your choice there's a lot of commercial products out there and open-source product as well like gee visual VM and again very important you understand that trend monitoring time so you start monitoring and then when you're more comfortable start to drill down into the tread dump analysis as we explained start with the console like this dumb tread stack and then when you're dealing with more deeper tread log type of problem used to like jiggle VM or kill - tree - kill - tree generate multiple snapshots about five seconds in each other and then at some point you'll get more experience and you'll be able to that type of analysis very quickly okay so I hope that you appreciate this WebLogic trend monitoring training today please feel free to post any comment you will also find reference to the articles from my blog below so please feel free to to post any question ok so thank you and have a good day

Channel: Pierre-Hugues Charbonneau

Views: 49,406

Keywords: Java (programming Language), java.lang.NullPointerException, Hosting, Java (software Platform), Software, Technology, Computer, Tutorial, Windows, Web Application Monitoring, Network Monitoring, Java Monitoring, Oracle Weblogic Server, Oracle Corporation (UoA Vendor)

Id: Ta9fyS_VMA8

Length: 20min 33sec (1233 seconds)

Published: Mon Oct 07 2013