Parsing Event Logs for FREE by Phil Bossman

foreign [Music] how you doing so we are here talking about welcome to Powershell Summit and I'm Phil Bossman um I'm going to talk about event logs just a quick little thing about me I'm also the co-leader of the Powershell User Group in Raleigh North Carolina we meet we're pretty active and if you're looking for other stuff in Powershell that user group is there as before the meeting for the the talk we were talking about um engagement that is certainly a place to do it so if you have more questions certainly reach out I'm always available I'm on Twitter a bunch of other places as schlog that's how you pronounce that word and it's a made-up word and if you go see me after hours I can tell you the whole story about how that comes along so all right so we're talking about event logs so really Windows event logs it's how it goes the original thing where it first started way back in the day when they started off was get event log right and it does it only does the basic for um the core events that are out there application security system and setup right so those are the base stuff that most people are looking for and they're getting stuff from when they move to 2008 or two and um Powershell version three they also introduced a new thing called get win event now this is its superseder superseder to get event log okay so some stuff isn't intuitive into get event log if you're still searching online you're doing a bunch of stuff you'll see a lot of people still using get get event log and it gives you basic stuff if you want to do some call it now advanced stuff you're going to be using get win event because you can do some more stuff with it and that's really what this really talk is all about and what we're talking about get a win event you have the ability to work with the ewt in the uh actually it was there event tracing for Windows so it's this old idea where they throw their logs into different but it can be logs of many types and so where I'm talking about lists and logs so you can get all the logs that are available to you you can also get providers now providers give you a template for all the different logs that are available to you so you look at hey event 4629 inside of this you know provider well that has a format a bunch of other stuff so we're working through that and so that's all available to you then you have to in doing all these different things it's very Dynamic and that it doesn't conform to a standard right everybody who creates a an event log itself it they have their own formats that you put into and so in doing that there are um you can search by hash tables XPath we're going to talk about XPath and then actually um XML itself because it's all based in nine side of XML now kind of as a precursor to that we're going to talk about um the events themselves and I did this kind of like split screen kind of thing because most people see the stuff on the bottom right and that's how they work with the events hey I'm scrolling through the event logs and I see all these messages and stuff like that but really what I want to impart on you is that if you kind of look through that process you see that it's actually XML and behind the scenes well if you want to understand this XML behind the scenes well then you can and I know Powershell XML won't work Powershell works well with XML now I can start using the power of the Powershell right so the event records and it's using you can search it by xpaths and then it's XML itself and so yeah let's see some code like so sorry that's my end of demos that's kind of everything we're going to talk about and now we're going to get into regurgitate everything we just talked about but just now in the code so we'll just close this we don't need that anymore all right so going back to to get one event so when you're working with get one event you have the option to like any other standard commandlet itself there are the properties here there is a limited number of properties that are available to you via computer name you have this idea of the newest so you can say give me the newest five seven twelve let's still get one event okay I'm gonna walk through that process so after before you can search by username even type whether it's errors information warnings and then it's Source it's kind of like that provider thing that we talked about before and then you could actually search put in the actual string for the for the message itself um so so I'm going to kind of shorten this up because then you can see more of the console versus looking through it all so you see we just got the entire application log so if you look in the event log itself I'm going to play with that it's this whole list that Scrolls down so we could scroll the whole thing but now I got to click in each one get them same thing same thing up in Powershell itself I Can See it'll give me the first 20. same kind of thing this is all Powershell it's all um using the power of the pipeline of course power the pipeline and here we go we're gonna like we're gonna start doing some work with it let's go get me all the group basically Group by the sources and say hey how many of what do we have right so we have this is very active in the in uh it's writing a whole bunch of stuff when our M isn't doing much in the event logs itself hey there are user 32 events and what are those and we'll talk about some of those later but it's actually created in on this machine 20 of them over the past long time oh there it is I think we were talking about it so we can also search because we kind of just did it from there right but then again we're talking Powershell so we can go and now we see before is that we only have a few comp columns available to you just like in everything else in Powershell is there is display sets so it only gives you the parts of the pieces big enough that you can see but I think we can still blow that up for each one so it does give you the event ID then the computer index that kind of thing so you can get more detail at that point but again we're still talking with talking event logs itself so we don't get much information so we're going to keep going uh we can do some Powershell magic say I can pass in partials or a a constructed property so I'm going to say hey take the date we're gonna take the last two minutes did we get any events in the past two minutes oh no let's say 20 minutes there we go last 20 minutes last time I've been sitting here for two minutes so that's why I don't get any events same thing uh same thing so just like before you can get the events and you see the the properties that are available to you one of the values kind of come back down to it one of the values of get event log is it does have the ability to tab complete for you or just do auto completion so event type you can say event information actually you know we'll do errors instead and I point that out is it's for discoverability is that you can go and get those pretty easily from the console if you're working from the console it becomes a real easy thing for you to do you don't have that with a kid win event when you're walking from that it does Port those things because it's pretty much Dynamic and so a lot of that information so we may even get to something later on where we can tap complete some of that too but that that's something you have to inject into the process yourself all right so now we go back to get one events and you can see at this point actually real quick help and log I just thought about it show and do so we did the other one and here's the other one and so you see get event log has only two ways in which to call it you basically give a computer name and you give us and talks about the list so give me all the logs myself but then there's one way to call get available so still good so use I still use it all the time especially for some old stuff but get one event is where it is the bee's knees as we call it I'll scroll up a little bit so there are multiple ways to call it so you can call it just by the log name and we'll kind of talk about that there are I point this out because it kind of works in multiple ways so if you're looking for information about the log events themselves you call it the event logs and stuff like that these two properties if you're using List log or this provider is it doesn't really bring you back any events it talks about like the metadata on one side so kind of we just talked about List log will give you all the logs that are available to you then there is this idea of providers and which providers provide are those templates we call it that then go from there and so the idea is that this provider is supported for inside of these logs so you could have multiple Log sets that are one provider and we'll see that a little bit so I point that out just know that you can see in different ways and of course specify just the provider name you can actually give it a path to the X the v d DTX file so you can export the event logs and write it to a file so you can be sending them to somewhere else if you want to do it that way and you can use get one event to actually go get the logs from that event from that export so it is possible and then it kind of talks about some filters and we kind of just talked about so filter hash tables and then coming from there so you notice in filter hash table and filter XML it doesn't really talk about which log name we're talking about because it's going to be inside the process I'm going to put it inside the filter so straight up this is exactly what we just saw but you notice it's a little bit different format we'll cancel right there and every time you have a new event it says hey here's the new provider and here's the events for that provider so we scroll up a little more and then these events use this provider these events and it's actually resetting every single time so these events are right next to each other these events are right next to each other in order but all of these came in this form all of these came in this form point that out because what details they get from different things still do the same thing we did before noting that get event log doesn't support a account but when events does so in that fashion you'd have to get when event log and then pipe it to count and select so many in that form so really at that point you're getting it on one side of the pipeline which can and cannot be different uh let's talk some more about the event themselves so the error records themselves we have some more details inside of it so just like when you're looking at an event there are additional options value to you so we have all these extra properties that really aren't available to you in event log but they are available to you in one event so the display name the level itself we'll talk about some of the levels a little bit op code version details Time created text display I'm gonna jump all around so we kind of looked before with event log and we can do this fun stuff with Powershell as well you know what I actually think I'm going to do this because I have to I have to make that go over there I might do that a bunch of other places just so we can see it all in one screen right so we have the ability because we're working Powershell because we can pass in dynamic content directly inside of the the code itself right um kind of talked about list logs so similar to what we did with event logs so what I want to do is go get me all the lists give me the list of all the logs available to me and so what being this is every single all along itself but you notice too is that here we have I want everybody who has more than a thousand before we just had this hey these are these log this base log didn't have much well this shows you all the logs and then only giving me the top 1000 or anybody has more than one thousand um and actually I don't see kind of dropped off because it's so big that you can kind of see you have more information so the logs have different types whether you're circular whether you keep them or they stay but it will slowly roll off and there are different ways this is Administrative and what kind of logs that they are so for looking at the event loans say Powershell come on over here there we go and so this is that form that which you see there are um the services themselves create whether it be an application I mean admin informational operational or not even either one so each one of these entries underneath this entire structure is basically listed in this form so you see them it's here's the root folder Dash the next folder Dash the next folder in the next folder down so that's how you can see it and then the final log is actually dropped out underneath and we still have the ability to look at the system application and security logs of course okay so let's kind of quickly look at hash tables and so yeah this is all kind of nice if we're going to do it all in the console coming like on the uh in the in the command line but one of the abilities is that we can actually take the whole thing and shrink it down into a hash table and so we can build all those these parts that we're looking for here this is what I'm looking for it's a little below so I'm just going to give it a quick example and then we'll talk through it and so all right so we now have the ability to say Hey you want application I want the MSI installer and this is the ID that I want so those are the different parts that I'm going to get and then go give me that filter so with just a simple filter hash table I gave it the log name the provider I want and then the ID same thing with this one I'm doing some filtering at the end yeah same thing so I could put it all inside the console or here I'm basically splatting the whole thing so what can you put in the hash table you can put log name start end times user ID similar to what this is kind of the direct replacement of get event log in putting it right there then you can just get the different parts you want you can give it the data which and we'll talk a little bit about that as well and we can set as individual pieces oh that's what I did with this one I do have this right yep there we go earlier before I did yeah there's the MSI gave me the first one and again if you're not really sure what code I'm writing you know feel free to ask so give me the first index of that machine so now we're working with just the individual events itself um actually I think we're gonna jump yeah kind of filtering left do this real quick server list should already be there so one of the things that when you're working with event logs overall is I want to talk to basically 5 or 10 15 servers and so you notice that if we work through Powershell as well as you typically want to filter left and what I mean by filter left is I go get me all the servers themselves and then for each server I go get me all the application logs right and I'm passing the cred for there and then I think basically after that then I think you'll give me um level two which is basically errors and then I throw it into events but actually we'll run this really quick again because it doesn't take too long but it I want you to notice that it takes longer to run then we run it again but what we're going to do is we're going to use the hash table that's why I actually talked about the hash table part okay there we go that runs a lot faster oh yeah I did measure so as an example one of the things you want to definitely do is you should be filtering left so if you go get all the events so this is that idea of just Powershell overall is that I got all the events set every single event and then I started using filtering down there oh then give me the all the errors well I could just tell the servers themselves go get me the application the logs but only give me the the errors divide these two and then pass that along so really when you're working with event logs you should be filtering left as far as possible everybody understands what that filter left means hopefully we're good with that because the idea is let those remote servers do the work for me and only give me back the pieces that I'm looking for same thing we're using doing in vote command and stuff like that you should be do all the work way over there and then bring it back to me all right so let's talk about so we've talked about hash filters and so now let's talk about XML filters and similar just like that you're like what the hell did you just get or what the heck did you just get so the idea let's let's jump back out that was my little note there see jump into the event log so let's do system at this point so if you're using the Event Viewer normally today and people work with the GUI itself you you may or may not know that you can filter the valve or kind of see what people do at this point and understanding that hey I have this nice GUI where I can click things and I can say oh give me this event ID and 74 should be and oh no that was the wrong one I think it's it's application anyways and 74. I'm still not that doesn't matter so we just say hey we're going to filter criticals and errors oh there's all the events I got but what do you really do so if we come back into filter our logs you have these things here called XML so this will build you the XML for you really use it and you can come down here in the bottom right here unless excuse me and click on edit the query manually manually and you can start messing with these values and say I don't want this one I want number three too and it's the same thing and there's the exact same filter so long as the filter passes it will go but the idea really is that you can use the GUI to build this yeah yeah yeah can use the GUI to build it it's a specific time and we'll just do that [Music] that's good enough we're back to here okay so we're working in here and we can build these filters right that's the the gist of what we're looking for right so it will give you this idea and you're like hey look at these words you know there's some other paths so I'll look at that so that's where we come along with this well once when I first started playing with I'm like okay so now I can build these just the hair string just build big XML XML it's an XML is my system there's my system and there's this kind of funky thing right here but we'll talk about what that is that's actually uh the next PATH it's an x-pass query so but it's just a just a random string I can play with strings I know exactly what to do with strings and I can do some funky funky stuff with it like all right well I'm gonna pull the the log name out so I can make it a variable eyes I can take my ID down you know what I'm gonna do the days thing we kind of chose you know how many how far back do you want to go well it's Unique too is that the XPath itself with a log it wants to use milliseconds so whereas less than so many milliseconds so you need to then again Powershell is pretty easy for you I know how to make milliseconds out of just a number how many days I can give you how many milliseconds in days well I'll just take that many days minus the current today and I just get a transient so this is just a simple as a time span so you can do new type spanner all that kind of stuff but uh that works from there so that'll give me the number of milliseconds I'm working with and same thing we're going to do the exact same thing application 10 24 I think it was 10 24. and there's those logs we're looking for as well that's what I was looking for I knew they were in there so 10 24. so we work from there so once you have an XML you can then do some funky magic to it um one of the things we've all I've also done in the past is taking this information and actually made it real Dynamic and actually put inside of a loop and so I was changing the information as I went through the loop and give me all the different parts I wanted so you can start doing some really wrapping um I wasn't able to really get some real complicated stuff because I can't work my uh in my prod environment for this and I couldn't really build a lab for that big just wasn't feasible all right so let's talk about XPath is this idea of we kind of talked before is if you're using XPath itself well then you can still you don't need to actually give it um well you can give it the XML give it a long you want but then if you just talk about the X the XPath itself you can say give me the X path okay how do they calculus then you know what my skirt because I rebuilt part of that you just said 24. we already know what you just did that well I think on our feet so there's those there there's the events we just kind of talked about at that point so the exact same um filter that we just did but now it's just in this form so you say well why would that help me well some people you know it's Powershell you can do a lot of things in a different ways so it's not one of these things where this is the set Zone way you don't have to do it you can certainly do it multiple ways um one of the other things is the the log name is not required and so what I've actually done many times is simple like you'll give me all the events across the environment you know so one of the things that that in Juju there's that that view I don't know if you you're sure there is that view of go get me all the holy mackerel it's not on here I mean I think to check it oh you know what it is this I think this filter gives me this is all work warnings errors and everything so that's the administrative events but still this this filter is that we can look inside and see what the filter is filter The View there you go so this is what that filter is looking for it just goes and gets the different parts so you can actually put a bottom together but I don't typically do that I just see all I really care about and I Define the XPath but then notice I don't give it a along today so this will take a short period of time but we can certainly do it so what I want to do oh that's why I did shorten it down because it was taking too long and I was scared about that so this gives me an output review of all the events across all the providers there are okay come on I can't scroll up um shows me all the events across the environment oh that's not what I'm looking for looking forward oh I was just in the wrong field that's why I was just looking at the wrong place come on so you can see that this one is an SMTP SMB client and then these are all the other errors across the environment so you can actually group them down you kind of go from there you can do algorith view to find your filters at that point so in doing this you can really easily query your environments on remote machines and we kind of saw that before um and then how do you build a quilt which I've talked about that use the event log itself use the not the event you then viewer to actually built these parts I think it's fine it's simple some people you know build a bunch of other stuff but it's good for that so now we're kind of talking about Gathering a bunch of information I want these filters I want those things we kind of talked all about the filters but now we're going to talk about the events themselves so it doesn't really matter I've actually not seen it I've never tried to Think Through Wonderland because ultimately it all dumps it back out to XML so like if you're doing games repair yeah so effectively what he was saying is that the the if you do just XPath itself you have to only do one filter that is you have to construct this XML filter path but the idea is that if you build an XML itself and you kind of saw it's actually it was a really good example in that this the marginal one is that the actual the XML itself it is actually querying multiple different sets there's a there is an expat here another one there another one there another one all for the individual Parts themselves yeah yeah but you you know you go as deep as you want to go at this point right yeah and that's it yeah yeah actually you know I think I might actually have some examples of some pretty funky ones but I think we'll get there if we get there and I wasn't sure if we'd get there um so let's talk about the events themselves so we kind of talked about some pretty easy stuff same hash filter itself just because it's more it's easier to visualize right so long name levels IDs the start time again we're just using something dynamic and provider path right and so what this gives me back is this is is a single event so we're going to look at that single event right you know the message that was just there and then we'll talk a little bit about we'll get eventually get to the message but there are individual parts to it right and then we have this thing down here called properties like what is that well that's all the other different parts we can get back out so we're going to run the same thing again we're going to actually split apart the properties I'm going to select the properties I'm going to put it all back together so just just give me back just these properties well lo and behold it's an array itself right that's not what I was looking for oh there it is gotta keep on scrolling together so the idea is those individual properties they are broken apart so what I've done is they basically you know pivot chart pivot table I've pivoted those these these those properties and made a ninja property itself from them actually that doesn't look very nice let's see if we can review again and as you kind of know I love awkward view because it just makes it things make it easier and so all those individual parts but those are the the details those are the details and I say that this is a really good example but uh we'll go from there oh this is one this one's better so the properties themselves it is it is a prop they are there is this thing called properties right but there's also a method like I said before that's that's what the what's inside that single record is just an array of those properties but I don't really want to I want to do some other stuff with it right I actually realized that it has a method called 2xml and so I'm going to take those events itself oh game again right so we have our event only just take me just do it this way right so it's an event right well that doesn't really help me but it's XML so I get the different parts I can say oh give me this this is what's inside the system section right so I have my date graded I have all these parts but I know how to work with that and the event data oh all right so it's just a simple XML that we can play with so those different parts yeah this is doesn't have much data inside of it another event I don't know why what happened to my logon events I don't know why it doesn't seem my login events foreign like I'm logged in what's wrong with this I'll skip over that because that's this is the parts we're looking for actually let's go look for it yeah I think on our feet here 46 24. you know what that's kind of fun let's look at it forward get to an event is it raining of course I don't have the ability to um all right still right yeah I'm gonna try but don't play with that one and 46.73 let's see we're gonna think on our feet here and try and figure out what we've got inside of this let's see what we get inside we still have events what's inside these events hey look at this this well I have the same property so effectively let's go find out what that event looks like just so we can kind of see it 46.73 wait what did we try this thing yeah it's gonna be hard to see inside of here so most people like look inside of it what is it then so the idea is that this message is um this is what we typically see and if you see some people writing a bunch of code like hey I'm going to search the event log and I'm looking for this data I want to change I want the all the events where this is the user I'm looking for well to try and write this regex to try and match this together this message that just comes really painful but again we have a bunch of other good stuff here there's details this looks pretty familiar this is that this is like an easy way to find it this I just want this property from it or again we're talking XML so this is what gets generated back out and we kind of saw we have an event we have system we have XML data and then we have property from there so in doing that I then want to build an object of those different parts service I'm going to change it real quick because we changed a bunch of stuff process name almost process ID no we did have a process name okay processed so what do we have we have zero and zero uh two and four and three good let's build this really fast and yeah it's going to be a problem with the same thing over and over again privilege process see if I can get a better better vent as maybe come back to it or if you guys are interested well maybe come back too let's see what else we have to do that event list do the same kind of thing I'll give you no this is other stuff that it was going to get to yeah so in a in a larger environment and I wasn't able to set up an entire like Citrix environment we're looking for but you can go the idea give me the username I'm going to give somebody a user Sid so I'm gonna go pull out the Sid we then come down here and we're going to use XPath to give me that data so this is kind of a funky X path we can kind of see so we see we got our system here's the event logs the event IDs for logins and the individual part is this is what that XPath filter looks like and so to actually get back where the Sid is in the right place so we're in here Sid and that's what it looks like so this event then data then its name is Sid and that's the part so if you're going to do the filter again send it over to the remote machine have it just give you the data that it's looking for so what I want to send it to all you know 400 citric servers I want just for this user well I give it a user ID it then spits out you know gives me a sid because that's what's in the event log itself and it goes from there and can go from there so this is that example then from there we can then take those events create an object that is the login events and username computer when it was created and whether it was a log Auto log off event so I don't think I did there trying to see if we had some other examples but we'll go we'll keep going all right so when you're working with events always remember drop it into an XML I think that's the easiest thing there is a little bit of generating an XML over and over and over again but uh it is from there so we kind of talked about I run out of time pretty soon but we're gonna get out of here pretty soon so we talked about providers this idea of what a provider is is so here's the first let me give me the first 50. I'm working with some details of it oh let's say finish still so go get me all the providers yeah there we go that's the one I'm looking for sorry so these different providers this is Group Policy but this provider has a form that is supported inside the system and the group policy operational uh log itself and so why is that important well it's important because if you then go look at the group policy these are all the formats that you can have these are all the template messages that you can use and so if there's an idea that hey I want to go and find all of the there's a policy yeah I had another one Group Policy stuff and you don't know what the the the login is going to be see if it's inside of here Orange provider that's you must say security hey that's what I'm looking for so here we go try to rush this through again this is a good example of what we're going through so give me the provider I want all the security auditing events and so if it has events and then I want the first 200 it doesn't really care but then um get me the events themselves select out just the ID and the message but then everywhere where the message says delete well why is that important well hey if I'm looking for you know some logs but I'm looking for a particular event so hey you're going to use something to find the events themselves this is a great way to find it because then like I didn't know that the 4060 is the one we're looking for but if you're looking for user account deleted well that's 47 47 26 and if but 4743 is the computer ones so this is an easy way for you to then globally find other stuff in those things so you can find the provider you're looking for and then get them from there so what's inside of this overall too we'll say for this point actually we don't want that It's like Star it almost like tea the object there you go details really fast really fast okay okay a lot of details all right so what's inside of this thing all right just pick one I don't know what was that one we've been looking for wow does anybody remember that number 46 47 26 . equals 47 26. yep there we go so user count deleted this kind of looks familiar this looks just like the message but we have a bunch of these variables and up this looks like the event data that we're looking for here's all the different parts and so you then can find the message and then know that here's the parts that we're looking for so without actually seeing a message itself you can look at the providers and get the individual parts from it and so this is the XML template it uses to build this message for any event of that type so if it's using this provider and then using this ID so that's what the the event triggering process is doing here's the data for this event ID in this provider and then it just gets written in that form um I think I'm running out of time yep we're running out of time sorry we're going from there I think I was rambling for a while but the idea is that you can use your list provider to get you other details about the metadata for event logs themselves you always should be filtering left when you're trying to do stuff use the multiple filters you can hash tables are really easy and easy to construct and easily visible but if you really want to get into the weeds XPath and XML filters are your friend and powershells are friends you saw me here I'm just pipe this the pipe that pipe that and it's easily discoverable that's exactly what Powershell is all about and uh hopefully you can really get some more information out of the event logs themselves and go from there thank you very much [Applause]

Channel: PowerShell.org

Views: 587

Keywords: powershell, windows powershell, techsession, powershell summit

Id: VgXtdM_j4nU

Length: 45min 58sec (2758 seconds)

Published: Wed Jun 28 2023