Understanding Office 365 Audit Logging

good day everyone Randy Franklin Smith here and today we're talking about auditing the cloud in particular the office 365 cloud and what they refer to as the unified audit log so logarithm are the folks that we want to thank for making today's real training for free possible logarithm sponsors a lot of these awesome training topics and today I have with me there office 365 expert you may not really like being referred to as that because I know he's been through the wringer with getting office 365 auditing working but let me introduce you to Bruce and Bruce thanks for making today's real training for free possible yeah thanks for having me ready let's see here so we'll get started off at the top and just discuss number one which applications in office 365 do you have the capability of auditing and then what can we add it inside those apps it comes down to basically three areas of activity and that is administrative or privileged user access what are your admins doing inside of office 365 what are your end-users doing in terms of what information are they accessing and sharing and looking at and changing and then third on there's also some authentication and logon activity that we can get from the unified audit block so next we will look at what does it take to turn this on and then how do we get this audit data out and I'll show you both the portal where you can search interactively I'll show you a PowerShell command called search unified audit log and talk about the management activity API and how to get this information into your sim and that's especially where Bruce comes in logarithm is putting a big investment into getting this data out of office 365 and into their normalized class of class you have classified not in the government sense Bruce but you have a common Oh what do you call it anyway yeah we got our our set of classifications common events and a couple dozen other metadata fields that are standardized between multiple log sources so I'll show that in my demo and how you can not only leverage the office 365 data but other data that you pulled in from various log sources that can be correlated together through these common metadata fields yeah and it's just so powerful because then what it means is once you have an event in logarithm if it's a log on event it's a log on event whether it came from office 365 or your Cisco router right so good stuff coming and yes today's webinar is being recorded and will automatically send out a link to that so let's get started here's the current list of applications you can audit in office 365 number one as your ad then you also get SharePoint which includes onedrive for business for the two or three of you using it no just kidding on exchange online swea also any of your discovery activities the searches that you do for your discovery those get audited now they are auditing what users do in power bi as well as Yammer now let's talk about the top three there so here is the typical environment of an organization using office 365 whether you realize it or not you're also using Azure Active Directory now whether you're using other Azure services like you know VMs and cloud storage and all the other exotic cloud services that a juror offers doesn't really play into it because the activities in there don't come into the unified audit lock so if you provision a VM or stand up a storage account okay that that's not coming in here to the unified auto block but what you do as far as users and groups and stuff like that and as your Active Directory yet that is part of the office 365 unified audit lock so when you create a user account in office 365 just want to make sure everyone understands you're not you're not there's no such thing really as an office 365 user account it's an azure active directory user account and most organizations I would say I would say most organizations are syncing between their on-premise Active Directory and Azure Active Directory it used to be with der sync now it's with fig it's called AD connector as rated Connect we'll look at that in a coming slide and then of course all of these different applications in office 365 whether it's exchange or SharePoint or onedrive or Yammer or stuff like that that that activity that's what we're talking about in today's webinar all of that goes into the unified on it log so naturally we've got user account maintenance we've got group maintenance also you can delegate admin authority in Active Directory that is an azure active directory so just like in an on-prem ad you can like delegate authority over a given organizational unit or give the helpdesk reset password authority over user accounts that sort of thing delegation all right you've got the same capability in a well a similar capability that is in Azure Active Directory and when you do that an audit trail is produced which is good and proper now some of the other stuff that gets audited in Azure ad has to do with the cloudy aspect of this this this flavor of Active Directory so one of the things that we do is we we like to integrate other applications maybe not even from Microsoft to rely on Active Directory for instance at our organization our Dropbox for business account uses Active Directory for authentication so when we log on when we access Dropbox you're actually logging on with our azure ad account so that is another thing that's audited and that's good because you certainly want to have a lot of trail if you suddenly integrate another application and make it dependent upon ad likewise the DNS domains that your Azure ad is connected with partners do you have consultants that you are granting access to your ad that that would be audited here as well stuff you're doing in terms of Federation if you change log on policies and add rating you know for instance you can control password requirements just like you can on Prem ad all of this stuff also is audited and you'll find it in the unified audit block my as a question as your ad can be used to drive authentication for Dropbox for business Mike yes answer is absolutely there's something like 150 other applications that have got integration it is facilitated by that vendor or by by Microsoft so yeah you can tie a lot of stuff in to Azure 88 so it you know it's it's a nice model in terms of you probably already have a non-prime ad once you start synchronizing up to the cloud up to Azure ad Y then you can pretty easily connect to lots of other applications out there um now there's third-party software that makes it even easier but it's this good support right out of the box for quite a few applications and there's nothing to do but basically turned on and copy and paste a couple keys but of course that's not the focus of what we're talking about today although you know it is important then to point out that once you connect up to stuff like Dropbox why then you will get a little more in your audit log because Azure ad does monitor or does audit logon events and so now you know the more you tie in via Federation to the same directory then the more centralization you have not just over policy but also over your monitoring let's see here so John I'm I'm not sure on your question there I don't keep up too well with what's an azure basic or the the free version the smoke free version of azure ad let's see here but moving on let's leave as our ad for a minute and let's now talk about Exchange Online so if you're familiar with what you can on it in exchange on premise then you have a pretty good idea of what you can audit with the office 365 unified audit log so let's start with privileged users for compliance and just for good security we need to have an audit trail of what administrators are doing this is a deterrent slash detective control over admins but it's also very important for being able to detect when admin accounts are taken over so here in exchange online just like with exchange on print anything that administrator does is ultimately resolved down to a PowerShell command so any kind of administration whether you give it through a GUI or whether you do it through the exchange administrative center a web-based portal at the end of the day it's a PowerShell command that's being run and that's the basis for the auditing here when we look at the admin audit log it's basically a record of all of the non read-only PowerShell commands that were executed with their parameters and everything else who did it and so on and it works great so if you've attended any of my webinars on exchange auditing then you basically have the same capabilities here with Exchange Online let's see here and keep an eye on these questions I'll come back and get all of these answered I might hold off on a couple of them for right now let's see here now what about end-user access in exchange well we have to begin with the very same events that you have with exchange on Prem so what this is mostly for is having an audit trail of non-owner access to other people's mailboxes so who's looking at a CEOs mailbox that is one of the most or other c-level people that is the most frequent use case that people come to me where the exchange mailbox audit log comes into play and so that's that's there it's a matter of auditing this message right here folder bind there's also one called message bind but unfortunately it doesn't get audited as much so the one that you really can look for is folder by and that tells you whenever a given user looks at a specified folder in somebody else's mailbox now this also gets logged incidentally if when I user adds somebody else's mailbox to their Outlook installation then whenever outlook synchronizes you're going to see that information the same thing but it doesn't mean that the user consciously went over there and looked at the data in that folder it's just a matter of their Outlook synchronizing to it so that that's just something to be aware of but the other thing we do get is some message tracking capability for instance as I'll show you I just realized that Exchange Online is also auditing whatever I create stuff in my own mailbox but I think that's because I've turned on all auditing even for ownership so it's not really message tracking it's still mailbox audit events it's just a matter of do you turn have you turned on auditing for the actual owner of course that's going to create a lot more information right okay so I'll come back I'm going to show you some examples of these events but right now I just want to give you a an introduction to all the applications the type of activity that's captured so we'll come back to exchange let's though talk about sharepoint online and onedrive for business so how many of you are familiar with sharepoint on-prem auditing so the sharepoint on-premise audit lock well if you are familiar with it and this is the shear point on prem audit log then forget what you know because there's nothing in common with that so you know whereas the exchange audit logs are very similar in office 365 in sharepoint they're completely different and it's a good thing because the raw audit log of exchange on prem actually was not useful there was just all of the really important data fields were just codes that you had no way to translate and so that's good that they didn't use that approach with sharepoint online now before we even talk about what you can audit in there it's important to know that onedrive for business is essentially sharepoint online and so the audit log that you have for sharepoint online and onedrive for business this has nothing to do with onedrive the consumer version not a log is basically the same you do have the ability to track every operation on files so every access every download when a file is checked in checked out you know copy it whatever even with you you create sharing links you know how you can take something in onedrive or Dropbox and make a link and email that link to people that gets audited and in terms of SharePoint itself there's some administrative activity that gets audited as well such as if you add new admin to a site collection or in a SharePoint has its own groups that are local to SharePoint have nothing to do with Active Directory groups so you can audit for instance membership changes on those groups that's important because that's going to be that's that's representative of granting access to to SharePoint data like I said will show you real examples of this data here in a second now those are the three applications that I'm going to talk about today how do you turn this on before you can even start getting this data the very first thing you have to do is find this link and if you've already turned on the link won't be there once you turn on the link goes away but it's here it's in fact let me take you there let's see here going to bring up my there we go here we go so if we come to audit log search I'm in fact where am I actually I'm here in office 365 under admin and then we get to log back in okay and then here under security and compliance bruce'll of the blinding speed of office 365 let's see here am i d-- i'm logged into the right place this is not exactly what I'm looking for folks admins there we go security and compliance and sir investigation audit log search okay so if this were a new tenant where we had not turned on auditing that's where we'd find that link that I was referring to earlier and you've got to turn that on and then it's not instantaneous it's going to be a maybe a good 24 hours before you really start seeing data come in and that by the way is a real factor here Bruce interested in your thoughts on it as well but that's a real factor is latency here if you're accustomed to like the windows security log where I mean it's by the by the time you go look in the security log after you perform it a bit the data is there but it doesn't work that way with the unified outline does it groups it doesn't you know sometimes data is there right away I've seen delays up to three six hours for some events coming in you know sign-on failed logins and the timestamps are all correct but it just won't be available through the audit log search or the API search for you know some indeterminate time period right something you hear Microsoft people saying is well remember this is the cloud so I think you're going to see though that you're going to hear that term used as a justification for lots of things and one of them is the latency in Autobots but here we are in the auto log search I've turned it on now is that all we have to do now there's a little more to it than that as I'll show you here in a second see here I can get back to my slides so once we turn that on we will start getting there's nothing else to enable in order to get auditing from like as your ad it's a it's an all-or-nothing thing it's it's going to audit everything for Azure ad and the same way for sharepoint online and onedrive for business it's once you flip that switch in the portal you're auditing everything that you don't need to go to individual that's not a big difference by the way with between SharePoint on-prem auditing and SharePoint Online on it or SharePoint on-prem oddity you had to go to each site collection and turn on are whereas with the cloud it's just a sister switch turned on for your office 365 tenant and now everything every action to every document and onedrive for business or SharePoint Online will get audited and with exchange that's a little bit different you have to go into PowerShell and turn on auditing both of the admin audit log that's with this command right here the set admin audit log config command and then for mailbox auditing for each and every mailbox you've got to run the set mailbox command and then specify the audit delegate be and the audit admin good one you want to make sure is that you've turned on auditing for admin level activities and this is just like exchange on Prem and of course you want to make sure that audit enabled is turned on how do you get a PowerShell session on office 365 just do a search to see the way to do it but it's pretty easy the very first thing you've got to do is set your execution policy then you've got to create a credential so when you run that command right there it's going to pop up a dialog and you'll enter in your username and password it gets stored in this little variable called dollar you see and then you say okay I want to open up a new powershell session and it's going to be connected to outlook.com our shell so what you're telling it is hey cut this powershell session to office 365 and here's my user credential and then it it connects up then the final thing that you need to do is tell it to import a session yeah right here import powershell session once you do that now it's as though you are while you are you're running a PowerShell session command prompt on on the office 365 environment with access to your tenant so then you can that's at the point at which you can configure auditing for your admin audit log so for instance the command you see right here is saying enable the admin autoblog and i want to audit all commandlets and all parameters now if you want more information on the admin audit log and all the stuff that you can track just go to my website ultimate Windows security comm and I've got an entire section devoted to exchange auditing both the admin on log and the mailbox audit log and it's all the nice thing is as far as Exchange Online is concerned it's it's all basically the same as exchange on Prem okay so this is whoops moderator thank you so this is also where we would turn on mailbox auditing and remember the big thing there is you've got to turn on for each and every mailbox one at a time there's no way to set a global auto policy for all mailboxes so this is convenient if there's just a few mailboxes that you want to audit you know simply a matter of turning it off for them you can leave it turned off for all the rest on the other hand if it's everybody then you're going to have to look at some kind of strict incapability okay so once we've done that auditing is coming in we let a day go by to where events are really pouring in how do we get at those logs one place you'll probably start is with the the portal there with me here and I there we go and here's the portal and so we can select which actions we want to audit now one of the first ones that I start off with just because I'm asked about it so much is being able to see who's looking at the CEOs mailbox now interestingly that one is not directly addressed by your options here if we come down to exchange mailbox activities notice that there's nothing in here for users accessing somebody else's mailbox or viewing the mailbox now I happen to know that getting that this event that we're talking about is called folder but I just I happen to know that by virtue of exchange on-prem on it so instead what I'm going to do is I'm going to see if we can just search on that and I'm starting with this tough example because I want to show you want to make sure you understand that there's real limitations here with the portal office 365 is nowadays doing a good job with generating the audit trail but getting the data out even for casual searching is not that easy now I am going to know that Patrick was looking at my mailbox and that's ok he's allowed to so let's search on him we'll print some events in and then now I start to get some folder bodied events but the problem is what if we're just trying to find out has anybody looked at Randy's mailbox there's no good way to do that here's what the event looks like by the way so this is telling us that Patrick performed a folder bind operation it was done without look folder bind means viewed a folder in exchange in an exchange mailbox the folder was it it was the permanent subfolder under inbox called permanent and whose mailbox was it Randy's ok so that's what the events telling you but what if we didn't know it was Patrick that had done that and that's Bruce that's the whole point of this right is who was looking at a CEOs mailbox and leaked that information to the press you know what I'm saying exactly and it's kind of surprising that you can't search that from the portal now I think I'll show later on within logarithm how you can do that is that possible to do from the search Commandment it's a little bit easier there because you've got some other filtering capability I'll come in and talk about a search command ilat here in a second but here's other stuff that we can it's pretty easy you know we could say I want to see who was it that accessed such-and-such file and we'll take Patrick off of there and we could put the name of the file in and then search that way but I don't even remember any files that we've been auditing here here we go so yeah accessed file and then we get the name of the file these are all reports from my rosetta library and there you go we were trying to look for a particular file who's been looking at that data we could put that in so any file name with permission in it and there we go so it's pretty cool as you can see the data is really there it's being audited it's just a matter of getting it out so I think everybody in fact people are already asking me I don't want to you know here in the webinar I don't want to go to the portal I need this information in by 7:00 and you're short-circuiting me that's what this is all about I totally agree so how do you get that data out to the center well first of all my point is that using the portal by the way you can there's an export capability here on the portal 2 so you can export what I call bits and pieces of the autoblog but not the whole thing that's unfortunate part from from the portal so then that takes us to PowerShell because the problem with the portal is number one leaving the audit data inside of office 365 breaks the first law of audit log management right Bruce you cannot leave audit logs on the system where they're generated because that means they're vulnerable to tampering either by the privileged users and the only control we have over privileged users is the audit log that is the only good if a log is your only deterrent slash detective control over privileged users so we have to get the audit log off the system where it's generated but also of course bad guys one of the first things they do is they're going to erase the audit log to cover up their tracks and complicate your forensics capability so we've got to get that those audit logs out plus it's not archived long enough on I think the limit is 90 days I think ad stuff may be kept around a little bit longer but in general audit data is only kept in Ex office 365 for 90 days and Microsoft themself says you know you need to get this audit data out so can we use this PowerShell command search - unified audit block well let's take a look at it so if I run the command without you know any criteria at all you can see the kind of data we get you're going to have a record type that tells you where the audit event came from did it come from Azure ad did it come from exchange that come from SharePoint or so on of course you get the event and you get the user ID that is associated with performing that event ok then you get the operation and that's that's like where you would find a word folder bond and so here we're looking at a mailbox login and then all the other information which that see this this can vary a lot of the fields can vary depending upon the type of event because remember what we're getting here is every action whether it's creating a user account nazzer ad looking at a file in onedrive or doing something in Yammer or creating a dashboard in power bi so all of that is in here in JSON format and the audit data field so like what's something interesting here we would have mailbox logins or anything interesting on that particular oh yeah thank you actions were many plans did go ahead Bruce yeah it's going to say that's just a sign-in event so probably not super interesting from first look but if you're going back and auditing if somebody was signed in from where they're not supposed to be signed in then it gets really interesting yeah yeah so they're you know we get the IP address that would be interesting in that event here's an azure active directory account login so this is the initial authentication to Azure Active Directory you're not going to see this event if you remain logged in in the same browser and access other stuff and that's why you see these two different login events logging on to exchange your exchange mailbox is one thing you may do that many times a day but probably you're only going to have to log into Azure ad maybe once a day depending upon how your Federation and policies are set up okay but here's the thing I'm limited to and this is unclear I mean just check out the documentation on this page it drives me absolutely nuts I'm definitely going to have to do for the office 365 out log what I did for the Windows security log and that is get in there really test all of us out but the bottom line is that I'm waiting up to is you're limited to how many records this PowerShell command will return now there is the way to specify a session ID and run multiple instances of this PowerShell command to basically get like pages of data but even that has an upward limit and it's unclear what that limit is just on this one page there's three different numbers bandied about says results are limited to 10,000 records not 50,000 then let's try Q is kind of weird that I mean we can also put not 51,000 then and not 60 2,383 either first I mean there's a lot of other numbers besides 10,000 there's definitely some inconsistencies across their documentation that we've run across throughout this whole process of getting o365 pulled into logarithm um but then down here when it talks about results size it says the maximum is 5,000 so I don't know if it's 10 some 5,000 but I know it's not 50,000 because they say it's not um the bottom line is though in an organization of any size Bruce you're going to have a lot more events than 10,000 especially given that this is all or nothing you turn them on and everything gets audited you have every access to every file in everyone's onedrive for instance or nothing at all so we're talking about lots of events and there's just no way with either the portal or the this command here the search mailbox on a sorry search unified out log to get all of the data out to just get your entire audit log and it makes sense though because it would maybe it be unfair to expect that because this command is called search it's not called export entire mailbox on it log so definitely that's going to be an issue and what it leaves then is so I'm going to X this out because of Max results limit that leaves the office 365 management activity API so to do that you've got to program against the so-called restful api to get that data and that is what you folks have done bruce you've actually gone down to that level and done the programming and that is the only way you can get your entire audit log out of office 365 and that's what we're going to show you here in a second and then i'm going to come back and answer some of these questions that you folks have been posing about different different areas of this audit capability the bottom line though is officer 65 does a great job on this producing the audit trail so you definitely can tell what's happening in the cloud and that's great that is the most important thing but it's up to you to fulfill the remaining compliance regulations and security requirements such as number one securing it from privileged users and intruders getting long-term archival actually monitoring it and alerting when there's stuff that's important to follow up on and then finally correlation with the rest of your organization's activity otherwise what you have is called a silo so break down the silo for us bruce show us what you can do and then we'll come back and answer these questions and any folks have for you going to make you the presenter all right sounds great select randy said we get on board to the office 365 management activity API and we can pull all those logs into logarithm I'm not going to lie this is one of the more did called AP is to set up microsoft has a couple limitations there you have to create an app and as your ad you have to have a certificate you know maybe self sign but then you have to upload that sir - azure ad it's really at some time waiting in bulk at around put up a start interrupt sometimes you have to open up a support case to get them to turn it on even though you've turned on the management API it's not there's some little switch they have to go slip in a closet before you access you're getting the events yeah and we're trying to get that better documented on our side of what you would see if that's the case because there are a lot of these edge cases that we're running into I will say once you start pulling that data in it's great to have in the o365 environment and that's what I'm going to demo today so I just want to make sure that Randy you can see my dashboard coming through clearly if you look good wonderful so before I start the demo I'll do a quick intro on myself I'm a technical product manager from logarithm which means I'm embedded in the development team but I work closely with customers and partners to answer our question you know what are we going to ship next my areas are course M which is basically log processing and then threat intelligence and cloud log collection which is why I'm here today in a former life I did a stint at Microsoft working on an O 365 product Skype for business so this is a cool opportunities kind of come full circle with all the areas I've gotten to work in over the past couple years now talk a little bit about sims in general so if you're here for webinar on 365 you've probably moved to the cloud or you're considering moving to the cloud and what we found is often that the IT department and C cell have different priorities in an organization you know it's the cloud it's cheaper it's easier so IT said ok you know let's move all of our o 365 services to the cloud but from a security perspective you've lost control over a lot of that data and furthermore you may have services like SharePoint that were locked down on your internal network now that you're up in the cloud they're open to the Internet if somebody gets the user mitchell's so clearly you need access to those logs and there are a lot of them like Randy said it's very difficult to get all of them and those limits were very unclear but to give you a concrete example and logarithms environment we're a small/medium company five to six hundred people and we generate over ten thousand office 365 events in one day so that's a volume can't be managed without the right tools you're not going to be able to manage that through the 365 portal or even through the Commandant unless you build a lot of custom scripting behind that and furthermore you know Randy talks about the long term access to your logs those disappear within I think 90 days in the portal and they're only available for seven days within REST API so that's why we're calling the REST API every 10 seconds saying hey what do you got for me what do you got for me and consistently pulling that data in for even longer storage of your audit logs so you know especially if you're not running a sim how long is it going to take you to discover if there was an incident and then how are you going to discover where that initial compromise was if you don't have them centralized in somewhere that you know even if you don't have those in live active storage you can still have a passive storage like archiving that you're writing everything to so you can go pull that back in if it was say a year ago that you think there may have been some sort of compromise all right you're sold now sims are great but let's do an actual demo to see the o365 data in action I've had this dashboard up for a couple minutes rather than looking at it and logarithm sends in general allow you to aggregate all these log sources together to see holistic patterns throughout your environment data in this case I'm just focusing on one source within the dashboard I'm going to expand that out later but I can audit my most critical capabilities in one place and then pivot if I need to so for example I have the office 365 classifications chart highlighted here but these classifications aren't unique 365 take something like authentication failure that you can see is our fifth most popular that's something that would show up in a Windows Event log we'll actually see that later on in the demo or if you're on board it to something like Salesforce and somebody has a couple Salesforce login failures they come up as the same classification so that you can write a rule or we actually have AB with the Box rules that say if you see a number of authentication failures no matter what source they're from then tell me about it fire an alarm all right so when the logs come in from o365 logarithm processes them and pulls data out into metadata fields that Randy talked about earlier you know we have the classification common event but we also have a couple dozen other metadata fields from users to host IP addresses senders if you're auditing things like send on behalf of and these are rules that our logarithm Labs department has written for probably close to a thousand sources now so in the cases go 365 you do a little bit of configuration to pull these logs in and everything else is taken care of for you what that means for you is consistency and correlation across all your log data now you see the dashboard on one of the cool features is is I can go into the grid and see the logs that are generating the correlations that you see above so the first thing that I'm going to do is the delegated events that Randy talked about earlier so right here I got the command pinned and I'm going to look at the folder bind because that's easy to search for and I can go to my identity tab let's say and we have the user Steve doing something to set account there's metadata here like user agent that's populated out to a field that could be aggregated on and I go into the log message itself I can see that this was a delegated so I can even search on delegated actions and I would get other commands like send on behalf of and then I can go in and see the exact folder so Steve was looking at Seth's folder conversation history now this may not be Steve actually looking at that but it says that he at least had access to it and leaving the audit trail in place furthermore you get the IP address so maybe this isn't actually Steve with somebody that has these credentials hey so Ruth yeah so you know I think right here we see the benefit of getting this data into a sin because you could if you you normally could be starting with Seth who's been looking at Seth's mailbox and we could do that because you're surfacing all of that here and the search exactly so it you know it doesn't matter of who you're looking at you know I can click on this Steve field and I can get everything where Steve was the origin or impacted user which means Steve was doing something or someone was doing something to Steve's account and you can pivot off of that so a lot of what we do in here is starting with something like a log and then expanding outward and that's not something you can do easily from the o365 portal and even if you could it would limit you to that siloed data that kind of jive with what you've seen so far yeah absolutely I mean I I just can't even do that basic search in office 365 portal even though the data is there another interesting events something like account created you know sharing invitation created will show you that this person is sharing with somebody outside of their organization and we actually have it set up so that if it's an external account against access it has kind of a higher risk based priority in the system so you can kind of prioritize events that okay you know maybe I shared with the user inside of my organization that's not too suspicious but sharing outside of my organization I may want to take a look at that so that covers our dashboarding capability Randy I'm not sure if there's anything else you'd want to point out in here otherwise I'm going to jump into the alarms no go for it and then we got plenty of questions coming in so this is fun great all right so you know I'm a stock analyst I come in in the morning I say is there anything interesting for me to look at and I have an alarm so this alarm comes from our advanced intelligence engine that's a set of rules that are labs Department security experts have said these are potentially malicious or suspicious things they have a saying that like they say this is suspicious let's find out if it's malicious and you can do that with a simple drill down you know you can see that the title here is brute force offs and what that actually is is a number of failed authentications and I think I said earlier it doesn't have to be just in oh three sixty five now in this example it is but it could be somebody trying to compromise credentials between office 365 Salesforce anywhere that they may have a single authentication now this is kind of interesting we have a 13 user failures in a pretty short time period and I'm curious now what this user was doing so I can hit it pretty quickly on the user I can see you know let's go 24 hours in the past and 6 hours in the future from when all these logs were generated that'll queue up a new search and now all results are in and this use been busy there's a lot of activity going on here you know you can see over a thousand events generated in just over a day I'm going to add a widget live will see if I can get this to go in a live demo and it's going to be a trend widget so I'm going to look at what this users been doing from a classification perspective over the past 36 hours and you can see right away that there's a lot of activity right before those auth failures I can mouse over and see the spike in all failures right around 9 p.m. but I also see a big spike in access successes which is kind of curious so I have a user that was potentially compromised with a group force attack I have access successes and authentication successes around the same time so what I can do now is go into my logs and look at classification has access successes and I'll go in and look at anything that has the file name populated and right away I can see a bunch of file downloads from this user and I get the file name which is great from an auditing perspective because if I'm looking to respond to this I need to know exactly what files were touched by this potentially malicious activity and I can go into the identity and see some you know additional data I can look at the IP address these files were downloaded to so a lot of great information coming from office 365 so let's go back real quick and look at the authentication failures because I want to figure out what's going on with this user so I'm going to do a filter to all failures and I can see a lot of log sources from the office 3 6 five API but I also see on one of my hosts from an MS event log that there's a bunch of failures around this time as well now at this point it could just be a user that forgot its password or you know change their password recently but there are some suspicious things around this host so let's pivot on that for a second so we'll just look at after that failed login we'll go one hour in the future that'll show me everything going on in that host and if you're running Network Monitor which is one of the things we just launched a freemium on so you can download and try that out for free and send events to your SEM you can see on this host you had 130 megabytes of Dropbox traffic within an hour after those failed logins so immediately this just in my brain went from suspicious to malicious one of the common ex filtrations that we see is Dropbox network monitor will classify that for us so now we've gone from an alarm we've identified a user that was potentially compromised we've identified the files that were accessed and we've identified that there's potential data exfiltration so this really speaks to the core value of a sum which is mean time to detect and mean time to respond we detected it quickly through the alarms through pulling the API in and you got to ask the question if you weren't bringing these logs into the sem would you have discovered this incident at all maybe you would have but how long would it have taken and now that you've discovered it how long is it going to take you to respond to lock down this event we already know what files were accessed and what could have been compromised but there's also capabilities within the alarms to have something called a smart response so these are packages where you can set up things like automatically disable account or add that user to a watchlist if you're just so Bish's of that user you know maybe elevate their potential threat so give you the ability to detect and respond super quickly and I'll summarize that by saying you know our logs are really valuable to get from o365 but they're unmanageable without some sort of log management tool they're just one piece to the puzzle in your environment and you can only get that holistic picture when you put the puzzle together with a Sam like logarithm so Randy I'll hand it back over to you if you have anything to add about the investigation the demo or if you want to just jump into questions okay that sounds great um first of all let's see Michael asks where can I get a copy Michael were you talking about logarithm I mean can they arrange a a demo or an eval or what they could you know you can reach out to me and Randy if you can share out my LinkedIn profile or my email with this in case anybody has questions on the logarithm side after the fact that would be grid they can reach out to me directly and whether it's questions on just getting 365 into their environment if they're an existing logarithm customer or if they aren't running its own completely and want a demo I can get them connected for the right people okay great let's see here Donal says does the office 365 shred activity map come from Microsoft or logarithm so let me go back to that threat activity map so the dashboard in general 403 65 was something that I created for the demo it's pretty easy to just drag and drop widgets and these things are filtered to where the log source is go 365 management activity and I added the threat map which is a widget on its own and I filtered that down to the office 365 man activity I also put in the message tracking that's one of the things that microsoft offers through api as well that we're working on official product support but it's pretty easy to do in PowerShell as well so you can see where he emails are coming in from where they're going to and you get IP address information like that so this is something to answer the question directly it comes from logarithm that you can build out a dashboard but I can also share out this dashboard file it's pretty easy to export and then if you wanted to add the dashboard all you would have to do is import the dashboard file and would come complete with the threat map which would be populated with your data now here's an awesome question that demonstrates you know the gets to the power of assume Steve says what about tying public IPS to internal private RFC 1918 IPS to figure out the internal machine or user that did the officer 65 activity says customers used NAT so all office 365 activity from the internal network will look like it came from the same you know public or you know internet NAT IP address right the IP address of your navigate way is there some way to map this or connect them so I mean that's the kind of thing that a logarithm is awesome at is being able to link disparate events to each other yeah there's a couple ways that you can do that there's entity creation where you can you know map a user box to an internal IP address there's also identity inference which might be the more appropriate one for this scenario so you may know the host that it's coming from but you don't know who's logged in at the time and one like dent inference does is it says I know this user signed on to the machine at 10 a.m. and signed off at 10:30 they were the only one signed in time and the activities at 10:15 so I can say with reasonable confidence that it was Bruce doing this activity that's right there would be a few different ways and you could definitely do it with yeah good timing is going to be part of it Adam says for those of us using logarithm already we please show us the config of your office 365 widget for the web console and so a couple other people maybe we can kind of knock these out at the same time Carlton says I didn't really catch how the data gets from office 365 to logarithm is this the API configuration process and so if British that's what's happening I don't know where it runs inside a logarithm I know you've got a program that is you've already gotten the appropriate what is it it's open ID keys what it's not no no you're using a certificate but I can't remember the standard nowadays for the kind of authentication normally used with rest but that's you've got that token and that's what you're using to connect office 365 office 365 knows that ok you are such and such application that's been pre authorized for this tenet and then you're saying is there any new audit data is there anything lot of data and then if there is it gives you back an ID for that chunk of data this is how the API works then you turn around and then ask for that chunk of data right and then parse it right it's a pretty involved process to set up I would say it's probably one of our most involved API is because of what you have to do on the Microsoft side to get it going and I don't have a demo readily available I don't want to be showing my private keys and everything on the demo but I am working on a blog post for this right now to talk about some best practices when you're adding it so walk through verbally on the steps that need to be done you start with in Azure ad and you add an application and this is the application that your agent is going to authenticate to you'll add permissions to the application within oh three sixty five to say that yes you have access to my oh three sixty five management activity data so once you've done that you'll go into the logarithm environment and you can either generate a self signed certificate or use one you have already and you'll export a couple values from that certificate you have to go upload those values back into the you 365 portal which tells Microsoft that you know these are my cert values you can use those to decode the token that I authenticate with from a logarithm perspective you'll put a couple values like your application ID the path to your certificate and your certificate password into a configuration file and then all you have to do is add the log source from the logarithm console how about it's a node 365 log source and pointed to that configuration file from there the logarithm agent is going to contact 365 it's going to create a JWT token and it's going to sign that token with your certificate and since 365 has private carriers typically it can decode that and say ok you are who you say you are now I can start giving you back log data so our agent will start calling the REST API and say I want to subscribe to events from Azure ad and exchange and SharePoint and give me everything you have as it comes down hey I took over for a second here just presenting to kind of give folks a feel of the office 365 part you have to come into actually azure ad into your directory in my case it's Monterey Tech recom and then you go to enterprise applications and this is where I set up my Unified autolog research basically you know it would be telling them I've got logarithm and there's other information that you have to provide including those keys that you were talking about but that's that's what you're talking about an office 365 part right yeah exactly creating that application and making sure the right permissions are on it yeah it needs an application ID and there's some where where you actually give it the permission until it what it can do but it changes every week so it looks different than the last time I Cana you may need so if you scroll back over to the right there's a couple places you can do this from you have to go to app registrations so the and then you go to your application so the enterprise application will give you kind of I want to call it read-only access or you can search audit logs from the application but when you go into the registered applications that's where you can do your set up on required permissions and yeah like you said it just changed every week they are in the process of switching over their portal from the old version to the new version and I'm still working on figuring out exactly what moved where but I'll have plenty of screenshots included in the blog post I will say all of Microsoft's documentation still have screenshots from the old portal so that's where it can get a little confusing and but there it is there's the office 365 management API permission where you give your application access to that yeah and I think there's only four of them there are seven permissions total in there and they've deprecated three of them okay and they do go into detail in their documentation on the exact permissions you need okay but what that is all about is simply authorizing an application to connect whether it's in the cloud or on Prem to connect via the rest standards not really protocol to office 365 and get this data out now you still can do all the programming against that REST API and that rust API document in MSDN okay it smoke wait yeah well I think doctor doing it well Marinette let's see here Tommy asks one of those unanswerable questions how much storage space will I need to store these audit logs but of course that all depends upon how many users you have what their activity is and then what's in do you have because every sim stores the stuff differently but this is very highly textual information with a lot of redundant bytes in it so I imagine you guys do some kind of compression we do and there's actually a couple places that you can send in store so when our labs departments are putting together the rules for what to parse out they decide hey this is probably an important event you know let's say a file share maybe with an external user something where you're doing delegated access whereas a sign-in probably wouldn't be as critical so we take the critical ones and we send them to our events database and our events live longer than the rest of the storage and you can do a little bit more with them everything goes into archives so that's you know maybe you put it on tape or long-term storage so that if you do go past your time to live then you'll pull those ins or something we call second look and you can say all right give me everything from o 365 from the past year and it'll put that into a special repository if you can then use to search against so to summarize that we have critical events with a longer TTL and some less critical events with a shorter TTL that go into our elastic search so Enzo wants to know is there a way to audit when people are forwarding email and I'm not aware of any event that actually catches that you forward any No our events for catching when you create an email if you've turned on Auditing of owner access to their mailbox and of course that's going to generate a lot of activity because it's going to track everything the user does inside their own mailbox but there's a mystery on that it took probably a hacker you're talking about an attacker forwarding email oh I see Enzo is saying what if we set up a forwarding rule like with PowerShell yes that is absolutely audible with and it's not the mailbox I log it's the admin audit log inside of exchange so yeah that little command that I gave you earlier would turn on auditing of that you just need to figure out what PowerShell command you're talking about um Dale asks are there default signatures to identify something like password spraying as well as brute force against one camp and yeah you guys are all over that with your knowledge engineering team right we are and that's one of the goals advanced intelligence engine to pull together a lot of these potentially suspicious events so it will look for things like you know authentication failures across a variety of log sources one of the interesting rules that I should have enabled for this because that would have made a better alarm than the one I had Mabel it was where you have a number of authentication failures followed by an authentication success and it's such a simple scenario but it also gives you more context when you're looking at it to say oh somebody might have been brute-forcing and it looks like they got in but that's grouped by a source IP so that would be one person coming in we also have another rule that's looking for distributed route force so you know potentially somebody hitting it from a number of different endpoints that we can detect and fire with a different rule that gives you even more context of what might be going on that's cool let's see here good to hear from science of what sort of bandwidth is the system monitor system monitor agent using when pulling down office365 data have you looked it I mean the biggest thing to consider is the polling and that's really just like every 10 seconds asking for a small webpage really it's all you're doing right so it's not really a significant amount of bandwidth I think it would be mostly from the audit logs and if you were concerned about a single agent doing a lot of bandwidth you can actually split that up into one of the agents idling as your active directory one of the monitoring exchange one of them on a SharePoint especially for a larger organization that may have you know 10 20 events every couple seconds or more than 100 events especially if you have a ten thousand person organization we're focus I know yet yeah and also many part numbers for you because it does depend on the size of the organization oh yeah totally I don't as is there any way like with IP address ranges or domains to determine a line of business or departments in your search I do connect up to Active Directory right you do import information from ad this isn't necessarily I don't think this question really escalates to do with office 365 it's just when you're looking at events can we can we start adding filters rules say well if it's the sales department this but if it's engineering that you know you can you can do the Active Directory sync that'll pull people down you know based off of security group and then you can search against security group you know you could pull certain users into a logarithm list and if you wanted to I think I mentioned earlier with highly suspicious users you could have one alarm that fires and says this is something that's suspicious when they automatically add that user to a list and then have another it says if you see activity and the user is on a suspicious list then elevate the priority of that alarm so there's a couple different scenarios there you know whether you're searching by department or trying to do something more dynamically it's all possible James asks can logarithm play with other cloud providers seems like we would need visibility into any cloud provide ad or I you know I would say any cloud provider that has important information you need to audit so I imagine that something you guys are working on like for instance Dropbox and box they both have audit logs as well right we do have box audit logs we're working on Dropbox AWS is another common one where folks will have the role infrastructure hosted in AWS and we can get all those logs back down you know they have a number of services like cloud trail cloud watch s3 so all those logs are important to your environment and we can pull them in and as long as your users are named appropriately you know most with most of the time you're probably thinking Active Directory anyway so then you get everybody in the same place so you can see Bruce is doing activity in exchange online and over an AWS and over in box Adrienne a while back asked me did the audit logs include exchange online DLP data lots of prevention and no I don't think they do Adrienne I'm not very familiar with the DLP stuff and exchange online but I can't tell you that I've never seen any events pertaining to it in a unified audit log Matt does is that which agrees then for a copy mailbox command to execute seems like this would be a major red flag and so I mean that's an example of totally what you could do is export and copy mailbox I mean those are very specific PowerShell commands so I imagine that that gets already gets put in a kind of standard operation or action field in logarithm and so yeah it'd be simple to do that I'm going to jump back to the DLP real quick because on your screen you're showing permissions and one of them is read DLP policy events including detected sensitive sensitive data oh yeah however that's one of the ones that Microsoft said were deprecated so it sounds like they wanted to do that and then they decided they weren't going to do that through the management API so they removed it and the next time I talked to the Microsoft folks I will ask them about that so I'm kind of curious what their roadmap is around DLP and exposing that through API cool first let me make you a presenter again in case anything comes up that we want to show and let's see here my next question Donald says compared with Amazon AWS and their SNS simple notification service there definitely is a latency with Microsoft so yeah now but technically the without the management activity API does support the idea of webhook where they will reach out to you the arrests and tell you immediately when new audit is available but that doesn't change the fact that that audited is available on a delay of anywhere from you know a few minutes to much much much longer I think thing to try to get stuff down to two hours but there's still some stuff that's subject to much longer delays than two hours maybe even 24 hours you say I don't think latency should be excused with cloud I would agree with you I mean it's the fact that matter is there's a lot that you give up when you go to the public cloud but yeah at the same time there's a lot that you get and it it all depends on whether what you're giving up and what you're getting are which ones are the most important um but definitely in terms of secure be in control and just latency the cloud takes longer like I'm amazed Bruce I don't know about you about how long it just takes to just spin up a virtual machine I you know there's no way I could switch over to using the cloud for my test environment because when I get ready to do some testing I want the VM up in a minute not 15 minutes is that as your AWS that you're using well that's a sure is a what's AWS like I don't have much experience with either I know kind of do most messed up locally as well yeah let's see here Michelle are there additional costs to enabling auditing in office 365 for storage and what is the default retention and maximum amount of time of retention so Michelle I think I answered that about the retention it's 90 days there's no charges for storing that automated inside of office 365 although I did point out that that you would fail any any well executed audit or compliance assessment if you're keeping your audit data on the same system where it's generated you've got to get that out because in any medium or larger size organization your security folks are going to own the sim and the storage that same uses as opposed to all the other operational admins that that own the systems that you're automating you know the separation of duties that I'm talking about there Bruce I do and you know in terms of that it is it does vary by organization as to who owns what but typically in an organization that's under compliance constraints they have certain things that need to be in place and you know logarithm is coming criteria certified and we do have a number of compliance reports out of the box so if you are pulling in for compliance reasons we can support that and all the reports that are there a let's see here Mickey s is there an option for offloading the same audit logging to ascend and so that is very dependent upon Semin you've seen today what logarithm has done what's D here Michelle as in auto log search and a group name and instead of users know it's not going to say did a member of this group I going to support being able to ask in the portal did a member of this group do this or do that Donald what permissions are logging on the audit log are available so what file permissions so done are you saying can I audit changes to file permissions or are you asking what permissions are needed to look at the audit log Dale asks how's the export of audit logs affected by the delay in the log data being present in a sure that Bruce mentioned so it's just a matter of asking for the lot of data and as soon as you see it you get it right Bruce right and we'll pull out the timestamp from the log itself so you know we'll be asking for the latest logs Microsoft they'll give us logs they may be four hours old at that point but it will include a timestamp in there that says this log is from 6:00 in the morning even though we're getting it at 11:00 in the afternoon and we'll use that as our timestamp in the system so you can still correlate that with other events that's happened around the same time you may just not get it when you want it yeah or as soon as you want to let's see here my s can filters be applied officers to 5:00 activity no it's it's an all or nothing officer subscribe is going to generate all these events and like many log sources it really becomes a function a matter of your sin filtering out the noise let's see here Oh saina s is the what console you're showing that dashboard is that available from logarithm support or it's something will be available soon I guess so the web console in general is available for logarithm I am on the 72 version which will be releasing within the next couple of weeks and that gives me the ability to do a couple of these new widgets I think that the trend but yes in general with any six three seven one or 72 version of logarithm you have the web console Sammy what are the possibilities of storing the logs log files besides as or so they're not necessarily stored in Azure they're just in office 365 and that's it unless you have something like logarithm going to use the management activity API to get the data out that's all you got James says so does this latency mess with the aie rules that say event B happens within one hour of event a and that's a great question right it is and it shouldn't because we're pulling out the timestamp and that's the timestamp that aie will use to identify when these things happened so Simon's really interested in getting your dashboard your office t5 dashboard and so he'll be the follow-up you guys can follow up later on we've got a transcript of the webinar and his email address and so on yeah I'm - he wants that dashboard shared so all right okay the import/export so I'll be happy to do it I'll be watching the community logarithm community or the support portal and a bunch of people are saying they want to see it so you're popular today Bruce oh good oh yeah the community is another good place that I can post this if lovers and customers aren't part of that we did launch a new community a couple my to go on a platform called lithium so get on to that and we're good about getting questions answered there so if you do have questions about office 365 or anything else logarithm it's a great resource if you don't really want to open a support ticket you set a quick question technical product managers are on there some of the developers and then a lot of our sales and professional services folks that are really knowledgeable about the entire product as well awesome Andreea asked his office 365 completely separate from reporting so she's asking about office 365 reports about which documents are most popular and being downloaded so yeah that would be coming that's usage statistics I think Andre not portable auditing Sam you'd like to know what is the licensing for logarithm is it per seat of concurrent users is it events per second or what there's an event per second and if you do have endpoint monitoring there's some additional licensing there like if you want real-time sim for PCI compliance or something but primarily it's a logs per second but that's averaged across a day and we don't cut you off so you know the worst thing that you could do is be under attack and you know let's say your license for 10,000 logs per second and all of a sudden you spiked up to 50 we're not going to cut you off we'll just average that out so if you do have spikes during the day and it comes down at night or comes down after the attack I you won't have to worry about losing data nice well I think that takes us to the end of our questions and folks we hope this was valuable to you we'll be sending out a copy of the webinar and the slides and other other follow-up information what should people do if they'd like to learn more about logarithm or it mo or whatever like that you can reach out to me directly or you can just go to logarithm comm and we have section on there I think it's called request a demo or contact us and we'll have somebody in touch right away to figure out what your organization needs are and how we get a demo set up for your environment all right terrific have a great day everyone and this won't be the last time we talk about office 365 and Azure thanks for joining by everybody

Channel: LogRhythm

Views: 11,572

Keywords: Microsoft, Microsoft windows, Microsoft Office 365, Microsoft Office 365 Logging, audit logging, logging, uws, ultimate windows security, logrhythm

Id: KUyE59E3EFY

Length: 88min 58sec (5338 seconds)

Published: Tue Oct 25 2016