Distributed Evidence Collection and Analysis with Velociraptor - SANS DFIR Summit 2019

[Music] we are really excited to be here today and we're gonna share with you something that has been in development for about the last 18 months or so we've kept it a little bit quiet until now we've been working on it actually using it in our team for the last year or so on a lot of real-life cases it's a forensic tool that we've built and we're making called Velociraptor and we are very keen to share it with you today now I'm gonna I'm gonna embarrass Mike for a minute because he's too humble for anyone who doesn't know Mike Cowan he's you know we talk about building a community and sharing knowledge and creating open-source tools Mike's pretty much spent his career doing that for us he's led a lot of groundbreaking research particularly in areas like memory forensics if you've ever used tools like volatility like recall like go rapid response then you've benefitted as we all have from work that Mike's done so and it's in that same so you can all buy him a drink later it's in that same spirit that we're kind of bringing Velociraptor out to everyone this is something that we're developing but we're releasing it we have released it as free and open source software so all the stuff that we're showing you today it has been used in real life on real life cases and it's something that we are contributing back to the community and we want everyone to get involved in and and benefit from it so what we're gonna be talking about today is what is it how does it work and most importantly how can you and all of us benefit from from what we're developing here all righty so what's the need here what's the point when you do digital forensics work you'll be familiar with you know we need to get visibility we need to see what's happening on the network if you're doing a network investigation you need to be able to move around the network to very surgically collect information for your investigation and not just monitor stuff that's happening but also be able to go back in the past we have some really good tools these days you know especially in the ADR space where we can see what's going on we can sort of build things to look at what's happening but you know you might be able to use our tool to see if a piece of malware is executing but what if you want to go back in time what if you want to say well I know about all these areas where I look for evidence of execution I want to go back and I want to look at those and I want to do it in a surgical way and I want to do it very quickly and I want to do it at scale that's kind of one of the gaps that we have with our tool set and that's what we're building velociraptor to to try to do so what we're gonna do we've got a lot of demonstration a lot of case studies that we're gonna be going through but before we go through them we're gonna give you a quick overview of the the architecture the design and some of the key points of this system so here's our very simple network over on the Left we have our no sorry I'm in Australia it's the wrong way around on the right we have our digital forensic ninjas who are needing to do their investigation on the Left we have our systems that we need to investigate what we're going to be doing is standing up a server a Velociraptor server and this is going to have a bit of data and some file store and then we're going to be pushing out agents to our systems now this is written in go it's very cross-platform compatible so we don't have any compatibility issues across platforms we have clients already for Windows for Linux and for Mac we push these out and we fire them up they talk back to the server and at this point they're ready and they maintain a consistent connection with the server and they're waiting to accept commands and then our investigators over here are going to be connecting to that server through the GUI just a web-based GUI and they're going to be doing whatever analysis they want to do and we're going to get into the details of this as we go now a few key design things to know about velociraptor is a single executable that's it there's no dll's there's no external dependencies there's no OS requirements it's one executable put for that particular operating system that same executable can be a server or a client so when you fire it up you you will tell it what you want it to be the can generation of them is controlled by a couple of text files that have the config and they have some simple information like the location of the server some encryption keys for secure comms and the directories where the data gets stored but that's it it's a very very simple lightweight setup and as I said very cross compatible at the moment there's no data base we're literally just running flat files in the backend which actually makes it very very quick very easy to manage very easy to maintain it also keeps the resource usage quite low the processing that we're going to do is going to be both on the clients and on the servers depending on the task that we're doing and you'll see how that works as we go through the scenarios and it's also very highly customizable you can change the service names the descriptions the executable names the server location obviously the port's all of that can be changed when we deploy this we typically put it in the cloud we get a DNS name that's very hard to see it just blends in with all the other traffic and we can be very very sneaky about how we put this out there and we'll explain why that kind of cloud deployment is good for other reasons as well so that's a high level architecture now key design principles again we're gonna get a little bit technical now because we want to focus right upfront on what makes this so good what why do we think this is so valuable as we've used it on jobs then we're going to go through a lot of case studies so the most important thing is and I'm going to hand over to Mike to give you the details of this because this is really the sort of the genius that he's created behind this what Velociraptor is is a query language it's not a tool it's not just a piece of software it's actually a very very specially crafted language for doing analysis of endpoints for forensic artifacts that's focused on dfi our cases and it's the vql that velociraptor query language that powers everything that we do so mike did you wanna describe it a bit more sure so so the whole point of of Velociraptor is really like Nick was saying is a query engine so all it does is it actually evaluates that query you can write that query you can change it you can write whatever you want and and it basically just very flexibly run runs out that allows people to adapt really quickly you don't have to recompile you don't have to write code you can just change the query and you can do something completely novel that you know wasn't even built-in but you might have to come up to it in the course of an investigation but at the same time we didn't really want to make make it so that everybody has to be a vql expert so that they have to you know just be able to write a query to use the tool it's kind of cumbersome and it's kind of hard right so what we actually have is this idea of velociraptor artifacts and an artifact is just a text yellow file that contains the queries and how a bunch of metadata will go into that in details and that kind of wraps that knowledge that specific mechanical knowledge of how to actually do this which is collect this particular forensic artifact in in sort of a human consumable sort of yeah no file right so then so that you can then basically have two classes of users like one that is you know highly technical they know the vql I can basically go ahead and and write and customize and you know everybody can get up to that that level but then the users can then simply use that work and and just pick the the artifact to collect the specific things that they that they wanted to either collect or analyze wanted to respond all these sort of things so you don't have to know vql to use the tool you can simply just select what you want and it goes ahead and does it yeah but if you want to customize it as you'll see it's very easy to do so yeah so again the concept behind this is we we have questions to answer we're doing an investigation we want to know what programs have executed across a computer or across our whole fleet we know as digital forensics people what to look for you know when we when we study forensics we know that there are six or ten or more places that we can look for evidence of execution so what we're doing is we're getting that knowledge and we're encapsulating that into a velociraptor artifact so now you don't have to think here are the ten places I need to go out and separately look you can just use one artifact and it'll do it for you and then if you want to customize that artifact maybe to focus on a particular some malware or a particular location or a user account or anything else the query language and the functions and the parameters and the other features allow you to customize it very easy very very easily and then once you've got that artifacts a evidence of execution you can reuse that in different places you can use it to look at one computer you can do it to hunt across multiple computers as you'll see you can even set up monitoring to do continual monitoring and you can even do response actions all based on that one sort of body of knowledge that forensic artifact so again I'll hand over to Mike and you can talk through some of the details of this and use the magic button okay so thanks so this is an overview of what an artifact looks like as I said before it's just a yamo file and it's it's supposed to be human readable human consumable and so on I've got a little bit of a zoom up so at the very top you know we just have some metadata we have some name in the description and that helps us search for it so you know so if you write your own artifact you can put some keywords in there and then the user can simply search for it evidence of execution lateral movement all the usual things and then we can basically find the right artifacts to look for what we need then we have this parameter section so you can customize the artifacts you can provide parameters with default values and that allows the user in the UI to to customize it to some extent and to you know change those parameters but but they will be pre-populated in this particular case the example that we're showing here it's just simply checks for for their homework parent-child process kind of thing right we're going to work with this example but you can see that you know essentially the parameter is a list of process name the expected parent name that you know it should have right so it's just a CSV file with these kind of information and then we get to the meat of the artifact which is the vql queries and I'm not going to go too much into detail here other than to say you know there's there's a bunch of queries this one's a little bit more complicated but you know once that query is is read then you basically can change that adding more process name relationship into the CSV file but you can see that it essentially pauses that CSV file that is provided in the parameter section and and and then just uses that yeah so one of the questions we sometimes get is hang on we've got ways of querying endpoints already we've got things like OS query and other tools why do we need another one so we thought it might be useful to just sort of explain some of the differences so our square is a really great tool like that would be the first ones to actually have this idea of like having a query engine on the endpoint that's really cool that was a breakthrough but the problem is that they use plain SQL which is very limited and if you can see there is a really good project there where they try to map the attack framework to our square queries and and you can see on the left side they are doing these queries it's a little obviously it's hard to see but it's actually quite quite an involved very complicated query there's lots of joins and each of those queries each block of text on the left is essentially one line of that CSV file because there's no really way to kind of write it concisely so if you were to you know add new queries it's quite involved you know it's very very hard to to deal with that but on the on the right hand side we have the Velociraptor version of this which is basically table driven and the query just kind of like just does the same thing you know for each element of the query yeah so the important thing is this might all seem very complicated you might think hang on this is this is more than I need I just want easy click button it's actually not that hard and when we go through some of the scenarios we're going to show you how you don't even need to edit any of this stuff you can literally just point and click and it will do a lot of default things for you because we have already a very strong library of artifacts and queries built up but if you want to edit it and change them you can so here for example this parameter table as Mike said is just gerrant child parent process relationships if you want to customize it all you need to do is change the line of that without going into any of the vql and you've got a customized artifact for your investigation or for your particular environment this is an example of one of these here we've got a query that tree our jizz event logs from a window system and you'll see what we're showing here are the parameters and again you don't need to be a vql expert you can see what it does it goes into these locations and it grabs all the e VTX files and ev2 files that it can again you can customize this by just changing this parameter you don't need to go into the code but because of the way it's designed you have that complete flexibility and we're gonna build some scenarios as we go to demonstrate how this works now a quick set up where's it is Alyssa here somewhere I promised that I'd show you how to set this up in like two minutes so here we go this is the setup it's a single executable so firstly we're going to set up a server on the server we run the Velociraptor executable and we've got a configuration generator that we run an interactive one so you run that it asks you like six questions what's the address of the server what port do you want to listen on what directories do you want to put the data in and give me a couple of user names for the interface that's it you press the buttons and away you go it's also got you can do self-signed certificates which we do by default which are actually quite secure because it really locks down the client-server communication without using a CA it has built-in ability to just to pull a let's encrypt certificate if you want that and of course you can always just sign it yourself so that's the set up you can literally press those buttons in about 10 seconds and once that set up your configuration is done it will spit out two files one of them is the configuration file for the clients one for the server that's it now we want to fire up our server we run again the same executable the Velociraptor executable we pointed at the server config and the server fires up we log in with our username that we created and password and the server's up and running that's it we also do SSO with Google OAuth if you want to link that together which is kind of cool that's the server set up when we usually deployed we set up in a cloud it's really easy to spin up a server do this set up routine generate the encryption keys and that's it as long as you've got your DNS or external you well available one of the huge benefits of doing it in the cloud is you're not locked to your network so if you have a user who gets their laptop and they go home if they fire up their laptop at home they connect to their internet the client the velociraptor clients going to wake up it's going to connect to the server on the internet and you've still got visibility and access to that box even when they're outside your network perimeter which is really really we find it very powerful on a lot of cases so our service set up now we want to deploy our clients again it's the same executable there's different ways to push this out you can choose whatever deployment method you want we usually do either group policy or sort of an ad push or SCCM we recommend using an MSI because then you can package it all up and we also recommend signing it so you tend not to get problems with many antivirus or security systems that's it you push it out as soon as that client executes so on Windows it'll install as a service and again you can customize all the parameters it'll talk to the server and your networks up and running you've got a velociraptor deployment this is the dashboard on the right you can actually see the number of clients that are connected the reason it goes up and down that's at the end of the day when people turn their computers off you get sort of spikes on its peak peaks and troughs but that's it your network is ready when the clients are on they have a active connection to the server this is not a polling system they are constantly connected when you browse their directories when you send out artifacts when you do data collection it happens instantly all right and it happens simultaneously it's not scripted it pushes out to everything at the same time so that's the overview now we've got a series of scenarios that we're gonna run you through again these are real-life scenarios a lot of these come from actual cases that we've worked on where we've used Velociraptor for the last year or so let's start with something really simple data collection now some of these slides are a little hard to read so I apologize I'll try to describe what's on them we have an interactive Explorer it's sort of like a File Explorer you click on the on the host and you can start browsing it straight away you know what this is like when you're doing an investigation if you have something happen on end point you just want to jump on it you might get something interesting happening in a directory and you're thinking what else is in that directory what else is there I might just want to jump on and grab a few files so you can do that interactively you can connect to the host just through the filesystem there's a raw NTFS access as well which bypasses all the filesystem security so you can access locked files like your MF T's event logs registry hives all those good things you can also access the registry directly all right you browse through you click through it populates the directories as you go you select your file so in this case we've got an enter user hive what we found on disk we've dragged it down from the clients and we can download a copy of that to our computer and then we can get whatever registry tool of choice we have if we want to and start exploring that so really simple a few clicks and where you go no vql required all the magic happens at the backend notice to enter user as you'll know as a locked file when the user is logged in you can't get it but because we have raw NTFS access that's not an issue for us so now let's so what we just showed you is very simple it's just pointing and clicking let's go back to what we described earlier this vql and these artifacts and let's look at how these are going to help us so I'm gonna zoom in on this one and let Mike explain the magic that's going on here thanks Nick so I mean everybody can click on the interface and try and find your enter user house but then you have to know oh you know where are they going to be you know they're gonna be in see users username and so that's already you're already starting to use your deify our knowledge of expert knowledge and what we wanted to do is have a way of encoding that so that you know people don't have to think about it don't have to know and they don't have to click and spend time doing that and you can see in this example we've got an artifact or Windows registry and user upload which just grabs the end-user files and upload them to the server and you can see even in a very simple vql that that knowledge is encoded in that so you can see it's basically numerating all the users on the system and then it finds their home directory and it knows that NT user door that will be in their home directory and then you know the as we mentioned before we have to use the NTFS driver to access that raw interface file and so all of that information is already encoded in this in this very you know few lines of bql so now as a user or you don't even have to worry about it you're just gonna choose that artifact and just go yeah go get it for me and we're not only encapsulating that friends acknowledge but we're building the start of automation repeatable processes things that you can do easily if this machine has multiple users you don't want to click through the directories and grab them manually you can just run this artifact it'll do the hard work for you and it'll because that forensic knowledge is in the artifact it knows where to look you don't need to worry about that you can also customize this so for example you're doing an investigation you've identified on one particular host that you've got an attacker who seems to have compromised a service account maybe one of your backup accounts so you get that million-dollar question what did the user steal what did the attacker take all right and we all know as forensic people one of the key places we look for evidence of a person's activity is in their user hides because we get the recent Docs and the office recent docs we get word wheel queries we get you know all that great user information in those artifacts so we want to get that but we don't want to search for every user we don't want to pull back every user hive we know our compromised account we just want to focus on that one and this is where again the flexibility of vql comes into place so what we're doing here the screen you're seeing now is the screen where we can make a copy of an existing artifact and edit it and customize it to our needs it's a little hard to see but at the top of that artifact it it's got a little description that says custom artifact just looking at our compromised backup account and all we've done is we've added that little string to the existing artifact that says go and get the list of users but match the one where the account name is compromised backup account so we've taken that first automation we've added our thread Intel from our investigation and now we've got a very surgical artifact that just pulls out that account into user hive pretty straightforward right pretty simple so hopefully you're kind of thinking okay this is interesting where's it gonna go but wait there's more so you can you can do all sorts of things with it here's another artifact where we're collecting registry hives from the system now we point this out for a couple of reasons this is actually a great example of how we're sort of sharing information in the community when we built this one we actually hooked up with our friend Eric who's up there back there there you go he's already done some great work to actually enumerate some of these key registry locations so we said well you know that's that's public that's out there why don't we reuse that get that forensic that great forensic knowledge and the effort that Eric's put together build it into an artifact and now we can all benefit from it very easily so we took all those registry definitions that Eric wrote and he kindly let us use them and we put them into the tool and obviously giving him credit for that so a great example of how we're really trying to build something that people in the community can share their knowledge and information with okay that's the simple version right that's the simple collection all we've done so far is we've looked at into user hubs on a particular box one of the key things to level up now is anything that you can look for on a single box you can look for across your network using the same artifacts so now what we're going to be doing is taking that into user hive and we're going to extend it across the whole network and this is a concept that we call a hunt we're hunting across the network and you can actually build a hunt not just to do one thing but to do a range of things so this is where we're building a hunt and in this particular example you'll see up the top of the screen we've got our big list of artifacts that we can select from each of those we can either use as a default or we can customize them for our investigation and then down the bottom you see we've chosen to get enter user hives we've chosen evidence of execution and we'll explain what that is soon and we've chosen Internet Explorer evidence because that's our investigative need for this particular hunt we line them up we press a few buttons we choose the hunt group which can be a group of users maybe your servers your senior executives your confidential staff or it could just be the whole network you press a few buttons and the hunt deploys and now that same collection that we did on an individual box we've done across the whole network it's that simple so let's talk about some other scenarios finding files one of the key things we do in investigations is we want to look for files we want to see files and we've done this in so many different cases so you know one scenario might be an organization who they've realized that this information that's been disclosed about them there's some files that have been disclosed and their thought is well who did this how do they get out where are they on the network we don't we we we don't know where these files are on our network let's go find them and this is where we've used Velociraptor to do just that now you can look for the file name if you know what the file name is you can choose globs for the directory so you might say well I don't want to look over the whole system I just want you looking maybe users directories so you can glob just the user home directories you might say well I know this file was being used within the last three weeks so rather than pulling back everything let's just filter it on the metadata the file system timestamps you might say well hang on what if we don't know what the file name is what if this is a file maybe it's been disclosed on the internet and it contains confidential data but it's been renamed you can actually look inside the contents of the files as well when you do a file phone so you can use just simple yara cigs to actually look inside the contents of the files and you can use all or none of those options again as you need this turns out to be a super super powerful artifact because you can not only look for documents you can look for any piece of data you want remember we have raw access to disk so this could be logs it could be artifacts like prefetch files registry Keys MF T's whatever you want it to be and again once you run it you can just run this across a group of computers or the entire network so let's look at another example hunting for other specific forensic evidence by the way I should have mentioned at the start we've got a lot to get through so we're going to move pretty quickly so sysinternals tools we've seen examples of attackers who like using sysinternals tools there are certain apt groups that we see them all the time now you're probably familiar with the idea that when you use a system eternals tool for the first time you have to be you have to be an ethical hacker agree to the License Agreement right but when you agree to that you're like it sets a registry key and the registry key is set in the user's registry hive which is great because that tells us that the tool was run the user account under which it was run and the timestamp of that change shows us the first time it was run so this becomes a great indicator by looking at those sysinternals registry we can track the use of these across our network and again you would look for anomaly so if you see s delete or Pierce exact being used by your IT people you might think well yeah that's what they do but if you see being run on executives computers or accountants computers or other key staff that might raise some suspicions so we have an artifact for that and this is all in again you don't even need to tailor anything you just run it on your network it will go out it can be a little bit hard to see but you can see here's a dodgy user and they've run Pierce exec and they've run Esteli notice too that the results we're looking at it in velociraptor so we can actually pass certain artifacts including registries so you don't need to grab the whole registry and export it and get another viewer you just run this artifact and it will just give you a report in the on the dashboard that says here's all the EULA keys across the whole network again it's it's that simple so let's extend this even more as we said at the start as dfi our professionals we know what to look for we have fantastic resources like this and we saw when Eric and Troy presented yesterday they showed this evidence off poster that's been around for a long time if you don't have a copy of this go get one I think there's some out I always recommend if you're gonna steal this steal two copies so when you put it on your wall you can have like each side show anyway on this poster we have different groups of evidence so this is evidence of execution and this is fantastic forensic knowledge because when you're doing an investigation this is your the stepping stones that you can follow to reconstruct what programs are executed so again we've taken that knowledge and we've built an artefact and Mike you want to maybe just quickly explain how this one works yeah so the idea is that basically if you're looking for evidence of execution for instance then you know there these are all the things you want to check for and each of those things you know may be involved as prefetch files as aim cache there's different things so each of those have their own artifacts but what we want to actually do is kind of have like a collection and say you know I just want this thing and then it was gonna just collect all of these you know sub artifacts and this this slide demonstrates that artifacts can be used inside vql as well so a vql query can reuse a whole other artifact which of contains its own vql query and so you can see here that essentially all of these are basically one-liners which just call the specific artifacts but you know they basically user sees timeline recent apps app compat and and that for the user that's really invaluable because they can just click this thing and they know they've got all of these things at once yeah so this artifact is actually made up of other ones so you can see down the right there's an artifact for user assist there's one for Windows timeline there's one for recent apps etc you can get very surgical and say I only want these specific ones or you can say you know what I don't care I just want all evidence of execution and you can just use that meta artifact that we've already created evidence of execution so again encapsulating forensic knowledge into a tool that's going to do this for you this is important too because this is again showing the results now again we're not collecting artifacts and pushing them out to another tool in this instance we're actually getting these analyzing them and presenting them directly on the interface so if this was an interactive demo I'd show you but if you scroll down that screen you can see all of those different execution artifacts that we selected are all listed underneath so you can just scroll down and see them all in one spot so fantastic you know triage analysis capability let's look at another example lateral movement so there's another awesome awesome poster which you have to get to two copies off for lateral movement and it lists a great deal of information about known lateral movement techniques and these are again fantastic indicators if you're doing threat hunting if you're doing a threat assessment in an environment or if you're tracking an actual attacker this is a laundry list of things to look for so again what we're trying to do is instead of just saying to you go out and do this individually we've built artifacts that can do this for you so if we focus on this one WMI we know what to look for on the source computer and the destination and we've built an artefact that does that and it gets all of that knowledge and it just looks in those places and returns the results to you what about specific IOC s we've had situations where we're doing an investigation and there might be an apt group that we're looking at it might be another group that we might be doing an investigation on an individual compromised and as we're learning about what the attacker is doing as we're understanding their TTP's building I OCS getting all our acronyms lined up we want to be scanning across the network so as we find things on an individual box or maybe as we're getting thread Intel externally and consuming it we can put that into an artifact and just hunt out across the network so if you see for example an interesting malware executed or an interesting connection you can just jump in build a quick artifact and say show me where else on the network disappears so again building knowledge and encapsulating it in the tool and we've done this on cases where clients have said to us look we're we're hunting a particular group we've got some thread Intel we've got some thread Intel of our own we've got other repositories we'll put that all together and we'll build an artifact and we'll just push it out and that and it does our threat hunting for us here's a more operational example shadow IT this is where you get people in the organization who say well we want to set up our own so we want to set up Dropbox we want to use Google Drive and they start pumping all these confidential company documents onto that now if your vendor if you ever have investigated Dropbox you'll know that there are many places you can look so you might say well let's look for people who's accessing the Dropbox website you might see that going forward but what if they've done it in the past you might look at their internet history but then you need to dig into that but again what if they've cleared their internet history well maybe you can go to their cookies and you can see if there's any references to Dropbox in their cookies maybe you get on their computers and you look for the Dropbox folders that contain the synchronized copies of documents so there's lots of places to look and again this is a problem that's well suited to Velociraptor because you can build an artifact as we've done here just looking at Dropbox usage and it will look in the various places and it'll put all that information in that but that effort into one spot for you so here we're seeing chrome cookies here we've got Windows programs that are installed so and again this is just taking existing vql and customizing it to just look for Dropbox activity so you get all your internet history and then you just filter it on Dropbox and that's the result of your query okay so far we've been hunting we've been looking for stuff so time to level up again what about monitoring before we said anything that you even look for on one box you can look for across the network it also turns out that pretty much anything you can search for on a network you can monitor for going forward and again using the same vql the same artifacts as the starting point so Mike you want to explain maybe what an event artifact is yeah so basically when I say a query language you're probably thinking SQL you know you're on the thing it comes back with the table of results right and that's for the most part what regular bql does but we but the vacuole query is asynchronous so it doesn't actually have to ever return and we have a whole bunch of event plugins which basically just start running the query and just block and wait for something to happen an event and when an event happens that row is returned you know back to the to the query and then you can filter it and manipulate it and so forth and so these are actually the query is never-ending it just starts running and it just starts listening for the event and then just processing it and it turns out this is extremely powerful because we can push it to clients and to the endpoint and then and then basically just wait for things to happen and then respond the query can respond by itself without going back to the server or anything like it just knows what it's supposed to be doing it's kind of a response plan and so we can do that and it just keep feeding and streaming those events back to the server as they happen so what kind of things do we look for here's one case study very simple one DNS now we know that DNS is a fantastic indicator it's it's great for c2 it's great for blacklisted IP is it's great for scoping the extent of which other boxes on our network talking out to our our bad actors a lot of organizations don't log DNS unfortunately and some that do do it in a limited way so they might do it say on their perimeter that's okay but what happens if he uses at home and they're accessing their internet through home through their home while it's you're not going to see that well they might do it on an internal DNS server which again is okay if someone's in your net but what if you've got your IT people who decide no no I don't want to use the company's DNS I'm gonna use Google's DNS because it's pasta so what we do is we actually connect DNS on the endpoint and that covers those bases because if the user changes their DNS we're still gonna see their queries if a user is outside of your network perimeter we're still gonna see their DNS queries and this is a really simple example of an event artifact every time we get these lookups these DNS lookups we're capturing them we're feeding them back to the server and again we can monitor for them if we're doing an investigation we've identified a c2 channel we can extend this monitoring to say look at the DNS and every time you see this alert me send me an email do something about it so this becomes a great monitoring tool here's another interesting one it's kind of not so much on the incident response but a huge issue for organizations USB devices now USBs are a constant threat we can tell a lot about USBs through digital forensic analysis when they're connected make model serial number when they're disconnected if someone opens files from them we see a lot of artifacts about that but we still have blind spots if someone connects a USB you can't see what's on it unless they start opening files and another big problem we've got in Windows is you can't see if they copy files to it and then just pull it out so again Velociraptor can cover these blind spots with some artifacts and we're using monitoring artifacts again to monitor for USB key insertion so what do we want to do in USB keys get inserted simplest thing list everything on them so imagine this imagine from you can go to your boss or your risk people and say hey from now on every time someone plugs a USB in we're gonna get a list of everything that's on it pretty cool you can then start looking for that as well so this is another artifact that looks at inside the documents and looks for keywords not the file names inside the documents on the USB now caveat caveat privacy issues think about it but the technical capability is there okay we're not saying go out and do this straight away because depending on where you're from there might be restrictions on whether you are allowed to scan personal devices that are connected to computers but we can do it functionally you'll notice also we're not just looking at keywords here this is a particular artifact that looks at office keywords why offers keywords because office documents are compressed so you can't just use a standard yaar you have to do some decompression first so we've built that capability in here's another interesting one office macros again we take our initial artifact which is detect USB insertion and we've written a custom artifact which says look for office documents and then look for evidence of macros inside them looking for malicious documents around the network all right server event artifacts now we've talked about clients event artifacts so monitoring things on the client what about on the server side and again Mike I'll actually I'll do the intro for this one then I move on so one of the other things that we are gathering with Velociraptor is program execution as programs are being run on the endpoints the the information about those is being fed back to the server and we get the full command line execution now you'll know that full command line particularly for something like PowerShell is also a blind-spot that we've had for a while in more recent versions of PowerShell and Windows we are getting fantastic logging capability but often you still need to enable that and we know that there are also for example downgrade attacks where attackers can manipulate their use of PowerShell to try to overcome some of that logging so again we can use Velociraptor as a counter to that we know that PowerShell gets encoded we know we often see base64 this can be it's very easy to spot on the endpoint if you've got the logging enabled if you capture the evidence you can decode that fairly easily but doing that at scale particularly on a network that's not configured to properly secure PowerShell yet that's a difficult thing so what we're doing with Velociraptor is as we're seeing PowerShell being executed we've got an artifact that's monitoring that server side and it's looking just as it's a simple regex that looks for references to the encoding switches in the PowerShell command and when it sees them it does a decode so here's an exam of a suspicious end point which is running an encoded PowerShell this has been automatically sent to the server the Velociraptor server has seen it it's detected the PowerShell encoding and it's done a decode for us this is showing you the navicular behind it and this is the decoded power this might be familiar this particular PowerShell attack to some people this might be this might be an apt that some of you have investigated before I'll just leave it at that but you can see how it works and this is all automated and and what happens is that this result just is stored on the server in essentially a distilled form so essentially you just have the one file that just has decoded PowerShell so all the other process execution craft is not in that particular file I mean it's an in different file but you've got these highly distilled high value events that you can just look at them and you can incorporate this into your security monitoring so every morning you can come in after you look at all of the list of documents on people's USB devices you can look at all of your list of encoded PowerShell that ran yesterday right extreme visibility very surgical very specific very very efficient and again encapsulating a lot of that friends acknowledge into the tool so you don't have to build this all yourself or figure out how to do it from scratch all right so we're hunting for things we're looking for things we're monitoring actively what else can we do the next logical step to this is well we see something bad happening can we respond in some way and even more can we automate it so in addition to vql there's another cool thing that we have in Velociraptor which is an API so a lot of the functionality that we've showed you through vql if you want to customize it through API integration you can do that too so here's an interesting scenario that we that we that we use here we're looking for a particular kind of maybe persistence or lateral movement misuse of Windows services and what we're going to be saying is when a Windows service gets created we want to grab the executable or the DLL that it's pointing to and we want to do an automatic sandbox detonation so that we can see straightaway what is this new service is this something we need to be concerned about and again I'll hand over to Mike to sort of walk the steps of hathor so think about what we want to do here we have some idea of you know what we actually want to do and we and this diagram sort of shows like the overall plan of what we want to do when a new service is installed you know that an event log is written into the event log and you know usually we can do event on forwarding and just see that event you know remotely but because we have in velocerator the world eruptive client actually running on the endpoint and so we can write an event monitoring query on the endpoint to the it which can also see that event being written to the log files just sit there and watch the log file locally and then we can actually do something with that event locally so you know we don't have to go to the server and then the server has to act or anything this is autonomous and so we could you know in this case the plan is to to watch for that event grab the binary that is actually installed and upload that to the server and then on the server side what we're gonna do is we're gonna write a Python program that watches for the upload and submits it to draw sandbox in in this case so that we could get some kind of triaging going on so this is how it looks we have a the service detection artifact this is looking for new services and when it sees them it uploads the information to the server this is the python script that we haven't liked you want to walk through this is basically the the script that does all of it the first function is just the Joe sandbox API to submit the the sample and then the the second function the first thing we do is connect to the API and essentially just copy-paste those lines it's a pretty template thing and the Velociraptor API is actually very simple all you only does is it accepts a vql query like anything else it's just like connecting to a database and running a query but using that we can actually do pretty much anything we want because the vql is so powerful and in this case the vql query says watch the monitoring log for system upload completion so any completed uploads that were pushed to the system from all clients at the same time and every time any file is uploaded that event will fire and when that fires then we get a row and so we can take that row and extract the the file name and then and then upload it using the Joe sandbox API and here's how it looks so the attackers on the Box they create a new service just with the SC command they call it what is it fake great service name and they point it at a malicious executable Velociraptor will pick this up on the client through the client monitoring artifact it'll grab the executable it'll upload it to the server the Python script which is running and monitoring those uploads will grab the executable it'll run its code submit it to the online sandbox and we'll get a report back so you'll get a report in your inbox that says here's a velociraptor initiated sandbox report for a new service that was created that's it it's all automated all right so let's take it up another level what about responding what about actually getting in the way of bad things that are happening and trying to stop them so this is the next logical step and this is the last kind of level up that we do I'm conscious of time so we're gonna wrap up very quickly simple example Pierce exact we've seen Pierce exact remoting we know how bad it is sophisticated attackers may or may not use it but it's still something bad that you want to detect and you want to respond to so now we know that we can monitor for new services we can monitor for executables why don't we actually block them at the same time and again Mike you might want to go through it's a bit hard to see on the slide but you can go through those one slide so this this particular artifact detects Pierce exec securin and as you know the the recent versions of PS exec have a switch to be able to rename the service so the old rules that we used to do just look for PS exec SRV it doesn't really work anymore because they can just call it whatever and so you can't see that in process execution logs so what we have to do is we have to go this step further and actually try and look at the binary and try and determine whether that is really the PS exec service that is installed and this is what this query does is it watches for service installations and then for just like before where we uploaded the binary it actually runs the arrow rule against that binary locally on the endpoint itself to try and look for that yara you're a signature and when it determines that yeah you know that that binary I mean it has the PS exec string it's probably the PS exec binary then it triggers and then what we do is we say okay so the first rule was about detection so we might want to know about it but now we're gonna go level up and actually try and and do something about it like maybe in our environment piece exactly is not really allowed so we can take that artifact that does the detection and then we can use it in another artifact that will actually actively kill that process when it starts up and so you can see that it essentially what we do is we we look for all events that come from the first the first detection artifact and then we just toss kill those right so in this this example and you can get more creative with this as well so if you've got a particular malware tool that you want to find you can use your ax to detect that so you know paste exec is a simple example but you know if you think back to the old pyramid of pain that attackers have there are certain things that are very easy for them to change like the names of their malware they can recompile they can do other things but there are there are other elements that are a lot harder so if someone is using something like a Metasploit or a cobalt strike beacon or something that requires some sort of code if you have good yara cigs for that you can embed those and it makes it a lot harder for an attacker to use those kinds of so mike apparently they tell me knows a couple of things about memory that's one of the other things you could do instead of killing the PS exec process you could actually say hey what's happening with PS exec let's get all the trial processes which is what we're doing here before we kill them let's just get a process memory dump and we get all the process memory and then we drop the process and then when you get that memory dump you can do your analysis and say what was this thing doing what was and not the benefit of doing that process dump rather than just sort of full memory dump is it overcomes a lot of those issues with paging and compression and those things that we were sort of talking about yesterday because the memory is all there it's the you're getting Windows to basically collect it for you and give it to you and then kill the process so this becomes a very powerful response capability as well and this is what it looks like from the attackers perspective they run Pierce exec they try to pop a shell and it kind of fires up and then it dies right now there's a really important thing here there is a race condition going on we are not in line in the execution process so we cannot block the Pierce exec because of the way where Velociraptor sits in the chain but we can detect it there are definitely situations where if the malware executes really quickly and comes and goes really fast we're not going to catch it but if you've got something that's more persistent like an implant or you know back door or something else you will you will pick that up so it depends on the speed of it so that's all the scenarios so we've gone through a lot right and there's lots of options there's lots of capability we've been using this on jobs I said for the last year or more and it's been fantastic in lots of different cases and every time we have a new job and then you need for it we kind of think how can I do this and we'll build an artefact and we'll use it so the question I put to you is what would you use it for how could you use and this is part of what we want to do in releasing it open-source is we really want to get people's feedback and ideas we want because we've already had some fantastic suggestions from people in the community that we're not even thought of yet but because that baseline capability is there we can extend it to doing those things so we're going to wrap up really really quickly a huge huge point to make here this is not a perfectly polished beautiful commercial tool right it can be clunky at times it will have bugs in it it will do things that you don't expect it to do right it is a work in progress but having said that this is something that we're committing a lot of time and effort to so we've act we actually do have a company established that we're using to develop this tool we've been using it this is Mike's full-time job and a lot of my time at the moment and people in our team who've been acting lis using it and developing it and we're working with people in the community so this is not something that's a hobby that we're doing on the side it's very much front and center for us and we're gonna keep working on it we already have a road map that has some interesting stuff in it and you can read some of those things there we want to expand it we want to make it better we want to get the bugs out of it we want to get feedback from people and we want to make it even more robust even more feature-rich and even even more useful so how can you get involved go to the website we've just popped up a new website in the last couple of days we just pushed out a new release as well that has all of the capabilities that we showed you today it's on github you can download it start using it after you download the latest release please take a minute to read the friendly manual ok your time spent reading it will pay off in spades when it comes to actually understanding how the tool works and how you can use again you don't have to be a forensic guru you don't have to be a vql guru you don't have to be a programmer you can just do a lot by just pointing and clicking and then if you get comfortable with it you can start playing around with parameters you can start doing your customizations as much or as little as you want to and please contribute back alright we already have some great people in the community who are contributing back ideas and artifacts and functionality we have some great stickers apparently you can't have software these days without stickers so if you want some stickers come and hit us up we have this really awesome gold shiny sticker but we reserve that for people that actually contribute so submit something and we'll submit something and we'll send you on that's it thank you [Applause] [Music] you

Channel: SANS Digital Forensics and Incident Response

Views: 6,566

Rating: 5 out of 5

Keywords: digital forensics, incident response, threat hunting, cyber threat intelligence, dfir training, dfir, learn digital forensics, learn computer forensics, forensic data, forensics artifacts, free digital forensics, free computer forensics, DFIR Summit

Id: u7KBwgHIZ3U

Length: 52min 23sec (3143 seconds)

Published: Mon Oct 07 2019